AI-Assisted Topic Modeling for Academic Research : Evaluating AI Adoption through the Lens of Interpretability, Trust, and Legitimacy

Sammanfattning

The accelerating growth of academic publishing has created a research environment characterized by both abundance and complexity, making it increasingly difficult for researchers to keep pace with emerging trends and developments. Artificial Intelligence (AI) offers promising approaches for address this challenge, yet its adoption in academia depends not only on technical performance but also on interpretability, trust, and legitimacy. This thesis investigates how an AI-assisted tool can support topic modeling and trend analysis of master’s theses in a way that enhances interpretability, fosters trust, and ensures contextual legitimacy. Using a Design Science Research (DSR) approach, a lightweight prototype was developed that combines embedding-based clustering, automated topic labeling, interactive visualizations, and report generation. The tool was evaluated through a workshop with students and faculty at KTH, where participants manually classified thesis abstracts before using the AIassisted system. Their experiences and feedback were collected through a structured survey. The findings show that the tool improved usability and interpretability while enabling efficient identification of key themes and trends across the dataset. Participants valued its usability and time-saving potential, although trust in the results remained moderate due to limited transparency in how clusters and labels were generated. Interpreting the results through Legitimacy Theory indicates that pragmatic legitimacy was achieved, while moral and cognitive legitimacy were only partially established. The thesis contributes practically by presenting a human-centered AI tool for academic trend detection and theoretically by applying Legitimacy Theory to explain how interpretability, transparency, and trust influence the acceptance of AI-assisted systems in academic research.