Use of Generative AI in Software Engineering Examination Contexts

Sammanfattning

Context: Generative artificial intelligence (GenAI), particularly Large Language Models (LLMs),is increasingly used by students in higher education. In software engineering education, studentsare assessed through a wide range of examination types, including written exams, laboratory work,individual assignments, projects, and seminars. Understanding how students use LLMs acrossthese assessment formats is important, as such use may affect learning processes, independentwork, and academic integrity.Goal: The research goal of this thesis is to investigate how software engineering students useLLMs in relation to different examination types within the Swedish higher education context, fromthe point of view of students and teaching staff.Method: We conducted a mixed-method study combining a systematic mapping study and anempirical survey. In the mapping study, we screened an initial set of 769 studies and selected 51primary studies, which we analysed through a data extraction and synthesis process. In parallel,we designed and distributed surveys to students and teaching staff, with the survey instrumentinformed by the emerging findings of the mapping study and the relevant literature on LLM use insoftware engineering education.Findings: We identify recurring themes in the primary studies, including performance andproductivity, overreliance, academic integrity, feedback, and collaboration. We also find that the useof LLMs differs across examination types, being more prevalent in practical and take-home forms ofassessment (e.g., assignments, labs, and projects) than in traditional written exams. While LLMscan support learning and efficiency, they also raise concerns related to reduced independent workand academic integrity.Conclusions: The results provide insights into how LLMs are used across different examinationtypes in software engineering education. These findings can support the adaptation ofassessment formats and teaching practices in response to AI-supported student work.