Leveraging AI for Automated Requirement to Test Alignment in Automotive Embedded Systems - A Design Science Research in the Automotive Industry

Sammanfattning

The rapid increase in the complexity of automotive software has made the manualverification of software traceability labour-intensive. Consequently, practitionerscan encounter “alignment gaps”, where a formal traceability link exists between arequirement and a test case, but the test case fails to exercise the requirement’sintended functionality. To address this challenge, this thesis develops a diagnosticartificial intelligence (AI) artefact. The artefact is aimed to automate the assessmentof requirement-to-test alignment within the automotive embedded systems domain.The study employs a three-cycle Design Science Research methodology. In thefirst cycle, a literature review and practitioner interviews were conducted to identifydesign attributes for the proposed artefact. Four design attributes were determined:semantic similarity, contextual information, structural consistency, and hierarchicaldependencies. During the second cycle, a system was developed utilizing Large Language Models (LLMs) integrated with a Retrieval-Augmented Generation (RAG)architecture.In the final cycle, the artefact was evaluated through a mutant injection analysisto measure its fault detection capabilities, alongside a case study involving industryprofessionals to assess its practical utility. The mutant injection analysis demonstrated that the RAG architecture improved its analytical consistency and its ability to detect injected logical defects. Furthermore, the case study, which utilizedthe NASA-TLX questionnaire and post-task interviews, indicated that the artefactreduced the general workload associated with evaluating test case alignment. However, due to the small sample size and sampling limitations, these findings shouldbe interpreted as suggestive. Ultimately, this thesis concludes that combining RAGarchitectures with LLMs represents a promising proof-of-concept for transparentdecision support in maintaining requirement-to-test alignment in safety-critical automotive environments.