LLM-Assisted TARA - Review Automating Verification and Quality Assurance in Automotive Cybersecurity Engineering

Sammanfattning

Context: Threat Analysis and Risk Assessment (TARA) is a central activity inautomotive cybersecurity engineering. TARA artefacts are used to document cybersecurity risks, threat scenarios, risk treatment decisions, cybersecurity goals, andsupporting evidence for compliance-oriented engineering activities. In industrial environments, these artefacts are often updated iteratively as system designs evolveand therefore require repeated review to ensure completeness, consistency, traceability, and documentation quality.Problem: Reviewing completed TARA artefacts is a time-consuming and expertintensive task. Manual sanity checking requires cybersecurity engineers to inspectstructured artefacts, verify relationships between elements, identify missing or inconsistent information, and assess whether the documentation is suitable for furtherengineering and compliance activities. This becomes challenging in large-scale development environments where expert availability is limited and TARA informationmay be distributed across multiple teams. At the same time, fully relying on LargeLanguage Models (LLMs) for review decisions introduces risks related to hallucination, inconsistent outputs, and limited auditability.Method: This thesis follows a Design Science Research approach and presents ahybrid LLM-assisted TARA review prototype. The prototype processes structuredXSAM/XML artefacts and combines deterministic checklist-based validation withan AI/RAG-based explanation layer. The deterministic component performs repeatable rule-based checks for machine-checkable aspects such as completeness andtraceability. The AI/RAG component uses retrieved domain knowledge, parsedXML evidence, and deterministic findings to support semantic review tasks, including consistency-oriented observations, compliance-oriented explanation, and reviewer support. The system was evaluated through an iterative expert-driven process using representative XSAM/TARA artefacts and predefined sanity-check rules.Result: The evaluation indicates that the hybrid approach can support TARAsanity checking by producing structured review reports, improving consistency ofrule-based validation, and providing grounded explanations for expert reviewers.Across refinement stages, agreement with expert assessment increased from 51.9%to 74.1%, while verdict accuracy improved from 70.4% to 88.9%. The results showthat deterministic validation is suitable for repeatable structural checks, while theivLLM/RAG layer is useful for explanation and interpretation. However, the studyalso shows that expert judgement remains necessary, especially for review criteriarequiring project-specific context, architectural information, or evidence not represented in the analysed artefacts. Overall, the thesis demonstrates the feasibilityof using a hybrid deterministic and LLM-assisted approach as a decision-supportmechanism for reviewing completed TARA artefacts in automotive cybersecurityengineering.