AI-ASSISTED GENERATION OF REQUIREMENT-BASED TEST CASES FOR EMBEDDED ROBOT CONTROLLERS

Sammanfattning

As embedded robot controllers grow in complexity, the creation and maintenance of verification test specifications still remain highly manual and time-consuming processes. While Large Language Models (LLMs) offer significant potential for document automation, their application in industrial engineering is constrained by strict domain-specific formatting standards and data privacy requirements, which can often prohibit the use of external cloud-based services. This thesis investigates the feasibility of automating the generation of domain-specific test cases using a locally deployed LLM, adapted via Parameter-Efficient Fine-Tuning (PEFT) methodologies. To overcome the structural degradation of legacy documents, a vision-based extraction pipeline was developed to process requirements into structured data. A 4 billion parameter model was then adapted via supervised fine-tuning (QLoRA) and benchmarked against both its zero-shot base variant and a state-of-the-art baseline (Claude Opus 4.6) using few-shot prompt engineering. The outputs were evaluated quantitatively using automated metrics (ROUGE, BERTScore) and qualitatively through a blind usability survey conducted by eight domain experts. The results demonstrate that embedding domain knowledge directly into the model's weights via fine-tuning outperforms contextual prompt engineering in enforcing strict structural adherence. The statistical correlation analysis revealed that automated textual metrics only moderately reflect practical usability (BERTScore Pearson r = 0.505), proving that reference-based mathematical evaluation cannot replace human-in-the-loop validation in safety critical domains. The study concludes that a small, locally hosted LLM can produce domain-adapted outputs that follow organizational formatting, while maintaining data security and cost-efficiency. When compared to a state-of-the-art commercial model, the fine-tuned model achieved comparable expert ratings, demonstrating that massive scale is not strictly necessary for this task. However, the quality of the generated test cases remains limited by the quality of the training data, highlighting that improvements in output quality will require changes in how requirements are authored.