Hybrid Retrieval-Augmented Generation for Automated ECU Test Case Generation : A Framework For Automated Verification of ECUs Using Large Language Models

Sammanfattning

Manual test case generation is a major bottleneck in the automotive industry, consuming significant engineering resources and slowing software verification cycles. This thesis addresses this challenge by developing a Hybrid Retrieval- Augmented Generation (RAG) pipeline that leverages Large Language Models (LLMs) to automate test case creation for Electronic Control Units (ECUs) within a custom testing framework. The pipeline integrates vector search, BM25 keyword retrieval, and Reciprocal Rank Fusion (RRF) to improve retrieval relevance and code generation accuracy. Three configurations were evaluated: baseline vector retrieval, Hybrid RAG, and Hybrid RAG with RRF across 100 test cases. The results show that the hybrid methods improve generation accuracy and functional correctness by up to 15%, while reducing test creation time from 15 minutes to approximately 15 seconds, with minimal runtime overhead. However, RRF’s benefits varied depending on query types, indicating that retrieval fusion effectiveness is contextdependent. Although further refinement is needed to enhance retrieval precision, contextual grounding, and consistency across diverse testing scenarios, the proposed RAG pipeline demonstrates overall that LLM-assisted automation can substantially accelerate ECU verification while maintaining code quality.