Automation of Test Case Specifications for High Performance ECU using NLP techniques

Sammanfattning

Background: Natural Language Processing (NLP) is a field within artificial intelligence (AI) that focuses on enabling computers to understand, interpret, and generate human language. It includes methods such as tokenization, part-of-speech tagging, parsing, named entity recognition, semantic analysis, machine translation, and text generation. NLP allows computers to learn from text, interact with people more effectively, and automate language-related tasks, improving human-computer interaction. Objectives: To analyze the High Performance ECU feature elements and convert them into comprehensive test case specifications. Then, evaluate the accuracy and efficiency of the generated test case specifications. Methods:The study focuses on automating the generation of test case specifications from feature element documents stored in Polarion. It evaluates around 400 feature elements using rule-based and Named Entity Recognition (NER) natural language processing techniques, comparing them against manual methods. Results:The rule-based approach achieves 95% accuracy for single-signal feature elements. SVM outperformed other algorithms in Named Entity Recognition and the rule-based approach dominated the NER method as well as manual methods. Conclusions: The Rule-Based Method and NER methods were more efficient and accurate than the manual method for generating test case specifications, demonstrating the potential of NLP-based automation to improve software testing. The Rule-Based Method out-performed both the NER and manual methods, particularly for less complex requirements. Further refinement of the NER approach is needed to match the performance of the Rule-Based Method, especially for more complex feature elements.