Testing Cyber-Physical Systems Using NLP Models

Sammanfattning

This thesis investigated the effectiveness of natural language processing models in automating test generation for Flutter applications that use Bluetooth com- munication, an aspect of cyber-physical systems that remains underexplored. The study evaluated four open-source natural language processing-based code generation models: StarCoder, GPT-NeoX, CodeGen, and CodeT5, focusing on their ability to generate end-to-end and integration tests. A structured experimental methodology was used to assess each model’s output across three levels of prompt complexity. The results showed that while models such as StarCoder demonstrate some logical structure, none of the models produced fully functional tests without manual intervention. Edge case handling, such as unstable connections and device compatibility, proved particularly challenging. The findings highlight the current limitations of small-scale natural language processing models in cyber-physical system testing scenarios and emphasize the need for more advanced models, improved prompt strategies, and domain-specific fine-tuning to close the performance gap between human and machine-generated tests.