Simulation Testing in Cyber-Physical Systems : A Case Study with Autonomous Underwater Vehicles

Sammanfattning

Autonomous underwater vehicles (AUVs) are complex cyber-physical systems (CPS) that operate in challenging underwater environments, where real-time monitoring is impractical and hardware failures are costly. While simulation-based end-to-end testing can reduce reliance on expensive field trials, its adoption in CPS development remains limited despite recognized benefits. This study investigates current simulation practices in AUV testing, identifies barriers to broader adoption, and proposes strategies for enhancement. A single-case study was conducted with one AUV development team. Data collection comprised semi-structured interviews with three engineers and qualitative analysis of the organization's software repository using pull request (PR) data. Interview responses were analyzed to identify recurring themes and patterns, and repository data were examined using descriptive and inferential statistics. Findings were triangulated across both data sources. Simulation is employed frequently but inconsistently, stabilizing at approximately 50\% of pull requests, with usage ranging from 85\% for sensor drivers to 40\% for control systems. Pull requests involving simulation tend to be broader in scope, containing approximately twice as many changes as those that do not involve simulation. The primary barriers to adoption in this case are organizational, including competing priorities and limited resources. The reality gap is not uniform but instead varies across different subsystems. The findings suggest that effective simulation adoption does not require perfect fidelity; instead, selective and pragmatic application should be informed by subsystem characteristics. Organizational prioritization remains the principal constraint. A phased adoption strategy is recommended: (1) in the short term, accept approximate simulation fidelity to achieve immediate testing benefits; (2) in the medium term, systematically integrate simulation into continuous integration and continuous deployment (CI/CD) pipelines to enable routine and automated testing; and (3) in the long term, extend simulation to additional applications such as customer support or training, thereby improving reliability and user confidence.