A Contract-Based Framework for GUI Regression Testing of IP Desk Phones

Sammanfattning

Introduction: GUI-based regression testing of IP desk phones is difficult to automate because the same user scenario may require different interaction steps on different phone models. This makes test scripts model-specific and difficult to reuse across devices. The problem is especially relevant in UCaaS environments, where several endpoint models must support the same call-flow functionalities despite differences in screen layout, buttons, firmware, and device behavior. Research Question: How can a contract-based framework be designed and implemented to run reusable regression-test scenarios across multiple IP desk phone models? Method: A framework was developed using Design Science Research and evaluated experimentally in collaboration with Dstny Communication Solutions SWE AB. The evaluation used three IP desk phone models: Poly E100, Poly E350, and Yealink T85. Four representative call-flow scenarios were tested: Call with Line-key, Hold and Resume Call, Blind Call-Transfer, and Conference Call. The framework uses scenario files, contracts, device context, LLM-supported decision-making, and prompt design to run test scenarios across phone models. Results: The framework executed the selected scenarios across the evaluated phone models. The results show that simpler scenarios were reliable in execution, while more complex scenarios exposed limitations in LLM-based reasoning, but also benefits from its flexibility. Moreover, contracts supported repeated execution reliably and required fewer LLM calls, shorter execution time, and lower reported cost than first-time scenario execution. Discussion: The results suggest that interaction contracts can act as a middle layer between generic test scenarios and device-specific execution. The framework uses LLM-supported interpretation where flexibility is needed, while stored contract implementations reduce repeated LLM usage and support repeatable deterministic replay. However, the evaluation was limited to three devices, four scenarios, and mostly nominal execution paths. Future work should improve recovering from failure, repairing contracts, and long-term evaluation across more devices, scenarios, and firmware versions.