AI Agent for Asynchronous Interaction in Telecom Configuration Tool

Sammanfattning

Configuration generation failures impose a substantial manual burden on test engineers: when a required parameter is absent or incorrectly specified in the network inventory system (NIS), the configuration generation tool (CGT) returns a structured error report instead of a deployable archive, and the node cannot go live until an engineer investigates, corrects the NI, and re-runs CGT. At scale, this repetitive resolution process represents a significant and largely avoidable engineering cost. This thesis presents JOTARO, a multi-agent system built using the design science research methodology to autonomously resolve configuration generation failures within an existing test workflow. JOTARO decomposes the resolution process into four specialised agents: (1) an orchestrator that routes each session by error code, (2) an investigator that diagnoses root causes and proposes parameter fixes, (3) a Validator that confirms each proposed fix by re-running the CGT as an objective oracle, and (4) an executor that writes corrected parameters to the NIS only after engineer approval. Because engineers cannot be expected to monitor the system continuously and the workflow must not block immediate human input, JOTARO supports asynchronous collaboration: engineers can review and approve proposed fixes at their own pace, enabled by persistent session state. Evaluated against a stratified sample of 1,170 production failure records, JOTARO achieves an automatic error resolution rate of 41.4%. Including escalations in which the agent correctly diagnoses a blocker it cannot fix autonomously, predominantly a CGT source-code defect, the overall correct-handling rate reaches 82.1%. Pass@k analysis reveals that running three independent attempts per sample raises the automatic rate from 41.4% to 50.2%, raises the automatic resolution rate from 41.4% to 50.2%, suggesting meaningful variance across independent attempts. Among the evaluated error codes with a fully deterministic retrieval-then-write fix path, JOTARO achieved a 100% resolution rate. Automatically resolved sessions have a median resolution time of 108 seconds, compared to a manual baseline of 30 to 60 minutes per case (expert estimate), indicating a roughly order-of-magnitude savings. The results demonstrate that combining deterministic validation, persistent state management, and human-in-the-loop oversight enables safe integration of LLM agents into asynchronous engineering workflows and reduces resolution time.