AI-Assisted Remediation in CI/CD : An Exploratory Study of a Human-in-the-Loop Code Repair Prototype.

Sammanfattning

Background. CI/CD pipelines underpin modern software delivery. When one fails, engineers must interrupt their work, parse lengthy build logs, isolate the root cause,and wait for a re run a cycle that lengthens recovery time and diverts effort fromfeature development. A developer survey conducted for this thesis found that 80.8%of respondents identified log reading as their primary frustration. Large Language Models (LLMs) can parse those logs and propose fixes autonomously, yet most teamslack the safety controls required to trust automated patches near production code. Objectives. This thesis sets out to design and evaluate a multi-agent AI system that can detect, diagnose and fix routine CI/CD failures without dropping the safeguards developers expect. We are equally interested in how to generate the fix and in how to keep the system from doing something unsafe with it. Methods. We follow a Design Science Research (DSR) approach. The artifactis a six-agent Python system, containerised with Docker Compose. Agents talk toeach other over the Model Context Protocol (MCP) and use NVIDIA NIM as the LLM backend. A four-stage log cleaner cuts log volume before any model sees it. Errors are first matched against eleven regex patterns and only sent to an LLM if no pattern fits. A traffic light scoring system decides how a fix gets reviewed. Bandit, Pylint and a secret scanner run on every patch. We tested the prototype against 15 seeded failures, conducted user acceptance testing with 10 students and 10 industry practitioners, ran two expert interviews, and collected 53 survey responses. Results. On a controlled proof-of-concept set of 15 seeded single defect Python fail-ures, the prototype repaired the large majority of cases, and most of those repairs also cleared the automated security and code-quality gates without human intervention. These figures describe the behaviour of the prototype on this specific, purpose-builtset and should be read as proof-of-concept evidence rather than a generalisable per-formance benchmark; the detailed measurements and their limitations are reportedin the results and discussion chapters. The log-cleaning stage substantially reduced build-log volume before any model call. Across the interviews and survey, three trustbarriers recurred: doubt about whether the LLM is actually right, lack of visible rea-soning, and fear of security regressions. A clear majority of survey respondents said the AI should never merge autonomously, always requiring human approval, and most said that being shown the diff before review would increase their trust. Conclusions. Within the scope of this exploratory study, the evidence suggests that the main factor for running this kind of system safely appears to be the layer of controls placed around the LLM: a confidence-based traffic light, two static-analysis gates, a secret scanner, permanent human approval (auto-merge is hard disabled), acircuit breaker, deduplication, loop protection, and an immutable audit trail. In this design the system proposes a candidate fix, while the authority to apply it is retained by a human reviewer. The most prominent reported barrier to adoption was not the technology but accountability something the architecture can help address by being transparent about what the agent did and why. These conclusions are drawn from a limited proof of concept on seeded Python failures and are offered as a candidate design rather than a validated, generalisable result.