Bridging the Gap Between ClickOps and DevOps : Exploring Retrieval-Augmented Generation for Infrastructure as Code Generation

Sammanfattning

Junior software developers transitioning from manual ClickOps workflows to Infrastructure as Code (IaC) face a steep learning curve and elevated risks of misconfigurations and security vulnerabilities. Recent research has explored the use of large language models (LLMs) and intent-based systems to automate infrastructure provisioning, with promising results in intent interpretation and orchestration. However, prior work primarily addresses automation performance and system architecture, with limited attention to how these tools support junior developers or mitigate risks in DevOps workflows. This bachelor thesis in Computer Science builds on this prior work by designing and evaluating a Retrieval-Augmented Generation (RAG)-based artifact for IaC generation, aimed at supporting junior developers. The study empirically compares misconfigurations and security outcomes between RAG-generated and baseline raw LLM-generated Infrastructure as Code configurations using identical prompts. The results suggest that integrating retrieval-augmented generation with supplementary review context and an LLM-based self-evaluation mechanism improves the initial quality of generated IaC relative to a raw LLM baseline. The findings also reveal clear model-dependent differences: Claude-based configurations generally required less refinement effort and achieved lower task difficulty ratings than OpenAI-based configurations. Notably, syntactic correctness and security scores alone were not reliable indicators of deployment readiness, as configurations that passed static validation still failed during execution. Overall, the study suggests that RAG combined with self-evaluation mechanisms can improve IaC generation quality and may offer pedagogical value for junior developers, but they do not eliminate the need for human oversight. The findings reinforce that LLM-based IaC tools should be viewed as assistive systems rather than fully reliable automation solutions.