AI in Code Production & Review : A Comparative Analysis of LLM-based Tools versus Human Expertise

Sammanfattning

This thesis investigates the reliability and quality of AI-generated code compared to human expertise in a professional software development context, with findings relevant to developers, software engineering students, and others who work with code production and review in practical settings. Specifically, it compares a Large Language Model (Gemini 3 Pro) against human developers in code production, and an AI agent (Claude CLI) against a static analysis tool (SonarQube) in code review. The study applies a quantitative experimental methodology across six Java tasks, using 90 human-authored files as a baseline and 60 AI-generated files for comparison. The code review component uses Defects4J as its ground-truth source. Defects4J is a dataset of real, confirmed bugs from Java projects together with their developer fixes, making it suitable for testing whether a review tool finds the correct defect. The primary experiment covers 40 verified defects, and a supplementary quality audit covers 497 issues. These results are complemented by a survey of 16 developers and an analysis of an existing organizational AI policy. The findings show that AI-generated code is statistically equivalent to human-written code on Cyclomatic Complexity (d= +0.01) and the Maintainability Index (d = −0.14), and significantly cleaner on Code Smell Density (d = −1.16). No security vulnerabilities were detected in either group, but a latent logical defect was identified in the most complex task, where the model omitted edge cases not covered by the standard test suite. In code review, Claude CLI detected 90% of the semantic defects in Defects4J compared to SonarQube’s 5%, while SonarQube reached 97.8% compared to Claude CLI’s 22.1% on a broader set of pattern-based quality issues. The tools are therefore complementary rather than competing. The policy analysis identified a three-layer constraint model governing AI use in client projects: information classification, contractual consent, and limited awareness of extraterritorial frameworks, notably the US Cloud Act. The conclusion is that AI tools can match junior human code quality on structural metrics but require active human review to catch implicit requirements and edge cases. A layered review pipeline, with static analysis handling deterministic code hygiene and an AI reviewer handling semantic defects, is recommended as the most effective architecture for professional development.