The Impact of AI-generated Code on Web Development: A Comparative Study of ChatGPT and GitHub Copilot

Sammanfattning

Background. Machine learning and artificial intelligence are advancing faster than ever, code generation is becoming a hot topic and is starting to gain traction in the industry. This creates the question, is it possible to create a complete website from scratch using only code generated by AI? Objectives. To determine whether it is possible to create complete websites from start to finish with the code-generating tools. The tools in question are OpenAI’s ChatGPT and GitHub’s Copilot. Methods. A design-based research was conducted where two tools were evaluated for the task of recreating a wireframe as closely as possible in terms of efficiency, accuracy, maintainability, and ease of use. The code was then analyzedboth manually with a code review and using the tools SonarQube, ESLint, and Pylint. Results. The experiment resulted in that both tools delivered code that was similar in quality, both tools managed to create the websites according to wireframe with minor styling differences. We found that it is easier to create a website from scratch using OpenAI's ChatGPT than it is with GitHub's Copilot even though it uses OpenAI's Codex model which focuses on code generation. Conclusion. Code-generating AI is not advanced enough to create systems from scratch in a time-efficient way without introducing bugs and security risks.