Assessing the Readability of AI Generated Code: A Comparisonof Two LLMs

Sammanfattning

As Large Language Models (LLMs) become increasingly integrated into software engineering workflows, it is important to understand their abilities and limitations when it comes to generating code. This thesis investigates the abilities of two LLMs, GPT-3.5-turbo and WizardCoder-15B-V1.0, to generate readable Python code. HumanEval was used to generate test samples and for functionality testing. The code generated by both models was analysed using the Radon and Flake8 code analysis tools and compared to human written solutions, on readability attributes established through workshops with four experienced developers. The findings show that LLMs have some ability to generate readable Python code for short HumanEval tasks. Both models generate code that is comparable to human written code in complexity and size while demonstrating higher alignment with best practice for the language. The models falter when it comes to the practice of commenting code, especially without being prompted to document. This study offers insight into code readability preferences of real world developers and proposes a practical methodology for evaluating readability. Furthermore, the results shed light on the strengths and weaknesses of current LLMs in generating readable code.