LLMs and Design Patterns : What are the Issues?

Sammanfattning

Large language models (LLMs) are increasingly being used for code writing. There has been a lot of research into LLM-generated code, but not as much research into specifically their ability to use design patterns. This study aims to investigate this by having LLMs refactor Java codebases with one of eight design patterns: Strategy Pattern, Observer Pattern, Composite Pattern, Factory Method Pattern, Singleton Pattern, Mediator Pattern, Builder Pattern and Bridge Pattern. The LLMs used were ChatGPT 5.4 Thinking, Opus 4.6 Extended (Claude) and Gemini Pro in canvas mode. These refactored codebases were then evaluated using specifications for design pattern implementation validity and the Quality Model for Object-Oriented Design (QMOOD) in order to see how well the design pattern had been implemented, and further iterations were made in case the initial refactor by the LLM was not good enough. In certain cases, the LLMs were able to perform the refactor correctly on their first attempt, while in other cases they did not get it right initially and had to be prompted one or more times extra to get it to perform the refactor correctly. In one case, an LLM’s refactor changed the behaviour of the application slightly and in that same case it did not fix one issue with the design pattern implementation even though it was prompted to fix it. Guidelines for performing design pattern refactorings with LLMs were also proposed based on the findings.