Generative AI for automating software uplifts

Sammanfattning

Third-party libraries are essential to modern software development since they provide pre-built functionality that accelerate the processes. These libraries are however updated frequently, which introduce changes that can potentially break software systems. Therefore, it is important to identify and manage these so-called breaking changes properly. Existing tools such as APIDiff offer capabilities to compare library versions and detect breaking changes, but it remains a research tool and is not production-ready, making it unsuitable for production use.Recently, Large Language Models (LLMs) have shown huge potential in various areas, including software development. However, the use of online LLM tools can pose risks such as information leaks, making a locally executed Generative AI model preferable for security and efficiency. Our aim is to develop a model capable of identifying whether changes are breaking or not, thereby helping software developers in maintaining software.For this study, two base models are selected: Code Llama (7 billion parameters), a decoder-only model, and Code T5+ (770 million parameters), an encoder-decoder model. Both models were pre-trained on code and natural language data. Datasets created based on APIDiff results applied on selected Java repositories are curated for fine-tuning the models. Quantized Low Rank Adaptation (QLoRA) and full fine-tuning are employed on Code Llama 7B and Code T5+ 770M, respectively. Experiments are conducted using various learning rates, maximum steps, and datasets, with evaluations based on metrics such as accuracy, precision, and recall. The final model achieves a high performance, with an accuracy of 98.76%, precision of 98.58%, and recall of 99.76% on the test dataset.This thesis demonstrates the feasibility of using generative AI models to detect breaking changes in software libraries. Furthermore, practical applications of the model are outlined.