Information Retrieval Augmentation- A quantitative study of enhancement implementations

Sammanfattning

In the world of Artificial Intelligence, Large Language Models (LLMs) are a fast growing technology. Since the launch of ChatGPT the usage of LLMs has greatly increased, both in private use and business implementations. For an LLM to work it has to be trained on vast amounts of data, but what happens when it is tasked with things that it has not been trained for? A number of problems can arise when this occurs, and the current answer for this is Retrieval Augmented Generation (RAG). RAG allows the LLM to retrieve what information it may need from varying sources and generate responses based on what it retrieved. This thesis aims to explore how the retrieval of information can be optimized using different retrieval methods and several retrieval enhancement methods, to explore their effects on performance using standardised metrics. The thesis presents several methods to increase retrieval performance: Re-Ranking, Hypothetical Document Embedding (HyDE) and Query Expansion. The study also presents several metrics to evalute these methods: nDCG@10, MAP@10 and Recall@1000, with mathematical backgrounds and motivations to how they are used. The results presented in the thesis are comparable to state of the art benchmarks, however, the thesis fails to ascertain a significant increase in retrieval performance for the presented methods. It is hypothesised that this is due to shortcomings of the data used, limitations of computational hardware and time constraints which influenced what methods were chosen.