Designing a Pipeline for Creating and Evaluating Swedish Instruction Datasets for Large Language Models

Sammanfattning

GPT-models have shown remarkable capabilities in natural language gener- ation (NLG) tasks. Despite the advanced multilingual capabilities of chat assistants such as ChatGPT, these models can often exhibit an underlying American bias. This research is motivated by the need to enhance linguistic and cultural representativity in the Swedish language by exploring a pipeline for creating and evaluating Swedish instruction datasets. The pipeline developed in this thesis incorporates multiple stages, including data collection, curation, fine-tuning, and evaluation. Data collection involves translating existing instruction datasets from English to Swedish, generating synthetic data that is culturally relevant, and sourcing original Swedish content. The curation process emphasizes automatic annotation and cleaning using advanced tools, ensuring high-quality, diverse datasets. Fine-tuning is performed using the GPT-SW3 base model, a Nordic-centric LLM developed by AI Sweden. This model is fine-tuned with the collected datasets using instruction tuning to create a chat assistant. This is further extended by briefly exploring Direct Preference Optimization (DPO), an emerging technique for aligning models with human preferences without the need for reinforcement learning. The evaluation phase leverages benchmarks such as ScandEval to assess the performance of the fine-tuned models, as well as utilizing tasks from the Swedish SAT. The results of this study have demonstrated a somewhat increased ability in Swedish language tasks, such as identifying toxic content, question/answering, and reasoning. While the pipeline has demonstrated potential for improving the language capability of Swedish LLMs, future work should focus on more diverse methods for gathering Swedish data, as well as more robust evaluation pipelines.