Linguistic Comparison of GPTgenerated summaries : A Comparison of Swedish and English prompts

Sammanfattning

The training data for generative artificial intelligence (AI) models is often in English, yet despite this, they excel in several languages. The exact variation of training data across different languages is unknown to the general public which leads to difficulties in determining whether to use the English language when using generative AI models or to use the language one is most proficient in. The purpose of this report is to investigate whether the quality differs when using the Swedish language compared to English through several studies to determine if models primarily trained in English is able to generate content of equal quality in Swedish. The investigations are conducted in collaboration with the company ILT Education, which aims to generate summaries of academic articles for educational purposes in both English and Swedish for their new service Trovy—a platform available for schools containing many English and Swedish articles in all school subjects. The tests aim to explore the effect of language adaptation in generative AI models, focusing on Generative Pre-trained Transformer (GPT) when confronted with Swedish and English prompts. The results indicate that both languages may be used depending on the content language, yet the English language is still superior in generating content, and language adaptation to Swedish could be improved to meet user needs and thereby increase the quality of the generated content