Fake News Detection: A suitability comparison of XLM-RoBERTa and SVM to detect fake news in low resource languages

Sammanfattning

Introduction This thesis seeks to evaluate the method of compensating for lack of datasets in low-resource languages by using a large English dataset. Fake news is a considerable issue in the modern world and various detection tools have been tried and considered to help counteract it. Most of these automatic tools require large amounts of training data, which poses an issue for many smaller languages which do not have these resources available. Research Question The primary research question is: Can machine learning fake news detection in Swedish be improved by using a large English dataset? Method This thesis implements experimental quantitative methodology. The experimental method employed was fine-tuning both XLM-RoBERTa model and training the SVM algorithm on a large English-language fake news dataset and then measuring their success in detecting fake news on a small Swedish-language dataset. The control is fine-tuning and evaluating both models on only the Swedish dataset which is presumed to insufficient to yield high accuracy. Results The measurements used to evaluate the methods is weighted and macro F1-score and accuracy. All results suggest that it was not helpful to add the English dataset as the highest results for both SVMand XLMRoBERTa were achieved when trained on only the Swedish dataset. Discussion No technique that incorporated the English dataset was better than training on just the Swedish dataset. There are many potential reasons for this, the dissimilarity of the two datasets, the unbalanced nature of the Swedish dataset or perhaps the whole method is flawed. In general SVM performed better than XLM-RoBERTa.