Machine learning and transformer-based models for fake news detection : Can a hybrid approach, combining BERT and SVM (BERT-SVM) outperform standalone BERT and SVM models in terms of accuracy, interpretability, and computational costs?

Sammanfattning

The spread of false information has expanded along with artificial intelligence. To detect fake news, this study assesses whether a hybrid model that combines transformer-based model BERT for contextual feature extraction and Support Vector Machines (SVM) for classification can outperform standalone models TF-IDF+SVM or end-to-end BERT classifiers in terms of predictive performance, interpretability, and computational cost. The experiments were conducted on two benchmark datasets namely, LIAR and Fakenewsnet. The reason why these two datasets were used consistently is because using different datasets would provide inconsistent results in this research. The hybrid approach had the highest Macro-F1 for both datasets at 0.6312 on LIAR and 0.9012 on FakeNewsNet. However, on LIAR the confidence intervals overlapped a great deal, showing that the advantage is not statistically valid for short texts only. SVM was the most computationally efficient and interpretable method, while the hybrid approach took 11-14% longer to train in comparison to standalone BERT, two models that produced similar inference times. The findings indicated that no single model performed well on all three measures of performance, while text length was the most influential factor when assessing relative model performance. Therefore, the results presented in this paper are useful in determining the model architecture you will choose based on their requirement and development purposes. The study concludes that while hybrid model BERT-SVM offers incremental performance gains, it necessitates trade-offs in computational cost compared to SVM. Due to computational resource constraints, interpretability analysis could not be completed and is identified as a direction for future work. Future work should evaluate alternative transformer variants, multi-seed stability, different languages and more on interpretability studies to assess practical deployment trade-offs.