An Empirical Study of Lightweight Transformer Models for Fake News Detection

Sammanfattning

Background: Fake news has become a growing problem on online platforms and can influence public trust, social stability, and democratic processes. Although transformer models such as BERT can produce strong performance on text classification tasks, they also usually require substantial computational resources, which limits their applicability in real-world environments. Objectives: The goal was to evaluate the performance of three lightweight transformer models (DistilBERT, ALBERT, MobileBERT) for fake news detection by comparing them with BERT in terms of classification performance and the computational efficiency of each model in achieving the same goal under the same experimental setting. Methods: An experimental study was performed using a fake news data set that is available to the public. All models were trained and tested using the same preprocessing procedures and training configurations. The performance of the models was evaluated in terms of accuracy, precision, recall, and F1-score. The efficiency of the models was evaluated in terms of training time, inference latency, and numberof parameters. Results: The results show that Lightweight transformer models demonstrated nearly equivalent performance to BERT with a significantly lower computational resource requirements. The best performing lightweight model was DistilBERT which provided the best combination of efficiency and performance. Although ALBERT and MobileBERT had fewer model parameters than BERT and lower inference latency between predictions, they had trade-offs in prediction performance compared to BERT. Conclusions: Lightweight transformer models are a viable alternative to BERT for fake news detection with respect to computational resources. This work has demonstrated the ability to produce a high level of classification performance while minimizing the overall cost of computation.