NLP-based Deepfake Text Detection - Identifying AI-generated Fraudulent Text

Sammanfattning

Introduction: The increasing accessibility of large language models (LLMs) such as GPT-4 has raised significant concerns about their misuse in generating fraudulent content. In particular, fraudulent information generated by artificial intelligence (AI) has become more complex, more convincing, and difficult to detect through traditional rule-based or keyword-driven filtering systems. Because fraudulent texts often rely on well-structured, human-like languages, they present new challenges to content auditing and security systems. Research Question: This study investigates the following question: How effective are current AI text detection tools in identifying LLM generated fraudulent texts? It aims to compare the performance of different detection tools to determine which performs best in identifying AI-generated fraudulent content. Method: To address this question, an experimental approach was adopted. A balanced dataset of AI-generated and human-written fraudulent texts was constructed. The AI texts were produced using ChatGPT, and the human-written samples were sourced from publicly available datasets. Three representative detection tools were selected for evaluation: Giant Language Model Test Room (GLTR), ZeroGPT and DetectGPT. Each tool was applied to the dataset, and its performance was evaluated using standard classification metrics including accuracy, precision, recall and F1-score. Results: The results demonstrate that GLTR provided the most balanced detection of AI-generated fraudulent texts, with accuracy, precision, and recall all close to 0.78. DetectGPT also performed strongly, maintaining recall above 0.70 and accuracy of 0.71. In contrast, ZeroGPT, while achieving perfect precision (1.00), detected only a small fraction of AI-generated texts, with recall of 0.18. Performance further varied by text length: medium-length texts yielded the highest detection rates, while short texts produced the most misclassifications. Combined, these findings highlight the different strengths and weaknesses of current AI text detection tools and underscore the limitations of relying on a single method for identifying fraudulent content. Discussion:These findings suggest that while current AI text detection tools exhibit promising performance under controlled conditions, their effectiveness in detecting LLM-generated fraudulent texts differs greatly across contexts. GLTR and DetectGPT show more balanced detection, but their accuracy drops with shorter texts or when the content is adversarially altered. ZeroGPT, while reaching perfect precision, misses most fraudulent AI-generated texts. These results highlight the need for multi-strategy detection systems and raise concerns about the reliability of existing detectors in high-risk environments such as phishing or impersonation scams.