Evaluation of AI-Based Intrusion Detection Systems for Real-Time Cybercrime

Sammanfattning

Introduction This thesis evaluates the application of AI-based methods for real-time cybercrime detection. The motivation for the study stems from the rapidly escalating and increasingly sophisticated nature of cybercrime, along with the growing use of AI by cybercriminals, developments that demand effective and intelligent defensive measures. Research Question The primary research question is “Can random forest & gradient boosting maintain ≥ 90% F1-score while sustaining ≤ 1 ms median inference latency under both baseline and 5x traffic loads?”. The goal is to explore and evaluate the suitability of AI methods for real-time detection with a focus on performance, responsiveness and the ability to provide actionable insights. Method An experimental strategy with a quantitative approach was employed, evaluating two machine learning models, Random Forest and Gradient Boosting, on the UNSW-NB15 dataset which is a widely used benchmark for intrusion detection. Model performance was measured using F1-score and median inference latency under simulated streaming loads of 1,000 and 5,000 samples per second. Performance was considered acceptable if the F1-score was ≥ 0.90 and median latency ≤ 1.0 ms at both traffic levels. Results Results showed that Gradient Boosting consistently achieved a F1-score of 1.0 and maintained a median latency well below 1 ms (approximately 0.6 ms under load), thus meeting all predefined performance criteria. While Random Forest achieved a high F1-score (0.936), it failed to meet the latency requirement, with median latencies around 55 ms under load. A statistical comparison confirmed a significant difference in F1-score in favor of Gradient Boosting. Discussion The study concludes that Gradient boosting is a suitable model for real-time cybercrime detection based on the tested criteria and dataset. The findings highlight the importance of including time-based performance metrics such as latency when evaluating models for real-time applications. However, limitations include reliance on a single static dataset, a narrow model selection and testing performed solely on CPU hardware. Future research should address these limitations by incorporating diverse datasets, evaluating deep learning and hybrid approaches, testing on different hardware platforms and exploring solutions for concept drift.