Isolation Forest for Multi-Level Anomaly Detection in Hierarchical System Logs

Sammanfattning

Manual inspection of large system logs is impractical for troubleshooting complex cyber-physical systems. This thesis investigates an unsupervised Isolation Forest approach for anomaly detection in a multi-level system at Saab. While the main system log can indicate failures, it is insufficient for identifying root causes distributed across subsystem logs. To address this, a hierarchical approach is introduced where subsystem-level models are trained separately and their outputs are used as features in a higher-level main model. A processing pipeline was developed using Drain3 for log parsing, fixed two-minute event windowing, and evaluation on a representative synthetic dataset containing point, contextual, and collective anomalies. The proposed hierarchical model improves detection performance, increasing F1-score from 0.392 to 0.639 and Precision@44 from 0.432 to 0.705, while also enabling detection of hidden subsystem anomalies not visible in the main log. Overall, the results show that a hierarchical Isolation Forest combined with targeted PCA and selective feature removal provides an effective and scalable approach for automated anomaly detection.