Comparative Evaluation of Machine Learning Models for Federated Security Alert Prioritisation in Security Operations Centers

Sammanfattning

Background: Security Operations Centers (SOCs) face a persistent mismatch between the volume of security alerts produced by modern detection infrastructure and the capacity of human analysts to investigate them. This mismatch produces alert fatigue and increases the risk that genuine attacks will be missed. Cross-organizational collaboration could improve the performance of automated alert prioritization but is constrained by the legal, regulatory, and competitive obstacles that prevent SOCs from sharing raw alert data. Objectives: The thesis evaluates whether federated learning, a training paradigm in which model updates are exchanged in place of raw data, can support cross-organizational alert prioritization without compromising classification performance and characterizes how the choice of model and the size of the federation affect the result. Methods: Four supervised models—logistic regression, random forest, XGBoost, and a feed-forward neural network—are compared on a four-class severity prioritization derived from the CIC-IDS2017 intrusion detection dataset. Federated training is performed using the Flower framework with Dirichlet partitioning across simulated SOC clients, under both non-IID (α = 0.5) and near-IID (α = 100) regimes, and the client count is varied between two and ten. Results: Federated training matches or modestly exceeds centralized performance across all federated-eligible models. XGBoost in cyclic mode reaches a macro-F1 of 0.994 to 0.996 across all configurations. For logistic regression and the feed-forward neural network, non-IID partitioning yields a higher rare-class F1 than near-IID, inverting the conventional federated learning expectation. The two XGBoost tree-aggregation strategies behave identically under near-IID partitioning but diverge catastrophically under non-IID partitioning. Conclusions: Federated alert prioritization is feasible without sacrificing classification performance, but the technical choices of aggregation strategy, partitioning regime, and client population have effects large enough to determine whether a deployment succeeds or fails.