Comparative Analysis of Decision-Making Algorithms for False Alarm Mitigation in Aerial Object Detection

Sammanfattning

Unauthorized Unmanned Aerial Vehicles (UAVs) over restricted airspace are an increasing threat, and aerial surveillance systems frequently produce false alarms when they encounter objects that resemble drones, such as birds, airplanes, or helicopters. A persistent limitation is that false alarms are not adequately addressed in current systems, and there is a lack of comparative studies of Decision-Making Algorithms (DMAs) specifically designed to mitigate them. In our thesis, we conduct a limited comparative analysis of DMAs for false alarm mitigation in aerial object detection. The evaluated algorithms include Greedy Non-Maximum Suppression (NMS), Weighted-Cluster NMS, Weighted Boxes Fusion (WBF), a Kalman Filter, Long Short-Term Memory (LSTM), and Signal Temporal Logic (STL). Nine object detection models — YOLOv8n, YOLOv8m, YOLOv9t, YOLOv10n, YOLOv11n, YOLOv12n, YOLOv26n, and Faster R-CNN, are trained and evaluated across two datasets: the AOD4 dataset and a custom mixed dataset. The three classification targets are bird, drone, and unknown, where unknown represents out-of-distribution objects such as airplanes, helicopters, hot air balloons, paragliders, or anything beyond that which is not known to the system. Under Greedy-NMS, Faster R-CNN achieves the highest average F1-score of 82.35%, while YOLOv8m leads among YOLO architectures at 77.98%. WC-NMS produces inconsistent results it collapses for YOLOv10n and YOLOv26n, likely due to incompatibility with their built-in suppression mechanisms, while offering only modest gains for other models. Our WBF ensemble of YOLOv8m, YOLOv11n, and YOLOv26n achieves an average F1 of 79.22%, outperforming all individual YOLO models. When applied in isolation, our STL layer correctly mitigated up to 94.74% of false alarms on standalone models. In the fully integrated and unified DMA, the mitigation rate was more conservative at 7.94%, which is not factually known, therefore this remains unexplored. We found that strictly suppressing false alarms consistently creates a trade-off with missed detection, and no single DMA resolves this for all classes simultaneously. Our proposed hybrid framework, which combines ensemble fusion, temporal tracking, and formal logic verification, offers the safest handling of out-of-distribution targets and provides a practical foundation for future safety-critical aerial surveillance systems.