Novelty-Aware Frame Selection for Streaming Object Detection: A Centralized and Federated Study on Driving Data

Sammanfattning

Modern vehicle fleets produce continuous camera streams, but training on everyframe is impractical under compute and bandwidth constraints. Many frames areredundant, while frames from altered lighting conditions, adverse weather, or unfamiliarroad contexts carry most of the adaptation signal. This thesis studies noveltyawareframe selection for streaming object detection, where frames are scored byMahalanobis distance to a periodically refreshed reference Gaussian. A bootstrapanchor keeps novelty tied to an initial urban daytime reference, and the scoringsnapshot, reference, and threshold are refreshed together so scores stay meaningfulas the detector evolves.Using the Zenseact Open Dataset (ZOD) and an FCOS/ResNet-50 detector, themethod is evaluated in centralized streaming and in a four-client federated setting.Each variant is paired with a random baseline at the same empirical acceptancerate to isolate selection quality from data volume. Results show three consistentpatterns. The filter is domain-sensitive, with acceptance varying strongly acrossstream conditions. Periodic refresh is essential for calibration; without it, the staticfilter drifts from a 20% target acceptance rate to 77% and saturates on novel segments.At matched acceptance rate, detection gains are positive but modest. Theprimary practical benefit is systematic per-domain coverage, with frames from novelor under-represented driving conditions reliably prioritized over familiar content.The federated setting reproduces the same three patterns.