Efficient Knowledge Transfer in Federated Learning for Heterogeneous Autonomous Driving Systems

Sammanfattning

Federated learning enables autonomous driving systems to collaboratively train perception models without sharing locally collected data, making it a promising approach for distributed vehicle fleets. However, conventional federated learning pipelines assume homogeneous model architectures, which limits their applicability in real-world deployments where vehicles often operate under diverse hardware constraints and computational capabilities. This thesis investigates the use of knowledge distillation to facilitate efficient knowledge transfer between heterogeneous object detection models in federated autonomous driving environments. A framework integrating knowledge distillation initialization and federated learning was developed and evaluated using the Zenseact Open Dataset, with experiments conducted on object detectors based on GFL and RetinaNet architectures in both centralized and federated settings under IID and non-IID data distributions. The proposed approach employs CrossKD to transfer knowledge from previously trained teacher models to heterogeneous student architectures. Results show that knowledge distillation consistently improves the performance of lightweight student models while maintaining computational efficiency. Distilled models also demonstrated improved data efficiency, achieving comparable or superior performance using approximately 55–65\% of the original training data, and exhibited faster convergence during the early stages of training. In federated learning experiments, the proposed framework consistently outperformed standard federated learning, achieving faster convergence, higher final detection accuracy, and lower variance between clients, with significant improvements across both IID and heterogeneous non-IID scenarios. The findings demonstrate that knowledge distillation can effectively support knowledge transfer between heterogeneous object detection models, reduce the need for costly retraining from scratch, and improve the scalability and practicality of collaborative learning in autonomous driving systems.