Communication Efficiency for Federated Learning in IoT Systems

Sammanfattning

The use of Artificial Intelligence (AI) increases every day in today’s society. Traditionally,AI models are trained on a central server using raw data collected fromparticipants. However, centralizing all data raises privacy concerns, prompting thedevelopment of distributed machine learning (ML) approaches. One such approachis Federated Learning (FL), where raw training data remains on local devices, andonly model updates are sent to the server for aggregation. While this shift towardsdistributed ML improves privacy, it introduces a new significant challenge: communicationefficiency, which is the focus of this project.This project investigates strategies to improve communication efficiency in FL. Itstudies the trade-off between model accuracy and communication cost when usingdifferent algorithms: Quantization for sketched updates, Low-Rank Adaptation(LoRA) for structured updates, and Communication-Mitigated FL for selective updates(CMFL). The experiments are conducted on local hardware representative ofInternet of Things (IoT) edge devices, specifically 30 Raspberry Pi 4 units.Through graphs and tables, we demonstrate the differences of the algorithms usingdifferent distributions, and using different training datasets. The results show thatusing quantization leads to a big reduction in the communication cost, with almostno change to the accuracy. LoRA shares a similar result in communication cost;however, with a larger percentage loss in the accuracy. LoRA is also more efficientin terms of Central Processing Unit (CPU) usage for the participants; however,it also increases the memory usage. CMFL showed potential in filtering out badupdates; however, it comes with a need of fine tuning.In conclusion, there are algorithms that reduce the communication cost while maintaininga high accuracy. Depending on what the model’s purpose is and the model’straining data, different algorithms yield different results in terms of accuracy, CPUusage, and average memory usage. Therefore, it is important to select the algorithmbased on the intended use of the model.