Stability and Performance Trade-offs in Federated Learning Under Non-IID Label Skew

Sammanfattning

Federated Learning enables multiple clients to collaboratively train a shared machine learning model while keeping raw data local. This makes it useful in distributed and privacy-sensitive environments such as healthcare, finance, mobile devices, and industrial systems. However, federated Learning becomes challenging when client data are non-identically distributed. Label skew can cause clients to train on different class distributions, leading to divergent model updates, slower convergence, and reduced global model performance. This thesis investigates how lightweight learning-rate adaptation affects the stability and performance of Federated Learning under Dirichlet-based non-IID label skew. The problem is studied using CIFAR-10 in a centralized Federated Learning setup with weighted FedAvg aggregation. Three strategies are compared: a fixed-learning-rate Baseline, Local Adaptation (LA), which adjusts each client’s learning rate based on local training-loss behaviour, and Server-Guided Adaptation (SGA), which adjusts the learning rate using cosine similarity between client model updates. The contribution is an empirical comparison of Baseline, LA, and SGA across three levels of label skew. Results show that the strategies achieve broadly similar final global accuracy, with LA obtaining the highest mean final accuracy under low and moderate label skew, while SGA obtains the highest mean final accuracy under strong label skew. The clearest difference appears in final-stage test-loss stability, where SGA gives the lowest loss variation across all skew levels. Overall, the findings suggest that lightweight learning-rate adaptation has a modest effect on final accuracy but can improve final-stage training stability under non-IID label skew.