Explainable Reinforcement Learning for Adaptive Resource Allocation in Distributed Systems : An Experimental Study of Performance, Stability, and Policy Behavior in Distributed Systems

Sammanfattning

This thesis investigates the use of explainable reinforcement learning for adaptive resource allocation in sharded NoSQL-based distributed systems. The study focuses on the problem of uneven load distribution, where skewed access patterns can create hot shards, increase tail latency, and reduce overall throughput. To address this, a reinforcement learning agent was designed to dynamically provision and remove Redis shard instances and migrate key slots between overloaded and underutilized nodes. The agent was trained using Proximal Policy Optimization in a custom Gymnasium environment and evaluated against a baseline approach using Redis Cluster, a modified Yahoo Cloud Serving Benchmark workload generator, and Poisson and Zipfian traffic patterns. The experimental results show that the reinforcement learning agent can improve load balancing and resource adaptation under skewed workloads, particularly by reducing utilization imbalance and responding to under-provisioned or over-provisioned cluster states. However, the results also indicate that migration and provisioning decisions introduce trade-offs between performance, stability, and resource efficiency. Explainability was explored using SHAP-based global and local explanations. While the explanations provided some insight into the importance of observed system features, they were not sufficient on their own to fully explain the agents sequential decision-making behavior. The thesis concludes that reinforcement learning is a promising approach for adaptive sharding, but further work is needed to improve real-world deployment, statistical validation, and human-centered explainability.