Self-tuning temperature controller using machine learning

Sammanfattning

Telecommunications is a keystone in society today, it is used everywhere and by everyone for many different tasks. It serves millions of people everyday with the possibility to communicate both in text and speech. For many people it is the main provider of news and social contacts. The backbone of telecommunications as we know it is provided by cellular base stations, they are equipped with antennas and radios to be able to transmit radio signals to end users. The radio in the cellular base station are working around the clock to provide coverage for everyone using them and they need to be stable and reliable. The radios are prone to damage if they reach to high temperatures due to failure in the electronics. To prevent heat damage they often rely on passive heat dissipation and software that shut them down or limit them if they reach high temperatures. One such countermeasure is the PID controller that limits the power input to the radio, and thus reducing the heat generation. The need for a PID controller to be accurate and stable is crucial for a well functioning radio, since limiting the power consumption in the radio also decreases the performance for the end users. In this thesis we investigate if it is possible to tune the PID controller in a radio using an reinforcement learning agent. The reason for tuning the PID controller is that the radios are located in completely different locations with different weather conditions. A hot place with no wind makes it harder for the heat to dissipate than the opposite, thus the PID controllers possibly can have different parameters depending on where they are deployed. For the training of the agent two different algorithms are considered, Proximal Policy Optimization (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3). After training the agents for 4*1e6 steps the evaluation was done. Evaluation showed that it was possible to use an agent to tune the PID controller. The performance of the agents did differ, with the TD3 agent running the radio much warmer than the temperature set as goal for the PID controller. The PPO agent did perform better in managing the heat, by limiting the traffic load. The balance between managing heat and allowing for traffic is hard and further studies are needed to analyse if the reinforcement learning agent is a good solution to tune the PID controller.