Reinforcement Learning for Predictive Maintenance in a Configurable Manufacturing Environment

Sammanfattning

This thesis investigates reinforcement learning for maintenance decision-making in configurable industrial environments. In addition to maintenance decisions the environment can be configured through production load configurations which effects the performance of the system. The problem is therefore formulated as a configurable Markov Decision Process where policies and environment configurations are jointly optimized. The problem is approached using a Deep Q-Network (DQN) framework combined with a finite configuration search over discrete production load configurations. The approach is evaluated in a simulated manufacturing environment with stochastic degradation, stochastic demand, and technician constraints. Moreover, the performances of the learned policies in different environmental setups, based on complexity, are compared to threshold-based maintenance policies. The results show that reinforcement learning performs similarly to threshold-based policies in the simpler environments, but outperforms them in environments with higher complexity where technician availability and stochastic degradation impact more. Here the DQN-based policy can use additional system information outside of component health which results in fewer failures, less downtime and thus better performance. Furthermore, the results show that environment configuration has a significant impact on performance. Additionally, different policies favor different configurations, this indicates that there is no universal optimal configuration independently of the policy in this environment. Overall, this thesis concludes that reinforcement learning combined with environment configuration is a promising approach for maintenance decision-making in complex manufacturing systems.