Adaptive Incentive Design for Dynamical Games with Unknown Dynamics: Model-Based and Model-Free Approaches

Sammanfattning

This thesis studies incentive design in a dynamic multi-agent system, where agents act according to individual objectives that may differ from the objective of a system-level leader. The problem is investigated using a simplified two-zone thermal building model formulated as a linear-quadratic-Gaussian control problem. The objective is to design incentives that align the agents' Nash equilibrium behavior with the leader’s optimal cost. As a reference, an analytical model-based incentive design method from existing literature is implemented for the case where the system dynamics are fully known. This serves to verify the implementation and provides a benchmark for comparison. Two data-driven approaches are then considered. First, a recursive least squares (RLS) method is used to estimate the unknown system matrix A, after which incentives are computed using the estimated model. Second, a model-free reinforcement learning approach based on deep deterministic policy gradient (DDPG) is used to learn incentives directly from observed performance. The results show that the RLS-based method provides stable convergence and achieves low cost error, although it relies on an accurate model estimate. The model-free DDPG approach is able to learn effective incentives, but exhibits slower and less stable convergence and requires more data to reach comparable performance. Overall, the results highlight a trade-off between model-based and model-free approaches: model-based methods provide superior performance when the system structure is known or can be accurately estimated, while model-free methods remain useful in settings where the system dynamics are unknown or difficult to model.