Model-Based and Model-Free Incentive Design with Unknown Cost Functions

Sammanfattning

This report studies incentive design in linear-quadratic dynamic games, with an application to temperature control. Unlike existing work, we consider unknown player cost functions, so the leader must design incentives without knowing the players’ true preferences. To address this problem, we compare two approaches. In the model-based approach, the unknown discomfort parameters are recursively estimated using recursive least squares, and incentives are designed to align the players’ equilibrium behavior with the leader’s desired outcome. In the model-free approach, a neural network surrogate is trained to approximate the leader’s cost under different incentives, and gradient-based optimization is then used to find effective incentive parameters. The results show that the model-based approach can learn the unknown parameters while designing incentives, and performs well when the dynamics and cost functions are accurately specified. The model-free approach is more flexible, but its performance is more sensitive to noise and the quality of the training data.