Benchmarking Optimization Methods Using Gaussian process-Based User Simulations : Bayesian optimization and OLS-Based Gradient Descent Applied to Video-Game Parameter Tuning

Sammanfattning

In many modern video games, there exists a large capacity to continuously adjust in-game parameters and observe their effect on revenue. Benchmarking optimization policies' ability to optimize revenue over these parameters becomes a central issue, but is often hindered by long time horizons and the risk associated with deploying potentially poorly calibrated optimizers in production. Instead, this report investigates if it is possible to create a simulation of user behavior, in which revenue responses can be observed after changes in the parameter levels, and if optimizers can, as a pilot study, be reliably evaluated on said simulation. To this end, a Gaussian process was fitted to a large dataset of real parameter and revenue levels observed in a certain video game, and used as the base for a user simulation. Multiple variations of Linear regression-based gradient descent methods and Bayesian optimization were then benchmarked on different versions and realizations of this simulation. In general, the regression-based gradient methods appeared to outperform Bayesian optimization when sufficiently many users (300) were included in the simulation, while Bayesian optimization performs better when the number of users was very small (15). The reason for this is likely that the gradient estimates capture more information than Bayesian optimization when the number of users is large, but become noise-dominated as it decreases. Meanwhile, Bayesian optimization is robust to large magnitudes of observational noise, and adjusts its model assumptions accordingly. However, the reliability of these results is limited by dissimilarity between the simulation and the real system. For the simulation to become more faithful, the simulation must incorporate heterogeneity across the users' responses to different parameter vectors, and user revenues must be made correlated across time. Whether pure revenue maximization is a desirable objective is discussed. It is suggested to investigate if metrics such perceived autonomy and competence can be incorporated into the objective, which might increase user retention and long-term revenue.