Evaluation of Cognitive Hierarchy-Based Sampling for Steering Large Language Models in Strategic Games

Sammanfattning

Strategic reasoning in multi-agent environments requires agents to form useful beliefs about the sophistication of their opponents. Recent work has shown that the reasoning process of large language models (LLMs) can be successfully steered through level-k inspired frameworks, but existing approaches assume unrealistic beliefs about opponents' levels of thinking. This thesis evaluates whether assuming a Poisson distribution over opponents' thinking levels could improve LLMs' performance in a strategic game. An existing LLM fictitious self-play framework is modified, replacing its uniform opponent-level sampling with a Poisson opponent-level sampling. The modified framework is evaluated against the original version, in Monte Carlo match-ups against an empirical human population, in Keynesian beauty contests. Metrics are expected payoff and similarity between the distributions of selected strategies. Compared with the original framework a Poisson opponent-level sampling with a Poisson rate of 1.5 increases Gpt-4o's expected payoff by 0.050 (95% CI: 0.037, 0.062) based on paired bootstrap differences, an approximate 15% improvement, and improves similarity to empirical human play by about 8%. However, matching the average reasoning depth of the opponent population is not sufficient by itself. The improvement depends on how the level-k strategies are aggregated and the selected LLM.