Evaluating the Effects of Reasoning and Language on the Theory of Response Sampling

Sammanfattning

This thesis evaluates Large Language Model (LLM) decision-making through the Theory of Response Sampling, which posits that LLMs' responses are driven by a concept's statistical reality (descriptive) with its idealized version (prescriptive). The theory is extended by examining how explicit reasoning and the input language affect these responses. This thesis addresses two primary research questions: (1) To what extent is the idealized value of a concept consistent across reasoning conditions (Baseline vs. Reasoning)? and (2) To what extent is the idealized value of a concept consistent across linguistic contexts (Swedish vs. English)? The study experimentally analyzes LLM sample distributions across 510 concepts using four open-weight models and three closed-weight baselines. Responses were generated under three conditions: baseline, reasoning, and cross-lingual. The magnitude and significance of sampling biases were evaluated using Binomial and Wilcoxon Signed-Rank tests. Explicit reasoning had a limited effect on sampling distributions, primarily tightening output variance without fundamentally affecting underlying biases. Conversely, prompting in Swedish altered the models' sampling distributions and shifted their idealized values. These findings suggest LLM sampling is not a clean cognitive compromise, but a statistical artifact highly sensitive to language and training. Future research should determine whether "thinking tokens" faithfully reflect decision-making across multilingual settings.