Large Language Model OptimisedMulti-Agent Coordination inLife-Simulating Games : A feasibility study of black-box parameter optimisation forLLM-driven agents in Minecraft

Sammanfattning

Large language models (LLMs) are increasingly used as decision-making policies for autonomous agents in industrial applications,such as customer-service automation, coding assistants, and multi-agent business workflows. Since each agent decision requires apaid LLM call, operating such systems at scale can be costly.This makes the tuning of behavioural prompt parameters, such asexploration, cohesion, communication, leadership, and task focus,both economically relevant and technically challenging.This thesis studies the problem as a cost-constrained, derivative-free black-box optimisation task. We compare three optimisers: uni-form random search, Bayesian optimisation using a Tree-structuredParzen Estimator (TPE), and an LLM-driven evolutionary optimiserwith domain context. The experiment is conducted in the MineLandMinecraft framework using five LLM-driven agents. Each agentis controlled by six behavioural parameters, resulting in a 30-dimensional mixed search space. Each optimiser is evaluated for 30iterations, and performance is measured using a return-on-investment(ROI) objective combining inventory value, deaths, and token cost.The results show that the LLM-driven evolutionary optimiserachieves the best mean ROI, while random search produces the highestsingle observed ROI. However, validation runs indicate that thesedifferences are not statistically robust. Re-running the best and worstobserved configurations produces highly variable and overlappingoutcomes, with a noise level larger than the spread of positive results.The main conclusion is therefore that, within this evaluationbudget, the objective is too noisy to support a definitive ranking of theoptimisers. We frame the contribution as a feasibility and complexitystudy rather than as a conclusive optimisation benchmark. The thesisshows that small-budget optimisation can reveal useful behaviouralpatterns, but that reliable optimiser comparison requires substantiallymore replications or a less noise-dominated reward signal.