Exploring Programmatic Tool-Calling for AI Agents

Sammanfattning

Large Language Model (LLM)-based agents rely on external tools to extend capabilities for solving complex tasks. However, conventional in-context tool-calling introduces scalability challenges as tool schemas, intermediate outputs, and execution histories accumulate in the model context. This context bloat can increase token usage and latency, elevating the risk of attention dilution and hallucinations in long-horizon workflows. This work investigates Programmatic Tool-Calling (PTC) as an alternative architecture, where the model generates executable code to orchestrate tool calls in an externalised runtime rather than repeatedly calling and passing intermediate tool results through the model’s context window. To study this, a PTC prototype was implemented on top of an existing LLM system. The prototype supports both traditional in-context tool-calling and PTC, enabling a controlled comparison between the two architectural approaches. The evaluation was conducted using the BFCL and NESTFUL benchmarks, alongside a use-case study conducted in collaboration with an industrial partner company, based on realistic financial tasks. The analysis considers both task efficacy and operational efficiency, including benchmark accuracy, token consumption, latency, tool-call behaviour, and user-perceived quality. Results reveal a clear performance trade-off across evaluations, with a distinct contrast between task variations. While performance varied slightly depending on the underlying LLM, motivating future research into models explicitly fine-tuned for PTC, general patterns emerged. In single-turn and planning tasks, PTC improved task efficacy with higher success rates, but required an operational trade-off, consuming more tokens in most tasks. This suggests that executable code assists LLMs with logical reasoning and expressing structured tool dependencies more effectively, at the cost of computational resources. Conversely, in long-horizon multi-turn tasks, PTC achieved much higher operational efficiency by mitigating context bloat and drastically reducing total token consumption. However, this efficiency gain came at the expense of task efficacy, indicating that models struggle to maintain stateful understanding over extended trajectories due to the additional abstraction layer introduced by PTC. The use-case study reflected a middle-ground, where PTC reduced overall input token usage and irrelevant tool calls while maintaining similar success ratings, but these efficiency gains did not consistently correlate with higher user satisfaction. Ultimately, the findings indicate that PTC is a promising but context-dependent alternative to standard in-context tool-calling. While it can improve efficiency and structured tool use in some settings, its benefits are not consistent across all models and tasks. Therefore, the adoption of PTC should be strategically tailored to specific use cases, favouring complex, data-dense single-turn tasks over extended conversational workflows.