Benchmarking Function Calling in LLMs

Sammanfattning

Function calling enables large language models to interact with external toolsby emitting structured calls that an application can execute. Asthe number of available tools grows, reliably selecting the correct tool andarguments becomes harder. A controlled benchmark is presented that varies thesize of the available tool set while holding task structure constant, enablingdirect comparison of function-calling performance at scale. A set ofproprietary and open-source models is evaluated on tasks that require a singletool call, and scored on tool-selection accuracy, call format validity,argument correctness, and strict success rate. The benchmarkprovides a systematic view of how tool set size affects function-callingreliability and offers practical guidance for building agentic systems.