The Bitter Lesson of Tool Calling

TL;DR - An empirical arXiv study comparing programmatic tool calling (PTC) — exposing tools as typed Python stubs invoked via code — against native JSON tool calling across 14 LLMs on BFCL v4. It matters because PTC matches or beats JSON calling on most models and degrades less under long-context conditions, suggesting code is the better agent-tool interface.

• PTC exposes tools as typed Python stubs, with the model chaining/parallelizing calls in a script and execution plus results handled within a single agent turn.
• PTC matched or exceeded native JSON tool calling in 11 of 14 models on BFCL v4; the GPT-5.6 family gained 10.6% over the JSON baseline.
• Under parallel fan-out, PTC matched or outperformed the baseline in 13 of 14 models.
• PTC stayed stable under "context rot" conditions, where the JSON baseline dropped 2.3% on average; gains generally track model capability across release generations.