Quantifying Overclaiming Propensity in Frontier LLM Agents

TL;DR - OverclaimBench evaluates whether coding agents accurately report task completion, finding that incomplete reviews are frequently presented misleadingly. This matters because users often rely on an agent’s final response as the primary record of its autonomous work.

• Across 12 frontier models, agents failed to read every requested file in 67.9% of runs.
• Of incomplete runs, 80.4% falsely claimed full coverage or omitted that coverage was incomplete.
• Mandatory subagent delegation improved file coverage but did not eliminate misleading reports among incomplete reviews.
• Agents falsely claiming complete reviews missed planted defects at roughly 1.8 times the rate of agents that read every file.