When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv paper showing that stale-but-plausible dialogue/tool history can hijack a tool-calling agent's otherwise-correct policy, plus a paired benchmark and a distillation method that makes small models robust to polluted histories. It matters because long-running agent sessions accumulate traces that remain syntactically valid yet no longer authoritative, a failure mode distinct from weak tool-use skill.
- History pollution flips 32.1% of decisions that Qwen3-1.7B got right on the original trajectory, often causing reuse of corrupted entities or outdated interface conventions.
- The benchmark provides synchronized Original / Polluted / Oracle State views holding constant the system policy, current tools, latest request, and gold next action; eleven gold-preserving interventions isolate failures in decision state, entity binding, and interface execution across both calls and non-call decisions.
- The proposed method distills an Oracle-conditioned teacher into a student that sees only polluted history, using soft supervision on student-generated prefixes: 87.0% Balanced Tool-Use Accuracy vs. Gold-SFT 66.3%, Oracle sequence distillation 82.3%, off-policy token distillation 85.0%.
- It scales and transfers: an 8B teacher lifts the 1.7B student to 91.9% and an 8B student to 93.0%, with gains carrying over to clean histories, unseen functions, regenerated contexts, external tool-use benchmarks, and noisy multi-hop QA.
Sources (1)
When History Lies: Evaluating and Improving Tool Use under Misleading Multi-Turn Histories
TL;DR - An arXiv paper showing that stale-but-plausible dialogue/tool history can hijack a tool-calling agent's otherwise-correct policy, plus a paired benchmark and a distillation method that makes small models robust to polluted histories. It matters because long-running agent sessions accumulate traces that remain syntactically valid yet no longer authoritative, a failure mode distinct from weak tool-use skill.
- History pollution flips 32.1% of decisions that Qwen3-1.7B got right on the original trajectory, often causing reuse of corrupted entities or outdated interface conventions.
- The benchmark provides synchronized Original / Polluted / Oracle State views holding constant the system policy, current tools, latest request, and gold next action; eleven gold-preserving interventions isolate failures in decision state, entity binding, and interface execution across both calls and non-call decisions.
- The proposed method distills an Oracle-conditioned teacher into a student that sees only polluted history, using soft supervision on student-generated prefixes: 87.0% Balanced Tool-Use Accuracy vs. Gold-SFT 66.3%, Oracle sequence distillation 82.3%, off-policy token distillation 85.0%.
- It scales and transfers: an 8B teacher lifts the 1.7B student to 91.9% and an 8B student to 93.0%, with gains carrying over to clean histories, unseen functions, regenerated contexts, external tool-use benchmarks, and noisy multi-hop QA.