Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents
Ranking
Overall
79
Content
95
Popularity
41
Observed public metrics from 1 member.
Merged summary
TL;DR - COTA improves LLM agents at runtime using a small comparator that selects promising alternative actions rather than solving tasks or generating corrections itself. It improved performance across all nine tested actor-environment settings, suggesting weaker auxiliary models can still provide effective recovery guidance.
- The comparator evaluates same-prefix alternative continuations against the agent’s proposed action using pairwise judgments.
- Repeated comparisons determine whether intervention is needed; preferred alternatives are offered as non-binding advice so the original agent can replan.
- Training uses pairwise supervision from counterfactual branches sharing the same task prefix.
- COTA outperformed the compared baselines across WebShop, ALFWorld, and tauÂł-Retail with three different actors.
Sources (1)
Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - COTA improves LLM agents at runtime using a small comparator that selects promising alternative actions rather than solving tasks or generating corrections itself. It improved performance across all nine tested actor-environment settings, suggesting weaker auxiliary models can still provide effective recovery guidance.
- The comparator evaluates same-prefix alternative continuations against the agent’s proposed action using pairwise judgments.
- Repeated comparisons determine whether intervention is needed; preferred alternatives are offered as non-binding advice so the original agent can replan.
- Training uses pairwise supervision from counterfactual branches sharing the same task prefix.
- COTA outperformed the compared baselines across WebShop, ALFWorld, and tauÂł-Retail with three different actors.