Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
Ranking
Overall
88
Content
95
Popularity
70
Observed public metrics from 1 member.
Merged summary
TL;DR - Task-CoEvolve reduces the cost of optimizing LLM agent harnesses by adaptively evaluating the validation tasks that best distinguish candidate harnesses. It matches full-validation-set search performance while using 80% fewer evaluations.
- Samples tasks using outcome variance, emphasizing examples near the agent’s evolving capability frontier.
- Estimates full-set performance from partial evaluations by correcting for each task’s sampling probability.
- Enables consistent candidate comparisons even when different validation subsets are used across iterations.
- Outperforms fixed-subset baselines on online text classification and Terminal-Bench 2.1.
Sources (1)
Task-CoEvolve: Efficient Harness Optimization via Adaptive Validation Task Selection
Public signals
Hugging Face upvotes 11
TL;DR - Task-CoEvolve reduces the cost of optimizing LLM agent harnesses by adaptively evaluating the validation tasks that best distinguish candidate harnesses. It matches full-validation-set search performance while using 80% fewer evaluations.
- Samples tasks using outcome variance, emphasizing examples near the agent’s evolving capability frontier.
- Estimates full-set performance from partial evaluations by correcting for each task’s sampling probability.
- Enables consistent candidate comparisons even when different validation subsets are used across iterations.
- Outperforms fixed-subset baselines on online text classification and Terminal-Bench 2.1.