AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Ranking
Overall
86
Content
95
Popularity
66
Observed public metrics from 1 member.
Merged summary
TL;DR - AutoSaddler is an offline-learning framework that automatically improves LLM-agent harnesses using failure traces, structured patches, and validation-based update selection. It raises performance by 9.0–10.0 percentage points across three long-horizon agent benchmarks.
- Diagnoses failures from execution traces and treats harness modifications as structured code patches.
- Iteratively learns from mini-batches of failures rather than relying on manual prompt, tool, and control-logic tuning.
- Improves base harnesses by 9.0 points on GAIA2, 9.6 on SWE-Bench Pro, and 10.0 on Terminal-Bench 2.0.
- Ablations favor deep debugging, targeted edits, and generalization-aware validation over shallow reflection or trajectory-specific repairs.
Sources (1)
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Public signals
Hugging Face upvotes 64
TL;DR - AutoSaddler is an offline-learning framework that automatically improves LLM-agent harnesses using failure traces, structured patches, and validation-based update selection. It raises performance by 9.0–10.0 percentage points across three long-horizon agent benchmarks.
- Diagnoses failures from execution traces and treats harness modifications as structured code patches.
- Iteratively learns from mini-batches of failures rather than relying on manual prompt, tool, and control-logic tuning.
- Improves base harnesses by 9.0 points on GAIA2, 9.6 on SWE-Bench Pro, and 10.0 on Terminal-Bench 2.0.
- Ablations favor deep debugging, targeted edits, and generalization-aware validation over shallow reflection or trajectory-specific repairs.