Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

TL;DR — A continual-learning evaluation on Terminal-Bench 2.0 tests whether agent-optimization gains compound across successive optimization rounds, finding that only regression-controlled optimization sustains and grows improvements. This matters because deployed agents are optimized recursively as new tasks and failures appear, not just once against a fixed benchmark.

• Compares three agent-harness optimizers (GEPA, Meta Harness, RELAI-VCL) under identical budgets in a two-phase continual setup; all three beat the baseline in the conventional single-phase static test.
• Behaviors diverge once new tasks arrive: GEPA transfers below the unoptimized baseline, Meta Harness transfers well but stops improving with a second budget, and RELAI-VCL both transfers positively and keeps improving.
• RELAI-VCL reaches the highest lifelong average pass rate (76.4% vs. 66.0% GEPA, 64.6% Meta Harness, 58.7% baseline).
• Key claim: gains compound only when regression control is built into the optimization loop, biasing against non-generalizing shortcut solutions.