RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

TL;DR - RRSI regularizes recursive self-improvement of LLM agent harnesses to reduce benchmark overfitting and promote reusable improvements. Across eight coding, workspace, and engineering-design benchmarks, it improved both in-distribution and out-of-distribution performance while using fewer policy tokens.

• Constrains candidate proposals with an annealed edit budget and encourages exploration of previously unused evolution paths.
• Uses a critic to reject benchmark-specific proposals and a pruner to remove changes that are marginal, costly, or obsolete.
• Achieved gains of up to 14.1 points on the evolution split and up to 4.7 points across five out-of-distribution benchmarks.
• Produced a harness using 30% fewer policy tokens than unregularized evolution.