Beyond the Capability Boundary: Zeroth-Order Optimization for Self-Evolving LLM Agents
Ranking
Overall
69
Content
80
Popularity
43
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv preprint proposing a zeroth-order optimization framework that lets self-evolving LLM agents learn from difficult examples they cannot solve by sampling, pushing past their inherent capability boundary without trajectory annotations.
- Perturbs LoRA parameters, runs the agent under perturbed vs. original weights, and uses the loss difference to estimate gradients and update LoRA — no backprop or labeled trajectories required.
- The improved model then samples trajectories used for supervised fine-tuning, closing a self-evolution loop that breaks the sampling-based capability ceiling.
- Efficiency and stability additions: parallel perturbation inference, an adaptive lookup mechanism to cut zeroth-order overhead, and an answer-perplexity loss for smooth, stable loss signals.
- Reported gains on multiple deep research benchmarks: more successful trajectories and consistent improvement over strong baselines, most notably on hard examples; code released at github.com/hidk1911/ZOForLLMAgents.
Sources (1)
Beyond the Capability Boundary: Zeroth-Order Optimization for Self-Evolving LLM Agents
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - An arXiv preprint proposing a zeroth-order optimization framework that lets self-evolving LLM agents learn from difficult examples they cannot solve by sampling, pushing past their inherent capability boundary without trajectory annotations.
- Perturbs LoRA parameters, runs the agent under perturbed vs. original weights, and uses the loss difference to estimate gradients and update LoRA — no backprop or labeled trajectories required.
- The improved model then samples trajectories used for supervised fine-tuning, closing a self-evolution loop that breaks the sampling-based capability ceiling.
- Efficiency and stability additions: parallel perturbation inference, an adaptive lookup mechanism to cut zeroth-order overhead, and an answer-perplexity loss for smooth, stable loss signals.
- Reported gains on multiple deep research benchmarks: more successful trajectories and consistent improvement over strong baselines, most notably on hard examples; code released at github.com/hidk1911/ZOForLLMAgents.