Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
Merged summary
TL;DR - MetaEvolve trains LLMs to iteratively improve solutions using execution feedback, evolution-aware reinforcement learning, and inference-time search. It substantially improves coding and out-of-domain algorithm optimization performance, suggesting self-evolution skills can generalize beyond training tasks.
- Synthesizes trajectories containing programs, fitness scores based on correctness and efficiency, and prior attempts.
- Uses reinforcement learning with verifiable rewards from test-case execution.
- Beats the strongest baseline by 10.01% absolute in-distribution and 24.12% out-of-distribution across seven coding benchmarks.
- Achieves a 46.9% relative gain on open-ended algorithm optimization problems outside the training domain.
Sources (1)
Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
TL;DR - MetaEvolve trains LLMs to iteratively improve solutions using execution feedback, evolution-aware reinforcement learning, and inference-time search. It substantially improves coding and out-of-domain algorithm optimization performance, suggesting self-evolution skills can generalize beyond training tasks.
- Synthesizes trajectories containing programs, fitness scores based on correctness and efficiency, and prior attempts.
- Uses reinforcement learning with verifiable rewards from test-case execution.
- Beats the strongest baseline by 10.01% absolute in-distribution and 24.12% out-of-distribution across seven coding benchmarks.
- Achieves a 46.9% relative gain on open-ended algorithm optimization problems outside the training domain.