Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - CoPES is a cooperative coevolutionary evolution-strategy method that splits an LLM's parameter space into lower-dimensional subspaces to make backprop-free, full-parameter post-training of tool-using agents practical on just a few GPUs. It matters because it narrows the memory-vs-training-time gap that has made ES impractical versus gradient-based RL in resource-constrained settings.
- Motivation: tool-using agents generate long multi-turn trajectories, making gradient-based post-training memory-heavy; ES avoids backpropagation but demands prohibitive GPU-hours on small clusters.
- Method: Cooperative Parameter-subspace Evolution Strategy (CoPES) decomposes the full parameter space into lower-dimensional subspaces and searches them cooperatively to improve optimization efficiency.
- Results: post-training a Qwen3.5-4B math agent, under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain vs. 67% for standard ES, with theoretical GPU memory under one-eighth of full-parameter GRPO.
- Generality: beats standard ES and LoRA-based GRPO on all pass@k metrics across five benchmarks, with additional gains shown on a question-answering task; code is open-sourced.
Sources (1)
Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training
TL;DR - CoPES is a cooperative coevolutionary evolution-strategy method that splits an LLM's parameter space into lower-dimensional subspaces to make backprop-free, full-parameter post-training of tool-using agents practical on just a few GPUs. It matters because it narrows the memory-vs-training-time gap that has made ES impractical versus gradient-based RL in resource-constrained settings.
- Motivation: tool-using agents generate long multi-turn trajectories, making gradient-based post-training memory-heavy; ES avoids backpropagation but demands prohibitive GPU-hours on small clusters.
- Method: Cooperative Parameter-subspace Evolution Strategy (CoPES) decomposes the full parameter space into lower-dimensional subspaces and searches them cooperatively to improve optimization efficiency.
- Results: post-training a Qwen3.5-4B math agent, under the GPU-hour budget of full-parameter GRPO's best validation checkpoint, CoPES recovers 92% of GRPO's validation-accuracy gain vs. 67% for standard ES, with theoretical GPU memory under one-eighth of full-parameter GRPO.
- Generality: beats standard ES and LoRA-based GRPO on all pass@k metrics across five benchmarks, with additional gains shown on a question-answering task; code is open-sourced.