Co-Evolving LLM Evaluators and Policies via DynamicRubric
Merged summary
TL;DR - DynamicRubric co-evolves LLM evaluators and policies by generating candidate-set-specific weighted rubrics, preserving useful score gaps as policy outputs converge in quality. With 8B backbones, it outperforms much larger evaluator baselines and improves reasoning, coding, and production search metrics.
- Frames relative evaluator score gaps as the exact optimization signal for reallocating probability between responses.
- Generates weighted binary rubric items conditioned on each candidate response set, then aggregates judgments into response-level scores.
- Provides stronger policy supervision than baselines using a 70B reward model or 235B static rubric generator.
- A resulting model serves all WeChat Search AI-answering traffic across tens of millions of daily requests.
Sources (1)
Co-Evolving LLM Evaluators and Policies via DynamicRubric
TL;DR - DynamicRubric co-evolves LLM evaluators and policies by generating candidate-set-specific weighted rubrics, preserving useful score gaps as policy outputs converge in quality. With 8B backbones, it outperforms much larger evaluator baselines and improves reasoning, coding, and production search metrics.
- Frames relative evaluator score gaps as the exact optimization signal for reallocating probability between responses.
- Generates weighted binary rubric items conditioned on each candidate response set, then aggregates judgments into response-level scores.
- Provides stronger policy supervision than baselines using a 70B reward model or 235B static rubric generator.
- A resulting model serves all WeChat Search AI-answering traffic across tens of millions of daily requests.