Co-Evolving LLM Evaluators and Policies via DynamicRubric
Ranking
Overall
86
Content
95
Popularity
66
Observed public metrics from 1 member.
Merged summary
TL;DR - DynamicRubric co-evolves LLM evaluators and policies by generating candidate-set-specific weighted rubrics, preserving useful score gaps as policy outputs converge in quality. With 8B backbones, it outperforms much larger evaluator baselines and improves reasoning, coding, and production search metrics.
- Frames relative evaluator score gaps as the exact optimization signal for reallocating probability between responses.
- Generates weighted binary rubric items conditioned on each candidate response set, then aggregates judgments into response-level scores.
- Provides stronger policy supervision than baselines using a 70B reward model or 235B static rubric generator.
- A resulting model serves all WeChat Search AI-answering traffic across tens of millions of daily requests.
Sources (1)
Co-Evolving LLM Evaluators and Policies via DynamicRubric
Public signals
Semantic Scholar citations 1 · Semantic Scholar influential citations 0
TL;DR - DynamicRubric co-evolves LLM evaluators and policies by generating candidate-set-specific weighted rubrics, preserving useful score gaps as policy outputs converge in quality. With 8B backbones, it outperforms much larger evaluator baselines and improves reasoning, coding, and production search metrics.
- Frames relative evaluator score gaps as the exact optimization signal for reallocating probability between responses.
- Generates weighted binary rubric items conditioned on each candidate response set, then aggregates judgments into response-level scores.
- Provides stronger policy supervision than baselines using a 70B reward model or 235B static rubric generator.
- A resulting model serves all WeChat Search AI-answering traffic across tens of millions of daily requests.