ConRub-Med: Reinforcement Learning with Consensus Rubrics for Open-Ended Medical Question Answering

TL;DR - ConRub-Med is an RL recipe that replaces expensive physician-written rubrics with consensus-filtered, model-generated rubrics to supervise open-ended medical QA, where cheap outcome verifiers don't exist. It shows scalable rubric supervision can beat larger-sample baselines on hard clinical benchmarks.

• Rubric construction: three heterogeneous LLMs independently propose atomic criteria, and a separate reviewer model keeps only criteria with semantic support from all three generators.
• Three-State scoring separates correct coverage, missing information, and incorrect claims, with errors given negative rather than zero credit.
• GRPO variant: when all responses in a group get identical rewards, a pairwise judge supplies sequence-level advantages only if both candidate orderings agree; untied groups use vanilla GRPO.
• Results: ranks first on 6 of 9 benchmarks with the best medical and generalization averages; 38.98 ± 1.04 on HealthBench-Hard from 5,166 prompts vs. InfiMed-ORBIT's 33.60 (8K) and 37.30 (28K). Blinded ratings by two medical experts favored the full pipeline's rubric panels over single-generator panels.