CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - CoRE replaces majority voting in test-time reinforcement learning with a graph-based consensus mechanism, extracting a dominant set from roll-outs to produce graded, calibrated self-supervised rewards on unlabeled data. It matters because it fixes two structural flaws of vote-based TTRL — discarding correct minority answers and treating all majority-matching roll-outs identically — at no extra sampling cost.
- N roll-outs form a graph whose edges combine answer agreement, reasoning similarity, and generation confidence; replicator dynamics extract the dominant set, yielding a refined pseudo-label, per-roll-out graded rewards, and a per-question cohesiveness gate.
- Theoretically, majority voting is recovered as a special case; a block-value analysis gives a sharp threshold for when consensus recovers a correct minority against a larger wrong plurality, and confidence calibration provably lowers that threshold multiplicatively.
- Empirically across 7 backbones and 5 benchmarks (42 model–benchmark cells, 3 seeds each), CoRE improves the untrained base by +21.7 points on average vs +20.4 for majority-vote TTRL, with margins up to +7.5 points where agreement is contestable.
- CoRE reaches the voting baseline's plateau accuracy in 54–70% fewer steps, indicating better sample/compute efficiency rather than just a higher ceiling.
Sources (1)
CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
TL;DR - CoRE replaces majority voting in test-time reinforcement learning with a graph-based consensus mechanism, extracting a dominant set from roll-outs to produce graded, calibrated self-supervised rewards on unlabeled data. It matters because it fixes two structural flaws of vote-based TTRL — discarding correct minority answers and treating all majority-matching roll-outs identically — at no extra sampling cost.
- N roll-outs form a graph whose edges combine answer agreement, reasoning similarity, and generation confidence; replicator dynamics extract the dominant set, yielding a refined pseudo-label, per-roll-out graded rewards, and a per-question cohesiveness gate.
- Theoretically, majority voting is recovered as a special case; a block-value analysis gives a sharp threshold for when consensus recovers a correct minority against a larger wrong plurality, and confidence calibration provably lowers that threshold multiplicatively.
- Empirically across 7 backbones and 5 benchmarks (42 model–benchmark cells, 3 seeds each), CoRE improves the untrained base by +21.7 points on average vs +20.4 for majority-vote TTRL, with margins up to +7.5 points where agreement is contestable.
- CoRE reaches the voting baseline's plateau accuracy in 54–70% fewer steps, indicating better sample/compute efficiency rather than just a higher ceiling.