Consilience for Verifier-Free Test-Time Scaling
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - An arXiv paper showing that confidence-based verifier-free test-time scaling collapses on hard reasoning tasks, and proposing "consilience," a selection metric based on the temporal shape of confidence across a rollout. It matters because verifiers are unavailable in most real-world settings, making cheap, model-agnostic rollout selection the practical path to better LLM reasoning.
- Diagnoses a failure mode: uniformly high confidence across a rollout signals a failure to explore, so confidence-ranking methods systematically favor confidently wrong answers on complex tasks.
- Core insight is that robust reasoning has an asymmetric confidence trajectory — exploratory branching (low initial confidence) converging to high final certainty.
- Operationalized as a combinatorial metric that penalizes high initial confidence while requiring high final confidence, retaining the near-zero evaluation overhead and minimal internal-state access of confidence-based VF-TTS.
- Reported to outperform existing baselines on graduate-level mathematics and free-form code generation; specific numbers and models are not given in the provided abstract.
Sources (1)
Consilience for Verifier-Free Test-Time Scaling
TL;DR - An arXiv paper showing that confidence-based verifier-free test-time scaling collapses on hard reasoning tasks, and proposing "consilience," a selection metric based on the temporal shape of confidence across a rollout. It matters because verifiers are unavailable in most real-world settings, making cheap, model-agnostic rollout selection the practical path to better LLM reasoning.
- Diagnoses a failure mode: uniformly high confidence across a rollout signals a failure to explore, so confidence-ranking methods systematically favor confidently wrong answers on complex tasks.
- Core insight is that robust reasoning has an asymmetric confidence trajectory — exploratory branching (low initial confidence) converging to high final certainty.
- Operationalized as a combinatorial metric that penalizes high initial confidence while requiring high final confidence, retaining the near-zero evaluation overhead and minimal internal-state access of confidence-based VF-TTS.
- Reported to outperform existing baselines on graduate-level mathematics and free-form code generation; specific numbers and models are not given in the provided abstract.