Consilience for Verifier-Free Test-Time Scaling

TL;DR - An arXiv paper showing that confidence-based verifier-free test-time scaling collapses on hard reasoning tasks, and proposing "consilience," a selection metric based on the temporal shape of confidence across a rollout. It matters because verifiers are unavailable in most real-world settings, making cheap, model-agnostic rollout selection the practical path to better LLM reasoning.

• Diagnoses a failure mode: uniformly high confidence across a rollout signals a failure to explore, so confidence-ranking methods systematically favor confidently wrong answers on complex tasks.
• Core insight is that robust reasoning has an asymmetric confidence trajectory — exploratory branching (low initial confidence) converging to high final certainty.
• Operationalized as a combinatorial metric that penalizes high initial confidence while requiring high final confidence, retaining the near-zero evaluation overhead and minimal internal-state access of confidence-based VF-TTS.
• Reported to outperform existing baselines on graduate-level mathematics and free-form code generation; specific numbers and models are not given in the provided abstract.