Stable Answers, Unfinished Reasoning: Why Self-Consensus Is Not a Safe Early-Exit Signal

TL;DR - Self-consensus—ending inference when repeated probes agree on an answer—is not a reliable early-exit signal for reasoning models. It can reduce token use, but often stops unfinished reasoning before the model corrects a provisional answer.

• A preregistered evaluation of 3,520 consensus rules across two models and three benchmarks found that none met all predefined safety and token-saving criteria.
• The failure reproduced on a held-out split and two unseen models, while the boundary-confidence method DEER passed all three acceptance gates.
• At 32% token savings, roughly one in nine early exits selected an answer the trajectory later abandoned, usually cutting off a subsequent correction.
• Larger agreement windows reduced but did not eliminate premature stops: the rate plateaued near 7% while token savings fell to 8%.