Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B

TL;DR - A controlled study finds that repeated answer sampling matches or outperforms self-refinement, reflection, debate, and selection methods when total generated-token costs are equal. This suggests many reported reasoning gains may come from extra inference compute rather than self-inspection itself.

• Evaluated seven methods across 1.5B, 3B, and 7B models on two 150-question mathematics benchmarks.
• None of 36 paired comparisons reliably beat repeated sampling at equal token cost; 10 were reliably worse.
• All 18 self-inspection comparisons were negative, with Self-Refine and forced Reflexion trailing by 3.6–10.1 points even at 7B.
• Model-based Best-of-N selection improved with scale, but still did not outperform majority voting; the smallest model’s Reflexion implementation never triggered a retry.