Do Small Models Know When to Stop? The Confidently-Wrong Trap and a Leak-Free Calibration Guard for Test-Time Stopping

Test-time scaling lets small models punch above their weight—but only if they stop thinking at the right moment. A popular, nearly-free way to de- cide is an intrinsic signal: token entropy, or a linear probe on hidden states that predicts whether the current intermediate answer is correct. These sig- nals were validated almost entirely on large, reasoning-post-trained models. We ask whether they survive at 0.5B–7B, across instruct and reasoning- distilled families, on GSM8K, MATH-500, and MMLU-Pro (7 models × 3 benchmarks; generate-once, evaluate-every-policy-offline). We report three findings. (1) The small-model failure is one of calibration, not discrim- ination: on math, probe AUC at 0.5B (0.67–0.74) matches larger sizes, but the confidently-wrong rate —the fraction of high-confidence interme- diate answers that are actually wrong—scales inversely with capability: a three-benchmark mean of 0.62 at 0.5B vs. 0.17 at 3B (bootstrap 95% CIs disjoint), reaching 0.80 (CI [0.50, 1.00]) at 0.5B on MMLU-Pro, where the probe is barely above chance. (2) Within-trace stopping is structurally inapplicable to small instruct models: 98–99% of their traces contain a sin- gle answer chunk, so every stopping policy degenerates to running in full. (3) On the reasoning models, probe-stop’s confident early exits are wrong up to 77% of the time; a calibration-aware guard removes them—but only when its calibration bins are estimated leak-free (cross-fitting) and aligned to the confident-stop gate. The naive in-fold estimate is a silent no-op, byte-identical to probe-stop in 21/21 runs. Temperature recalibration is no substitute: once honestly calibrated (T=3.3–10.8), the probe is almost never confident. The leak-free guard is the only policy with a safe (near- zero confidently-wrong) operating point, and at matched average budget it is consistently more accurate than naive fixed truncation (all six reasoning cells)—using only the generator’s own hidden states, no verifier. We release the pipeline.