The Coverage Tax of Concise Reasoning: A pass@k Audit of Efficiency Methods

Reasoning-efficiency methods promise a near-free lunch: cut chain-of-thought (CoT) length by half or more with little accuracy loss. That promise is almost always measured with pass@1. We ask what this single ruler hides. We run a systematic cross-mechanism pass@$k$ audit of the efficiency-method family (length-penalty RL, iterative CoT pruning, elastic reasoning, and verification fine-tuning), evaluating publicly released checkpoints against the uncompressed reasoning ancestors they were trained from. Two findings emerge. First, pass@1 does not predict pass@$k$ ranking: on competition math the checkpoint with the best pass@1 is out-ranked at pass@256 by several more-compressed models with lower pass@1 (Kendall $\tau = 0.27$). Second, the coverage cost is dose-dependent: pooled over 460 hard problems, aggressive length-penalty compression (65 to 70% fewer tokens) loses 4 to 5 points of coverage relative to its uncompressed parent, whereas moderate pruning (30 to 40%) preserves coverage. This is not an oracle artifact: the same ordering holds under actual self-consistency (maj@$k$) and under fixed generated-token budgets, and answer-distribution entropy tracks it. We argue efficiency methods should report a coverage metric alongside pass@1 and token cost.