Below the Reliability Floor: Recovering True Agent Success from Judge-Gated Retry Loops

**Accepted paper — no in-person presentation.** This paper remains accepted. No in-person poster presentation is scheduled. LLM judges are increasingly placed inside an agent's loop, scoring the agent's own attempts and re-prompting until one passes. We show this quietly corrupts measurement—a pattern we first observed in a deployed compliance verifier, where a violation that slips the gate is a real harm, and then made exact on real agent loops with objective ground truth. Across $44$ capable-agent loops on GSM8K, MATH, and code (no authored weakness), the inflation is systematic—median $+0.16$, largest on code where a judge that cannot run the candidate rubber-stamps plausible programs (in the worst case a true $0.76$ reported as $0.99$)—and it is governed by a closed-form law that predicts the slip from per-attempt statistics, both across configs and within each task (pooled $r=0.95$, within-task mean $r=0.86$). We trace this to optional stopping: retry-until-PASS keeps drawing until a noisy judge slips, so $45$–$73\%$ of apparent retry gains on a fallible agent are judge errors rather than real fixes—judges rubber-stamp what they cannot verify. The fix is a calibrated de-biasing correction (the Rogan–Gladen prevalence estimator) whose reliability is governed by a single judge-discrimination coefficient $J$ (Youden's index) that retrying only pushes lower, toward a reliability floor; reporting $J$ alongside the corrected rate recovers true success up to $11\times$ more accurately than the raw rate where the bias is large, and cuts recovery error $\sim4\times$ (MAE $0.04$ vs. $0.16$) across the capable-agent loops. On near-balanced re-judgment gates the per-gate gain is within noise, so we report the reliability $J$ and the calibrated bias—not a single point estimate—as the deliverables. We release all code and content-free data.