Single Canonical Prompts Underestimate LLM Safety's Surface-Form Sensitivity

A benchmark score is a measurement instrument, yet most benchmarks read each item at a single canonical surface form. We treat single-prompt safety evaluation as such an instrument and ask two coupled questions: a construct-validity question—does the canonical-form score faithfully estimate behavior when an item's harmful intent is held fixed and only its meaning-preserving surface form varies?—and a reliability question—how much of any variation is decoding/judge noise rather than signal? To answer them without prior confounds, we pre-author the reformulations (refusal-free, mostly non-LLM: machine back-translation and a Matrix-Language-Frame code-switch generator) so an identical surface form reaches every model, score all responses with one human-anchored, vendor-neutral judge (Claude—κ = 0.86 vs. human on unsafe compliance, stable across languages, cross-checked by GPT-4o), and verify intent preservation. On 370 seeds × 5 surface forms × 5 models, no single transformation is uniformly most dangerous (6 of 20 per-transformation McNemar tests survive correction, most protective). Yet reading only the canonical prompt underestimates unsafe compliance: the union of unsafe outcomes across forms exceeds even the worst single form by 3.3–12.9 pp (bootstrap 95% CIs exclude zero for all five models), and 5–13% of seeds safe on canonical are unsafe under some reformulation—above a zero stochasticity floor (canonical resampled five times at temperature 0 gives 0/370 new exposures). The canonical prompt is thus a biased, optimistic estimator of a model's unsafe surface, and the gap is model-dependent (largest on Gemini 2.5 Pro). One form recovers only ~53% of a model's observed unsafe surface; about three reach 85%. A benign control (XSTest) suggests the instability is bidirectional. We release the dataset, code, and per-response labels.