Measuring Prompt-Injection Defenses for Computer-Use Agents: Silent Failures, Landed Attacks, and the Block– Confirm Frontier

How should we measure whether a prompt-injection defense for a computer-use agent actually works? We argue that the usual construct, Detection Success Rate (the fraction of attack cases hard-blocked), is the wrong target; the valid target is silent-failure rate: a landed attack that the defense nonetheless allows. This reshapes measurement: the denominator should be landed attacks, meaning attacks the unguarded agent actually executes; escalation to a human should be scored separately from hard blocking; and single-model attack-success numbers must be pinned because they shift under model-generation drift. We instantiate this protocol on ASP-Guard, a per-slot typed runtime monitor, using per-case landed-attack inspection across two benchmarks and several models to show what aggregate DSR hides. Across all four visual VPI-Bench surfaces (126 attack cases, 41 landed on gpt-4o-mini in the pinned reference run) and tool-calling AgentDojo, ASP-Guard blocks or escalates every landed attack with zero silent failures, while achieving the highest caught-or-escalated rate among baselines that preserve benign utility. On the browser set, it reaches hard-block DSR 0.77 while holding benign hard-block FPR to 0.08 on the same model, or 0.23 strict FPR when human escalations are counted, far below untyped global-suppression baselines at 0.56. The gain comes from typing: an untyped rule that ties ASP-Guard on one attack family (DSR 1.00) collapses to 0.33 as the surface diversifies, while the typed predicate does not. Against a CausalArmor-style attribution baseline, per-slot typing also removes silent allows left by the untyped rule. Open-weights models confirm the pattern: the typed-vs-untyped gap reproduces on AgentDojo, most clearly on Llama-3.3-70B, and on VPI-Bench’s 26 landed open-VLM attacks, ASP-Guard silently allows 1 versus the untyped rule’s 5.