Counterfactual Privacy Audits for Language-Model Agents

Language-model agents increasingly act on private emails, calendars, messages, and records. A privacy failure need not copy a secret string: an action can reveal a diagnosis, salary, address, or family fact by implication. We study a simple audit question: after fixing the public context and the privacy norm, can an observer infer which protected value was present from the agent's final action? We introduce a counterfactual audit in which adjacent worlds differ in one protected value, generated actions are shown to a fixed judge, and leakage is measured as distinguishability between worlds with finite-sample estimates and confidence intervals. On a validated set of 130 agent-action counterfactuals, we find graded leakage: many cases are near chance, but a substantial right tail reveals the hidden value, often without verbatim copying. These results show that counterfactual distinguishability provides a practical, statistically grounded measure of privacy leakage beyond binary labels and string matching.