Act on Your Behalf, Beyond Your Authorization: Benchmarking LLM Agent Authorization Alignment
Zhiping Zhang ⋅ Zihao Huang ⋅ Franklin M Li ⋅ Tianshi Li
Abstract
When an LLM agent is delegated to act on behalf of a user, it is expected to act within the scope of authorization the user grants, a basic principle of agency. Yet authorization conveyed through natural-language instructions is often ambiguous, making it challenging for agents to reason about and stay within these boundaries. We present AuthorizationBench, the benchmark measuring how well LLM agents act within their delegated authorization scope, complementing prevailing consequence-oriented evaluation by capturing failures of delegation rather than only safety or privacy harm. Grounded in 42 real-world out-of-authorization cases, it comprises 84 tasks run in an authorization-aware sandbox that keeps both the task-completion and out-of-authorization paths open and observable. Across six frontier models and 504 trajectories, out-of-authorization behavior is prevalent, with rates ranging from 20.2% to 57.1%. Agents act out of scope even while failing the task and, for proprietary models, more readily when no safety or privacy harm is at stake ($p<0.01$). We further find that agents that turn to the user at least once before completing a task act out of authorization significantly less often than those that never do ($p<0.05$). Our results expose a blind spot in agent evaluation and motivate authorization alignment as a complementary target for agent development.
Chat is not available.
Successful Page Load