Diagnosing Agent Capabilities through Information-Axis Knockouts

Agent evaluations typically report aggregate success rates, implicitly treating task completion as a proxy for capability. However, success in interactive environments depends on multiple underlying information sources ( observations of the world state, understanding of environment mechanics, and access to feedback) and a model's performance may be limited by deficiencies in any single channel. We introduce a diagnostic framework that decomposes task specification into three information axes (World, Mechanics, and Feedback) and measures agent robustness through single-axis knockout ablations. We instantiate the framework in three ARC-AGI-3 environments and evaluate GPT-5.2 and DeepSeek-V4-Pro with and without explicit reasoning. World knockouts collapse win rates to near zero across all models and environments. Mechanics knockouts are environment-dependent, ranging from full recoverability on BP35 to complete collapse on KA59-Simple. Feedback knockouts are largely survivable, suggesting agents reconstruct action consequences directly from the observation stream. These findings suggest that aggregate success rates alone are insufficient for diagnosing the sources of agent competence and failure, and that benchmark performance reflects an interaction between model capability and the information structure of the task.