The Profit Alignment Problem: How Profit Mandates Induce Alignment Failures in LLMs
Abstract
This study shows that ordinary business language – "maximize profitability" – induces emergent negligence: LLMs systematically dismiss ambiguous signals of potential safety violations to serve business objectives. In 3,600 controlled trials across eight reasoning-capable LLMs, adding a profit mandate to otherwise identical prompts increases risk-dismissing judgments by 6.8 percentage points, suppresses board escalation recommendations by 13.9pp, and shifts severity assessments downward. The mandate never instructs models to downplay risks; instead, chain-of-thought traces reveal motivated reasoning – models acknowledge concerns, then invoke profit logic to justify dismissing them. We characterize these findings as the Profit Alignment Problem: when AI systems are given ordinary business objectives, they develop emergent strategies for suppressing inconvenient information that no designer intended or specified.