Behavioral Game Theory for Collaborative LLM Agents

Multi-agent systems, from cooperative robots to teams of large language model (LLM) agents, increasingly must collaborate with partners they were not trained with. Existing approaches to learning collaborative policies, however, produce brittle solutions that fail when paired with new partners. We attribute these failures to two coupled causes: free-riding during training, a socially harmful failure mode where agents learn to delegate costly effort to their partners, and a lack of strategic robustness to deviations in a partner's behavior. We argue that strategic risk aversion, a principle from behavioral game theory, where risk aversion and bounded rationality together best predict how people actually play games, is a principled and scalable inductive bias that addresses both. Building on the risk-averse quantal response equilibrium (RQE), an agent is robust to partner deviations by design. Remarkably, we further prove that in structured collaborative games, strategic risk aversion brings a "free lunch" that also increases shared reward and eliminates free-riding equilibria, so robustness need not sacrifice performance. We translate RQE into Strategically Risk-Averse Policy Optimization (SRPO), a drop-in modification of independent PPO with a convergence guarantee. Empirically, SRPO eliminates free-riding and improves robustness to heterogeneous and unseen partners across collaborative benchmarks; most notably, in a multi-agent LLM debate on GSM8K, SRPO substantially improves cross-play accuracy over independent PPO, demonstrating that a behavioral-game-theoretic inductive bias transfers to teams of language-model agents.