Algorithmic Primitives of Theory of Mind Benchmarks

A theory of mind (ToM) benchmark conditions test takers by supplying sequences of events as context. Human language often anchors its reference to an event sequence through fixing semantics of the list of verbs that appear in each event description. This implies a sequence of algorithmic state changes solvers must simulate to reliably obtain the results. Benchmark papers name the capacity they intend to measure and leave such encoded algorithm implicit. This matters because successive benchmarks vary agents, modality, and dialogue while re-encoding the same few state transition algorithms; diagnostic results on those benchmarks are thus compressible to common symptoms. This paper recovers such algorithms and upstream inductive biases of ToM benchmarks explicitly. PDDL-Mind parses narratives into the Planning Domain Definition Language and executes candidate actions against a predefined set of update algorithms, yielding a verified state trace. We use the performance gain of providing such parsed states to the model to audit how much of a benchmark's difficulty our inductive bias absorbs. We further hypothesize that Large Language Models' failures on complex ToM benchmarks primarily arise from the language models' unreliable implicit implementation of those state tracking algorithms rather than limitations in belief and counterfactual reasoning. PDDL-Mind is also efficient: state execution is offloaded to a deterministic symbolic verifier, so the full pipeline needs only three LLM calls per example, far fewer than agentic ToM methods that average tens of calls. Across MMToM-QA, MuMa-ToM, and FanToM, parsing alone brings performance close to strong agentic pipelines, while verified state tracking moves GPT-4o to close-to-human performance.