Formalize Once, Solve Many: When to Write Code Instead of Reasoning Per Case on Rule-Governed Tasks

A language model facing a rule-governed task, say computing a baggage fee or filing a tax return, can reason through each case, or formalize the rules once into a program and run it on every case. The first spends test-time reasoning compute on every case; the second pays that cost once and reuses it. We ask when the second choice is the right one, and why. On RuleArena, a benchmark of realistic business rules where the difficulty is composing a fixed ruleset rather than recalling a formula, a deterministic oracle labels every case, so a generated program’s accuracy is exactly its formalization fidelity. We compare both strategies on the same few-shot examples and test every gap with a paired McNemar test. Across two rulesets (airline baggage fees and U.S. Form 1040 tax) and five frontier models, formalize-once is never significantly less accurate than per-case reasoning (in no cell does plain significantly win) and is 10–50×cheaper, because one program amortizes over the whole test set. The result is equal accuracy at far lower cost. At base difficulty most cells tie; the gain concentrates where a capable formalizer reasons unreliably. Sonnet 5 is the clearest case (0.98 vs. 0.41, a +0.57 gain), and it stays at +0.22 and significant even under matched self-consistency sampling. The gain is not confined to that cell: on harder rule levels it is larger for other models too, tracking task difficulty. When the model can write the program, formalize-once Pareto-dominates. Every label is recomputed by a deterministic oracle rather than stored, and the oracle labels any structured input, so the comparison stays reproducible and extensible to fresh cases as models evolve.