DimPFN: Units as an Exact Symmetry for In-Context Learning of Physical Laws

Physical quantities carry units, and units encode an exact symmetry: rescale the kilogram and every mass-bearing quantity in a consistent equation rescales with it. Learned regressors discard this structure—they see raw numbers—so a model fit at one scale has no principled reason to hold at another, and it must rediscover from data that a relationship really depends on only a few dimensionless combinations of its inputs. We ask what a small in-context regressor gains by respecting the unit symmetry exactly. We build DimPFN, a tiny prior-fitted network that reads a few measurements of an unknown law together with the variables' units, nondimensionalizes through the Buckingham-$\Pi$ map inside its forward pass, predicts in the dimensionless quotient, and re-dimensionalizes—amortized across a distribution of laws, with no per-law training. Two things follow, and we separate them. First, a guarantee: the model is exactly equivariant to unit rescaling, so its error is invariant to input scale; it stays flat across six decades where a units-blind network and a Gaussian process degrade to meaningless within two. Second, a learned effect: because the dimensionless function lives on fewer coordinates than the raw inputs, DimPFN sidesteps the curse of dimensionality. In a controlled sweep that fixes the function's complexity and adds only units-removable nuisance variables, DimPFN's few-shot error stays flat while an identical units-blind network's grows, so the advantage rises with the number of dimensions the units remove ($2.6\times$ to $4.0\times$). Meta-trained only on synthetic dimensional laws, DimPFN transfers zero-shot to 29 real Feynman equations with units, where it is $\sim$$8\times$ more sample-efficient than both baselines.