Controlling for Feature Frequency in Sparse Autoencoder Steering of Mathematical Reasoning

Sparse autoencoders (SAEs) provide a feature-level view of language-model activations, but interpretable or decodable features are not necessarily causally faithful for model behavior. This probe-causality gap is especially important in internal reasoning, where many signals may correlate with successful or failed reasoning, with no way of supporting reliable interventions. We therefore study whether SAE feature interventions in mathematical reasoning traces remain meaningful under stronger controls. We generate multiple GSM8K rollouts and focus on mixed-outcome problems, rank SAE features by their causal effect on gold-answer likelihood, and compare them against matched controls that preserve activation frequency and decoder-vector norm. Causally ranked features ($+0.00349$ mean $\Delta\log p_{\mathrm{ans}}$) outperform random controls, but activation-frequency matched controls ($+0.00443$) achieve equal or larger effects even without any task-specific selection. We frame SAE steering as a useful diagnostic for reasoning traces while emphasizing the need for matched controls before attributing feature interventions to reasoning-relevant mechanisms. Our code will be released upon acceptance.