Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
Devesh Tiwari ⋅ Camille Davis ⋅ Shivank Sinha ⋅ Talia Weaver ⋅ Aditya Shah ⋅ Maheep Chaudhary
Abstract
**Accepted paper — no in-person presentation.**
This paper remains accepted. No in-person poster presentation is scheduled.
Linear probes can decode safety-relevant concepts such as truthfulness from language-model activations, but high probe accuracy shows only that a concept is decodable, not that the features the probe weights causally drive the model's behavior. We show that this gap cannot be closed from the probe's weights alone: the features a probe is geometrically aligned with need not be the ones the model uses, so causal relevance requires intervention. We introduce a feature-level diagnostic that decomposes a deployed *True*/*False* probe into sparse-autoencoder (SAE) features, ranks those features by both probe alignment and by gradient sensitivity of the model’s behavior, and ablates the resulting shared, probe-only, and random feature sets under a coherence gate. On the truth probe of Burger et al. (2024) (TTPD), applied in the instructed truth/deception setting of Long et al. (2025) for Gemma-2-9B, the two rankings overlap only weakly (about 12%, Spearman $\rho = 0.10$), and ablation dissociates them sharply: features the probe shares with the model flip the output far more (up to 27%) than equally sized probe-only (6%) or random (1%) features at full coherence, while probe-only features instead perturb the probe's own readout. The dissociation holds across five seeds and a held-out split, and an activation-aware selection of features flips behavior nearly three times as often as the probe's geometric top features (17.6% vs. 6.1%). In this setting, therefore, a probe's weight vector alone does not identify the features the model causally uses; coherence-gated SAE intervention is needed to separate behaviorally causal features from probe readouts.
Chat is not available.
Successful Page Load