Look Before You Steer: Geometry Predicts SAE Feature Steerability

Steering with SAE features requires per-feature coefficient tuning, which currently demands intervention sweeps. We ask whether properties of the SAE itself, computable before any forward pass, predict which features will be cheap or expensive to steer. We show that variation in SAE feature steerability is partially predicted by decoder-space geometry: neighbor density and maximum cosine similarity to nearby decoder directions — both computable from the SAE weight matrix before any intervention — rank features by how much steering they require for a fixed behavioral effect ($\rho$ up to $-0.546$, $p < 10^{-6}$, AUROC 0.610–0.822 across conditions; the signal is rank-based, consistent with grid discreteness). Neighbor density is the single strongest predictor in the well-identified Gemma-2 conditions ($\rho = -0.546$ at 9B layer 20, where censoring is only 3%), while maximum cosine similarity is the more robust signal cross-architecturally ($\rho = -0.266$ on Llama-3.1-8B at layer 15). This geometry–steerability relationship replicates across two Gemma-2 model scales (2B and 9B), two SAE widths (16K and 65K), and is detectable cross-architecturally on Llama-3.1-8B-Instruct ($\rho = -0.266$, $n = 300$). On Qwen3-8B with BatchTopK SAEs, geometry predicts whether a feature is steerable at all but not the continuous ordering among responsive features, revealing a boundary condition tied to SAE training regime. The signal weakens at deep proportional layer depth in both models, where the cost of steering exceeds our intervention budget — a consistent depth boundary. These results provide preliminary evidence that, given a candidate set of behaviorally-relevant SAE features, pre-steering geometry can partially inform coefficient selection. We frame this as a proof-of-concept for the geometric hypothesis rather than a deployment-ready screening tool: validation is restricted to a single behavioral domain (refusal) on one benchmark.