Where Does Persistent Memory Live? The Optimizer, Not Only the Architecture, Decides

Fixed-state sequence models—linear attention and state-space models (SSMs)—replace the unbounded KV cache of softmax attention with a constant-size recurrent state, and their persistent memory beyond the context window is usually summarized spectrally: a decay gate sets the recurrence's largest eigenvalue magnitude $\rho$, and $\tau_{\mathrm{mem}}=-1/\log\rho$ is read as the retention horizon. We test this view with one substrate-agnostic currency, recoverable information $R(d,B,C)$: the bits of a known needle planted at distance $d$ that a frozen probe recovers from internal memory under memory budget $B$ and compute $C$. We find: (i) the optimizer is a second, training-time knob on retention—with architecture and data fixed, weight decay monotonically shortens the learned $\tau_{\mathrm{mem}}$ ($4.13 \to 2.37 \to 1.05$ as weight decay grows $0 \to 0.1 \to 0.5$ on a diagonal SSM), and it even decides whether the task is learned; (ii) the decay spectrum does not predict realized retention—among trained Gated DeltaNet-2 models that all retrieve perfectly ($R \approx 3$ bits at every distance), $\tau_{\mathrm{mem}}$ spans $1.3$–$55$ with Pearson correlation $-0.46$ to $R$; and (iii) causal mechanism knock-outs show which mechanism carries the memory is optimizer-dependent—decay-driven under Muon, write-gate-driven under AdamW, and fully redundant under Adafactor. Persistent memory in these models is therefore placed by optimization, not fixed by the architecture's spectrum. We release a small reproducible harness that measures all of this in $R(d,B,C)$.