Interpreting Attention Sink Neurons in Large Language Models

We investigate attention sinks in large language models through \textit{attention sink neurons}: a sparse set of neurons that activate selectively on tokens which attract disproportionately large amounts of attention. To establish their causal effect, we introduce a simple inference-time intervention that transfers the activation of these neurons from the original sink token to an inserted dummy token and reliably relocates attention sink behavior while largely preserving perplexity across a diverse set of model families. We further show that these neurons emerge alongside attention sinks during training and find evidence consistent with an attention ``No-Op'' interpretation of sink behavior. Together, our results provide causal evidence that attention sink neurons play a central role in mediating attention sink behavior in large language models.