Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length
Abstract
Large language models are often assumed to benefit from longer context windows, yet retrieval from context requires not only locating the target information but also suppressing competing information tied to the same cue. Inspired by the proactive interference (PI) paradigm in cognitive science, we introduce PI-LLM, an evaluation in which interleaved key--value updates are streamed and models are queried to retrieve only the final value of each key, therefore isolating interference rather than search as the primary variable. Across 35+ models ranging from 0.6B open-weight models to frontier-scale proprietary systems, retrieval accuracy declines approximately log-linearly as repeated same-key updates accumulate, with errors dominated by outdated values. Similar log-linear degradation patterns emerge across multiple independent load dimensions and persist when total input length is held constant, pointing to a working-memory-like bottleneck in maintaining and retrieving the current binding under interference. This degradation is not alleviated by explicit `forget' instructions or chain-of-thought reasoning, and---unlike the plateau observed in human PI studies---shows no sign of leveling off, suggesting that LLMs' working-memory-like capacity, though large, lacks the flexible executive control that supports human resilience to interference.