RLEM: Reinforcement Learning for External Memory

Position-interpolation methods extend LLM context windows but remain bounded, limiting continual information accumulation and reuse. External- memory systems can persist information beyond the active context, yet pretrained LLMs often interact with such systems unreliably when read and write decisions is induced without training. We propose Reinforcement Learning for External Memory (RLEM), which uses a minimal memory sys- tem consisting of an in-context buffer and an external key–value store with two operations, PUT and GET. It fine-tunes the model with outcome-based reinforcement learning to improve memory interaction reliability under a fixed context budget. On RULER-HotpotQA with a 8K context window, RLEM maintains strong accuracy as task-relevant evidence extends beyond the context window: at 896K tokens in the query-aware setting, RLEM improves +14.0% over the same memory system without training (70.3% vs 56.3%); at 112K tokens in the query-unaware setting, it improves 25.4% (57.0% vs 31.6%).