StateBind: Evaluating Executable Agent State Across Context Windows and Handoffs
Xiaonan Fu
Abstract
Long-context and memory-augmented agents increasingly resume software work across windows, sessions, and handoff artifacts. We study a failure mode that relevance-oriented evaluation can obscure: the correct executable handle may be visible while the target--role--handle relation is absent, ambiguous, or not successfully consumed. We formalize executable state binding and introduce StateBindBench, a public same-log diagnostic that measures handle visibility, explicit relation availability, and exact downstream consumption separately. Across 792 synthetic cases built from realistic public GitHub seeds, direct annotation shows two distinct regimes. Identifier reranking often exposes the gold handle without the full relation; by contrast, BM25-$k{=}15$, parent--child, and agentic-RAG contexts contain the annotated relation in every case yet still produce wrong-role or wrong-record exact-ID errors. The latter are consumption or selection failures, not evidence that the relation is literally missing. Structured tuple controls are reliably consumed once correctly constructed, while wrong-slot and shuffled-ID controls fail. A 300-case public OpenHands proxy supplies a conservative ambiguity-adjusted action-level bridge. The contribution is a focused diagnostic invariant and failure decomposition, not a universal memory module, causal prevalence estimate, or claim that tuple formatting alone solves handoffs.
Chat is not available.
Successful Page Load