LiveIdeaBench: Forecasting Emerging Research Ideas Against Future Literature

While large language models increasingly automate scientific ideation, their proposals are typically evaluated on immediate plausibility rather than whether the field actually moves there. We formalize research idea forecasting: given only literature published before a temporal cutoff, a system must produce ideas that later emerge in the published record. To evaluate this task, we introduce LiveIdeaBench, a temporally grounded benchmark of 67,656 arXiv papers across 52 machine-learning topics (2023–2025), scored against realized future literature under a reproducible retrieve-then-judge protocol. Our central finding is that what separates strong forecasters is how they organize the past, not how recent the abstracts they read: methods that compress broad historical context into structured summaries reach 61\% Hit@5, against 21\% for surface-trend extrapolation. We additionally release a reference forecaster (MDF) that decomposes each prediction into a latent innovation and a grounded realization. No method saturates the benchmark (best Precision@5 of 0.20), establishing research idea forecasting as an open problem for automated discovery.