LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs

Existing evaluations of scientific discovery agents largely treat discovery as passive inference over fixed datasets, assuming that observations are given independently of the agent’s actions. This setting misses a central measurement challenge for interactive AI systems: in closed-loop discovery, the agent induces its own data distribution by choosing experiments, observing outcomes, and updating future decisions. We introduce ActiveSciBench, a benchmark suite for measuring active, budget-constrained scientific discovery with language-model agents. ActiveSciBench contains two queryable oracle datasets: ActiveSciBench-Chem, with 57 enzyme-kinetics tasks requiring symbolic rate-law recovery under hidden relevant variables, and ActiveSciBench-GRN, with 45 gene-regulatory-network tasks requiring signed causal graph recovery from perturbation-response experiments. Both settings evaluate sequential experiment design, path-dependent evidence accumulation, variable selection, and mechanism recovery rather than static prediction. As a case study, we propose LLM-AutoSciLab, a closed-loop framework that generates candidate mechanisms, selects experiments where hypotheses disagree, and refines mechanisms using acquired evidence. Across NewtonBench, ActiveSciBench-Chem, and ActiveSciBench-GRN, LLM-AutoSciLab outperforms passive, active-learning, and LLM-based baselines, achieving 67.6% symbolic accuracy on NewtonBench, 35.1% symbolic accuracy on ActiveSciBench-Chem, and 31.1% exact graph recovery on ActiveSciBench-GRN. It also requires 2–5× fewer oracle queries than the strongest competing baselines, showing that hypothesis-conditioned experimentation provides a rigorous and sample-efficient evaluation paradigm for interactive scientific agents.