Adaptive AI Evaluation via Entropy-Stopped Bayesian Probing of Recurrent Agents

Fixed-budget evaluation is inefficient for interactive AI systems: some agents require many observations before their latent behavioural properties can be estimated reliably, whereas others can be characterised much earlier. We study adaptive AI evaluation in a controlled setting where ground-truth parameters are known by construction: recurrent agents performing two-armed bandit tasks. The evaluator observes only choices and rewards, maintains a Bayesian grid posterior over the Rescorla--Wagner (RW) learning rate $\alpha$ and inverse temperature $\beta$, and terminates evaluation when the joint posterior entropy falls below a predefined threshold. Across exact RW agents, mechanistic recurrent agents, and learned parameter-conditioned Gated Recurrent Unit (GRU) policies, we find that stationary task designs create a substantial parameter identifiability problem: learning rate $\alpha$ and inverse temperature $\beta$ produce similar behavioural patterns and are difficult to recover separately. Expected-information-gain designs improve recovery of $\beta$ but do not fully resolve underidentification of $\alpha$. In contrast, fixed-interval reversals that occur independently of agent behaviour generate diagnostic post-reversal signals that reveal how quickly agents adapt to changing reward contingencies. Combined with entropy-based stopping, this design reduces the mean evaluation length from 250 to approximately 118 trials while improving parameter recovery relative to fixed stationary baselines. Performance degrades only moderately in learned GRU agents, indicating robustness under mild model mismatch. These results suggest that entropy-stopped Bayesian probing provides a practical ground-truth validation step toward adaptive evaluation of larger black-box AI systems.