UGTC: Uncertainty-Gated Temporal Credit — Adaptive Advantage Estimation for Continually Learning Agents

Continually learning agents require credit assignment mechanisms that adapt as the agent's internal model improves. The standard Generalized Advantage Estimator (GAE) fixes a single bias-variance tradeoff ($\lambda$) for every state and training phase—a static choice mismatched to the non-stationary nature of lifelong learning. We introduce UGTC, a plug-in advantage estimator that replaces this fixed tradeoff with a state-dependent blend of a fast critic ($\lambda=0.80$) and a slow critic ensemble ($\lambda=0.99$, $M=3$), gated by ensemble disagreement. The gate provides a self-monitoring signal: it automatically favors low-variance estimates in uncertain states and high-variance estimates where the ensemble has converged, enabling the agent to continuously adapt its credit horizon without meta-optimization. UGTC composes with PPO, TD3, SAC, and DreamerV3, adding <15% parameters. Across six benchmarks (64 tasks, 10 seeds, bootstrap 95% CIs), UGTC-PPO converges 2.7x faster on Hopper (AUC 0.79 vs. 0.62); UGTC-DreamerV3 achieves 2.4x on MetaWorld ML45 (45 tasks); UGTC-PPO improves Procgen Hard by +19.9%. We report two failure cases—Crafter (-23.3%) and Ant peak return (-14%)—with deep analysis, and provide a practical applicability criterion. A systematic ablation including a fixed-$\lambda$ baseline confirms that adaptive gating, not static blending, drives the gains.