Step-Level Optimization for Efficient Computer-Use Agents

Computer-use agents can automate interaction with arbitrary software interfaces, but strong agents remain costly and slow because they rely on large models at nearly every step. We observe that long-horizon GUI trajectories are heterogeneous: many steps are routine, while failures concentrate at a small set of high-risk moments, especially progress stalls and semantic drift. We propose an event-driven, step-level cascade that runs a small policy by default and escalates to a stronger model only when lightweight learned monitors detect elevated risk. Specifically, A Stuck Monitor triggers recovery when progress degrades, and a Milestone Monitor identifies checkpoints where sparse verification is most informative. Across OSWorld and WebArena, our framework improves the cost--quality trade-off, recovering much of the performance of always-large policies while reducing large-model usage, latency, and inference cost.