Efficient Rewarding of Agent Plans Without LLM-as-Judge: Goal Drift Score for GRPO

Large language models are increasingly used as agents that plan actions and select tools over multiple steps. We study goal drift: the valid but out-of-scope selection of actions that leave the user’s objective or authorization boundary. We frame the work around scoring efficiency and RL reward cost. We introduce GDS, a judge-free Goal Drift Score combining semantic distance, NLI contradiction/neutrality, and deterministic actionscope tracking. Because GDS avoids a large LLM judge, it can be used inside a GRPO-style reward. GDS-RL balances drift reduction with F1- style in-scope tool selection and uses LoRA adapters for larger models. On GoalDrift-Bench, a 200-task plan-level benchmark, GDS-RL reduces mean held-out action drift from 0.200 to 0.133 across seven models and increases mean tool-F1 from 0.721 to 0.788. The claim is intentionally narrow: the current evidence supports plan-level behavioral correction, not full environment success.