Trace-Use Audits as Measurement: Interventional Evaluation of Post-Training Trace Reliance

Reasoning traces are now used as distillation targets, preference objects, on-policy prefixes, outcome-reward scaffolds, and monitorable evidence of reasoning. For evaluation, however, trace fluency is a weak construct: a model can produce plausible traces without using supplied trace evidence when its answer should change. We frame trace-use auditing as a measurement problem. A behavioral audit intervenes on the trace channel---supplying helpful traces, masking problem text, corrupting steps, inserting wrong hints, creating trace-answer conflicts, or starting from wrong prefixes---and asks whether the final answer moves in the construct-valid direction. Under deliberately bounded settings spanning controlled arithmetic, public multistep ProofWriter, GSM8K rationale audits, Qwen2.5/Mistral LoRA adapters, and 7B stress checks, common post-training signals cover different measurement cells. Trace-SFT and outcome reward can preserve helpful-trace competence while leaving direct bad-trace cells uncovered; on-policy distillation (OPD) and DPO-style signals can reduce some bad-trace following while losing helpful-trace competence; GRPO/RLVR is strong on open and prefix behavior but weaker on direct conflict cells. Trace-use-aware coverage is therefore an oracle diagnostic and construct-validity probe, not a deployable recipe or complete mechanistic explanation.