How Should Code LLMs Learn Execution? A Study of Trace Representations

Large language models can generate code but often fail to track intermediate execution states faithfully. We study Python execution tracing as structured prediction using datasets derived from OpenCoderCorpus, TACO, CRUXEval, and LeetCode-style problems. We compare answer-only, diff-based, and full-trace supervision in in-context learning and LoRA fine-tuning of Qwen3.5 and Apertus models. Few-shot performance rises quickly but saturates beyond $k\approx5$. Full traces most reliably improve execution-state fidelity, whereas compact diff traces are inconsistent despite their lower sequence cost. Fine-tuning is model-dependent: Apertus-8B benefits in several settings, while Qwen3.5 can improve surface accuracy without reliable changed-state tracking. These results frame trace representation as an efficiency--fidelity trade-off for program reasoning.