Can a Lightweight Multimodal Model Estimate LLM Reasoning Performance? A Study for Compute-Optimal Document Inference

Modern frontier LLMs expose controllable reasoning budgets at inference time. However, allocating these budgets uniformly across all inputs is both expensive and often counterproductive, a phenomenon commonly known as the over-thinking penalty. Frontier providers offer in-flight controls to mitigate this, such as Gemini's dynamic auto-thinking, but these mechanisms rely on the model's own self-assessment of difficulty. Recent work shows this is insufficient: 32\% of nominally cheaper model pairs cost more in practice due to thinking-token overrun, with up to 9.7$\times$ per-query cost variance for identical prompts Chen et al., 2026. For document tasks, where difficulty is driven by visual layout, table structure, and multi-modal content, a pre-flight estimator that reads the document directly is better positioned to make this judgment than a model reasoning in-flight. Resolving this requires an external pre-flight estimator that can predict, per sample, how well a given model will perform at each budget level before committing to expensive inference. In this work, we explore if a lightweight multimodal model can accurately estimate the reasoning performance of a much larger LLM across variable reasoning budgets. To study this, we introduce BudgetDoc, the first multimodal benchmark providing explicit supervision for model-budget-performance trade-offs across three document-centric tasks. We train DRB (Document-Reasoning Balancer), a $\sim$1B-parameter estimator (SigLIP-2 + Qwen3-0.6B) that predicts a 7-class ordinal performance label for any (document, prompt, model, budget) configuration, achieving a weighted F1 of 0.753 on the BudgetDoc test set. We then apply DRB's estimates to optimize reasoning budgets across five frontier models and three datasets: in 9 of 15 model-dataset configurations, DRB matches or improves F1 over always-maximum-budget inference while mostly reducing cost. As a secondary case study, we probe whether the same estimator can generalize to model selection across intra- and inter-family configurations, finding encouraging preliminary results and identifying dedicated multi-provider training as an important direction for future work.