Selection Efficiency: Auditing Coherence-Based Selection in Small-Scale Multi-Agent Reasoning
Manya Gupta ⋅ Shreeyam Bangera
Abstract
**Abstract:** A multi-agent reasoning system that beats a single-agent baseline is usually credited with two things at once: its agents make complementary errors, and its selector tells a correct answer from an incorrect one. These are rarely separated. We introduce *Selection Efficiency* ($SE$), a pool-relative normalization of the generation–verification gap that rescales oracle-minus-selected onto the interval between random selection and a perfect oracle, isolating the selector's contribution from task difficulty and pool diversity. We pair it with a decomposition splitting any multi-agent gain into sampling, correlated-error loss, and the selector's own net contribution. Using both, we audit four selectors — self-consistency, majority vote, LLM-as-judge, and NLI coherence scoring (ToTh) — across three sub-3B instruction-tuned models and four datasets (twelve configurations). The selectors fall into a stable order (self-consistency $>$ majority $>$ LLM-judge $\approx$ ToTh; majority $>$ ToTh in 10/12, sign test $p\approx0.04$). The two selectors that score the same style-prompted pool, LLM-judge and ToTh, both capture little headroom ($SE\le0.18$ for ToTh), and their edge over a single agent is almost entirely sampling. Self-consistency, which differs only in how its candidates are drawn, recovers far more of the same headroom on the same models — locating the constraint in the *candidate pool*, not any one scorer. A symbolic verifier reaches 100% on the cached traces where ToTh manages 48%, confirming the correct answers were present all along. Everything is inference-only and involves no training.
Chat is not available.
Successful Page Load