OrchOpt: Validation-Gated Architecture Search for Self-Organizing Multi-Agent Orchestration

Multi-agent architectures today are hand-designed, generated one-shot, or evolved through unconstrained code search—none of which treats the number of agents as a searchable parameter, and none of which can discover that a single agent is optimal. We argue agent count and coordination topology should be optimized with the same discipline applied to neural architectures: bounded edits, held-out validation gating, and bidirectional search that can both add and remove agents without information loss. OrchOpt is a validation-gated architecture search over a unified design space where single-agent and multi-agent configurations compete directly—with bidirectional traversal enabled by knowledge preserving merge. A proposer model turns failure analysis into bounded merge/split/skill edits on the current architecture, and an edit is accepted only when it strictly improves validation. Knowledge preserving merge where absorbed agents’ expertise becomes activatable skills—makes the search fully reversible at zero additional inference cost. Across nine benchmarks spanning math, knowledge QA, code generation, reading comprehension, multi-hop reasoning, negotiation, and research: the optimizer simplifies to 1 agent on 5 domains and elaborates to 2–3 agents on 4 domains. It finds architectures scoring +20pp over ADAS on MGSM, +16.7pp over AFlow on HotpotQA, and +27.5pp over MARBLE’s published-best topology on Bargaining— converging in 7–42 candidate evaluations.