Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separation
Abstract
Hybrid-thinking language models expose explicit \think and \nothink modes, but current designs do not separate them cleanly. Even in \nothink mode, models often emit long and self-reflective responses, causing reasoning leakage. Existing work reduces this issue through better data curation and multi-stage training, yet leakage remains because both modes are still encoded in the same feed-forward parameters.We propose Path-Lock Expert (PLE), an architecture-level solution that replaces the single MLP in each decoder layer with two semantically locked experts, one for \think and one for \nothink, while keeping attention, embeddings, normalization, and the language-model head shared. A deterministic control-token router selects exactly one expert path for the entire sequence, so inference preserves the dense model's per-token computation pattern and each expert receives mode-pure updates during supervised fine-tuning.Across math and science reasoning benchmarks, PLE maintains strong \think performance while producing a substantially stronger \nothink mode that is more accurate, more concise, and far less prone to reasoning leakage. On Qwen3-4B, for example, PLE reduces \nothink reflective tokens on AIME24 from 2.54 to 0.39 and improves \nothink accuracy from 20.67% to 40.00%, all while preserving \think-mode performance. These results suggest that controllable hybrid thinking is fundamentally an architectural problem, and separating mode-specific feed-forward pathways is a simple and effective solution.