Chain-of-Thought Depth Modulates Performance by Task and Model: Evidence from Ablation Studies Across Five Model Families

Chain-of-thought (CoT) prompting is widely assumed to be monotonically beneficial, yet recent work shows it can reduce accuracy on implicit-pattern tasks. We introduce a three-condition depth manipulation (zero-shot, short CoT, long CoT) and evaluate five configurations of four model families across artificial grammar learning (AGL) and natural language inference (NLI). Over 68,872 evaluations, we report eight findings. (1) Three architectures produce three qualitatively different depth-accuracy patterns on grammar: monotonic improvement, gradual decline with zero response failures, and response-generation collapse. (2) A task×depth crossover emerges within individual models, confirmed across two independent families. (3) We decompose CoT failure into mechanistically distinct response-generation failure and reasoning failure. (4) Ablation studies show the collapse is a generation-budget artifact that disappears at 4,000 tokens and is prompt-specific. (5) These findings replicate on 150 held-out NLI pairs. (6) Item difficulty moderates CoT degradation: easy grammars suffer the largest drop (−15.8 pp) while hard grammars are relatively spared (−3.7 pp). (7) Cross-model error agreement drops below chance under deep CoT (42.3%), indicating model-specific failure mechanisms. (8) Cross-temperature consistency collapses from 79.4% to 2.8% under long CoT, revealing that deep reasoning on implicit tasks is not only less accurate but fundamentally unreliable. For efficient reasoning systems, short CoT is the strictly dominant strategy on implicit-pattern tasks across accuracy, cost, and reliability.