No Harvestable Depth Redundancy in a Sparse-Mixture-of-Experts Block-Diffusion Language Model

Post-hoc per-pass compute reduction is the cheapest remaining acceleration lever for block-diffusion language models, and depth pruning is its most attractive form. Dense diffusion LMs tolerate layer removal far better than autoregressive models, with double-digit FLOP reductions at little quality cost. These results were established only on dense diffusion LMs. The frontier has shifted to lean \emph{sparse Mixture-of-Experts} block-diffusion models, which have already routed away much of the redundant width that depth pruning exploits. Does the depth headroom transfer? On a 26B-parameter, $\approx$4B-active sparse-MoE block-diffusion model served on vLLM, we find that it does not. We test depth redundancy at three granularities: whole-layer and MoE-sublayer Block Influence, and prune-then-distill recovery, judging quality on AIME, MMLU-Pro, and GPQA-Diamond against our already-deployed token- and step-axis accelerators with a multi-seed noise floor. No layer or MoE sublayer is near-identity (minimum Block Influence $0.052$), and skipping the eight-layer span the influence metric calls \emph{least} disruptive collapses reasoning (AIME $18.0\!\to\!4.7$ of $30$) even at $+62\%$ throughput. A converged LoRA recovery adapter \emph{degrades} the model rather than restoring it (recovery ratio $-0.50$), because a marginal-matching heal optimizes a target the multi-step denoise never exercises. The one post-hoc compute axis that yields is the expert-routing tail: a static top-8\,$\to$\,top-6 prune is lossless on all three benchmarks and orthogonal to both of our deployed accelerators, until it competes with the step-cap at top-5. The depth headroom is a property of \emph{density}, not of diffusion: on a lean sparse MoE the dominant-FLOP sublayer is the dominant-contribution sublayer, so only the temporally-stable routing head can be trimmed.