Noise Your Prompt: Prompt Noising for Conditional Generation in Continuous Diffusion Language Models

We revisit a standard accepted practice in the continuous diffusion language model literature of fixing conditioning prompt tokens clean during training. Rather, we make a very simple modification: also noise the conditioning prompt tokens during training. We demonstrate that under this modified training objective, we achieve better generalization in combinatorial reasoning tasks such as Sudoku and N-Queens, with significant improvements on harder variants ($3.73 \to 24.25 $ solve rate on Sudoku Hard). Moreover, our models exhibit higher generation diversity. Our method can be seen as conditioning augmentation and a generalization of classifier-free guidance training.