Representation Steering for Emotions in Diffusion Language Models

Diffusion Language Models (DLMs) have emerged as a promising alterna- tive to Auto-Regressive (AR) Language Models due to their higher token generation speed. Several recent DLMs have been released which are rapidly closing the gap in performance compared to Auto-Regressive LMs. This opens up many new applications for DLMs where speed and efficiency are critical. However, very few works have explored understanding the in- ternal circuitry in DLMs. In this work, we use carefully crafted contrasting prompts to extract emotion steering vectors from model activations of two DLMs (LLaDA-8B-Instruct and Dream-Instruct-7B), and demonstrate the utility of these vectors in imposing target emotions during generation.