Back to feed
arXiv cs.CL
arXiv cs.CL
7/23/2026
Multi-Mask Diffusion Language Models for Few-Step Generation

Multi-Mask Diffusion Language Models for Few-Step Generation

Short summary

MultiMDM extends masked diffusion models by pushing each clean token toward a designated mask before mixing over the mask set, enabling drafting capability in the backward process. The authors derive a closed-form ELBO supporting continual training from pretrained MDMs and introduce a discrete-state consistency distillation with shared-Gumbel coupling. Experiments show MultiMDM provides an effective foundation for principled few-step generation.

  • •MultiMDM preserves masking structure for few-step generation in diffusion LMs
  • •Closed-form ELBO supports continual training from pretrained MDMs
  • •Discrete-state consistency distillation with shared-Gumbel coupling reduces pathwise entropy

Generated with AI, which can make mistakes.

#research-breakthrough
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL
arXiv CS.AI
Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
arXiv cs.CL
Token Time Continuous Diffusion for Language Modeling
arXiv cs.CL
SLIM-RL: Risk-Budgeted Random-Masking RL for Diffusion LLMs Without Trajectory Slicing
arXiv cs.CL