Back to feed
arXiv cs.LG
arXiv cs.LG
7/31/2026
Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding

Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding

Short summary

This paper addresses the problem that converting MHA/GQA attention to Multi-head Latent Attention (MLA) for speculative decoding reduces draft-token acceptance due to attention-function errors. It proposes Functional Reconstruction, which optimizes converted MLA modules to reproduce original attention responses on calibration hidden states without verifier supervision. Across 192 configurations spanning Llama/Qwen pairs and multiple backends, the method materially improves acceptance in 37 of 64 matched task cells.

  • •Direct MHA/GQA-to-MLA conversion hurts speculative decoding draft-token acceptance
  • •Functional Reconstruction optimizes converted MLA modules to match original attention responses
  • •Improves acceptance in 37 of 64 task cells across 192 tested configurations, code released on GitHub

Generated with AI, which can make mistakes.

#research-breakthrough#open-source#ai-tools
Read full article at arXiv cs.LG

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

SpecLA: Efficient Speculative Decoding for Linear-Attention Models
arXiv cs.CL
Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
arXiv cs.CL
Transforming LLMs into Efficient Cross-Encoders via Knowledge Distillation for RAG Reranking
arXiv cs.CL
Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
arXiv cs.CL