Back to feed
arXiv cs.CL
arXiv cs.CL
7/24/2026
Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing

Moir: Let the Model Direct Its Own Story for Robust Cross-Domain Knowledge Editing

Short summary

Moir addresses the problem that knowledge editing in LLMs degrades reasoning capabilities like math and coding while preserving encyclopedic recall. It traces this to distributional mismatch between reference corpora and the model's post-training operative distribution. Moir estimates preservation covariance directly from the model's own decoding distribution, requiring no external data. On Qwen3-8B after 20K batch edits, it retains 79.9% GSM8K accuracy versus 10.9% with the Wikipedia baseline.

  • •Moir preserves reasoning capabilities during knowledge editing by sampling from the model's own decoding distribution
  • •Requires no external data; works as drop-in for any covariance-based editor
  • •Retains 79.9% GSM8K accuracy on Qwen3-8B after 20K edits vs 10.9% with Wikipedia baseline

Generated with AI, which can make mistakes.

#research-breakthrough#open-source
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Comments

Failed to load comments. Please try again.

Explore more

Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations
arXiv cs.CL
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
arXiv cs.LG
Self-Generated Error Training for Token Editing in Diffusion Language Models
arXiv cs.CL
MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference
arXiv cs.LG