MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Data

Short summary

MindAlign enables direct text generation from fMRI brain signals using a two-stage neural-semantic alignment framework. The approach first maps subject-specific brain activity to a shared semantic space, then prompts a frozen multimodal language model for free-form generation. Results outperform fMRI-only baselines and demonstrate cross-subject generalization potential for scalable brain-computer interfaces.

• •Two-stage decoupled framework: neural alignment + language model prompting
• •Subject-specific alignment generalizes across participants for scalable deployment
• •Outperforms fMRI-only and random baselines on silent image description task

Generated with AI, which can make mistakes.