DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions

Short summary

DecodeShare identifies a low-dimensional subspace shared across tasks in LLM decode-time hidden states and demonstrates its causal role by selectively removing it during decoding. Disturbing this shared subspace degrades performance more than prefill-derived or random subspaces. The work shows practical implications for activation steering, where projecting out the shared subspace improves steering vector reliability for downstream deployment.

• •Identifies a shared low-dimensional subspace in LLM decode-time states
• •Causal experiments confirm its critical role in task performance
• •Practical implications for improving activation steering reliability

Generated with AI, which can make mistakes.