Back to feed
arXiv cs.CL
arXiv cs.CL
6/26/2026
Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars

Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars

Short summary

Researchers developed NEST-V1, a lightweight ML system translating Nepali speech into emotion-conditioned sign language avatars. With 81.1% speech recognition and 79.2% emotion detection across 600 labeled samples, the proof-of-concept uses only 22.1M parameters for edge deployment. This establishes a foundation for emotionally expressive sign language communication in low-resource languages.

  • •NEST-V1 converts Nepali speech to emotion-aware sign language avatars using a shared acoustic encoder
  • •Achieves 81.1% ASR accuracy and 79.2% emotion recognition with 37% parameter efficiency over baseline
  • •Proof-of-concept validates accessibility technology for hearing-impaired communities in underserved language markets

Generated with AI, which can make mistakes.

#research-breakthrough#ai-tools
Read full article at arXiv cs.CL

Is this a good recommendation for you?

Explore more

Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards
arXiv cs.CL
EmoInstruct-TTS: Dual-Path Instruction-Guided Emotional Speech Synthesis
arXiv cs.CL
MindAlign: Decoding Inner Speech from fMRI Signals via Multimodal Embedding Alignment under Limited Data
arXiv cs.CL
Continuous Audio Thinking for Large Audio Language Models
arXiv cs.CL