The headline needs to capture: AMD, Instella-MoE-16B-A3B, open-source, MoE LLM, 2.8B active parameters, trained on Instinct GPUs.

Short summary

AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts LLM with 16B total parameters and 2.8B active per token, trained on Instinct MI300X and MI325X GPUs. AMD published weights from all training stages along with data mixtures, configs, and inference code. The announcement is brief with no benchmark results or performance comparisons.

• •AMD open-sourced Instella-MoE-16B-A3B: 16B total params, 2.8B active per token
• •Trained from scratch on Instinct MI300X and MI325X GPUs
• •All training-stage weights, data mixtures, configs, and inference code published

Generated with AI, which can make mistakes.