Prime Intellect Releases prime-rl 0.6.0 to Train Trillion-Parameter MoE Models on Agentic RL Workloads

Short summary

Prime Intellect released prime-rl 0.6.0, an open framework for training trillion-parameter Mixture-of-Experts models on agentic RL tasks. The framework achieved sub-5-minute training steps on GLM-5 with 256 rollouts across 28 H200 nodes using FP8 inference and 3-D parallelism. Key optimizations include Wide Expert Parallelism and prefill/decode disaggregation.

• •prime-rl 0.6.0 enables training trillion-parameter MoE models for agentic RL workloads at scale
• •Achieved sub-5-minute training steps with 256 rollouts on 28 H200 nodes
• •Core optimizations: FP8 inference, Wide Expert Parallelism, prefill/decode disaggregation, 3-D parallelism

Generated with AI, which can make mistakes.