Stanford CS25: Transformers United V6 I Serving Transformers: Lessons from the Trenches

Short summary

Stanford CS25 seminar featuring Charles Frye (Modal) on lessons from serving transformer models in production at scale across thousands of GPUs. Covers infrastructure optimization, operational trade-offs, and real gnarly problems from production deployments. Taught alongside Christopher Manning and Stanford's top ML researchers, aimed at engineers and technical leads building large-scale transformer inference systems.

• •Production insights from serving transformers at thousand-GPU scale
• •Led by Charles Frye (Modal) with Christopher Manning and Stanford ML faculty
• •Operational lessons on infrastructure, trade-offs, and deployment challenges

Generated with AI, which can make mistakes.