Running Qwen3 Through the ExecuTorch MLX Delegate: Up to 4.52x Faster on M1 Max

Short summary

Benchmark of Qwen3-0.6B running through ExecuTorch's experimental MLX delegate on an M1 Max, compared against PyTorch MPS. MLX BF16 achieved 134.8 tokens/s vs 41.8 tokens/s for PyTorch MPS BF16, and MLX INT4 reached 188.9 tokens/s — 4.52x faster with 71.8% smaller files. INT4 quantization altered output on two of three test prompts, highlighting the speed-accuracy tradeoff.

• •MLX INT4 is 4.52x faster than PyTorch MPS BF16 with 71.8% smaller files
• •INT4 quantization changed output on 2 of 3 prompts — accuracy tradeoff is real
• •Full reproducible lab with code, JSON report, and setup instructions provided

Generated with AI, which can make mistakes.