FineServe: A Fine-Grained Dataset and Characterization of Global LLM Serving Workloads

Short summary

FineServe is a fine-grained, multi-model LLM serving workload dataset collected from a global commercial marketplace, enabling realistic characterization of arrival dynamics and token behavior across heterogeneous models and tasks. The analysis reveals fundamentally different fluctuation regimes across model architectures, scales, and task intents. The authors also release a configurable workload generator for benchmarking multi-model serving platforms, available on GitHub.

• •FineServe dataset captures real-world multi-model LLM serving workloads from a commercial marketplace
• •Analysis reveals distinct fluctuation regimes across model architectures, scales, and task intents
• •Includes a configurable workload generator for benchmarking routing, scheduling, and capacity planning

Generated with AI, which can make mistakes.