TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes
Ranking
Overall
80
Content
100
Popularity
34
Observed public metrics from 1 member.
Merged summary
TL;DR - TEMPO is a makespan-aware dispatcher for expert-parallel MoE inference that models both memory-bound expert activation and compute-bound token processing. It improves Qwen3-235B throughput by 4–6% and reduces p99 latency by about 15.6% in mixed-regime workloads.
- Models expert execution with a max-affine profile that captures HBM weight streaming and padded grouped-GEMM costs.
- Formulates per-batch dispatch as a fixed-charge makespan problem and solves it off the critical path in milliseconds.
- Stays within 1% of the best fixed baseline across tested conditions and improves microbenchmark performance by up to 15.5%.
- A phase diagram predicts when TEMPO helps; communication-dominated DeepSeek-V3 workloads show only its mechanism cost.
Sources (1)
TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - TEMPO is a makespan-aware dispatcher for expert-parallel MoE inference that models both memory-bound expert activation and compute-bound token processing. It improves Qwen3-235B throughput by 4–6% and reduces p99 latency by about 15.6% in mixed-regime workloads.
- Models expert execution with a max-affine profile that captures HBM weight streaming and padded grouped-GEMM costs.
- Formulates per-batch dispatch as a fixed-charge makespan problem and solves it off the critical path in milliseconds.
- Stays within 1% of the best fixed baseline across tested conditions and improves microbenchmark performance by up to 15.5%.
- A phase diagram predicts when TEMPO helps; communication-dominated DeepSeek-V3 workloads show only its mechanism cost.