TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound Regimes

TL;DR - TEMPO is a makespan-aware dispatcher for expert-parallel MoE inference that models both memory-bound expert activation and compute-bound token processing. It improves Qwen3-235B throughput by 4–6% and reduces p99 latency by about 15.6% in mixed-regime workloads.

• Models expert execution with a max-affine profile that captures HBM weight streaming and padded grouped-GEMM costs.
• Formulates per-batch dispatch as a fixed-charge makespan problem and solves it off the critical path in milliseconds.
• Stays within 1% of the best fixed baseline across tested conditions and improves microbenchmark performance by up to 15.5%.
• A phase diagram predicts when TEMPO helps; communication-dominated DeepSeek-V3 workloads show only its mechanism cost.