“接力跑”盘活全国算力，PD分离终于破局：延迟砍半、成本直降近40%！

TL;DR - Infinigence introduced PDD, a three-stage cross-cluster LLM inference architecture that uses a local relay decoder to hide wide-area KV-cache transfer latency. Tests on agentic workloads showed lower latency and up to a 37.5% improvement in cost efficiency.

• PDD splits inference into Prefill, local RelayDecode, and remote MainDecode stages across heterogeneous clusters.
• The relay begins generating tokens while KV cache transfers remotely, then hands off using token IDs and local KV-cache recomputation.
• Routing reserves the relay for low-cache-hit requests; it generated only 6.2% of tokens while remote decoders handled 93.8%.
• Compared with conventional cross-cluster PD separation, P90 TTFT fell from 18.3 to 9.8 seconds; versus same-datacenter homogeneous deployment, goodput rose 27.8% with 3.5%–7.1% lower cost.