“接力跑”盘活全国算力，PD分离终于破局：延迟砍半、成本直降近40%！
Ranking
Overall
78
Content
90
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - Infinigence introduced PDD, a three-stage cross-cluster LLM inference architecture that uses a local relay decoder to hide wide-area KV-cache transfer latency. Tests on agentic workloads showed lower latency and up to a 37.5% improvement in cost efficiency.
- PDD splits inference into Prefill, local RelayDecode, and remote MainDecode stages across heterogeneous clusters.
- The relay begins generating tokens while KV cache transfers remotely, then hands off using token IDs and local KV-cache recomputation.
- Routing reserves the relay for low-cache-hit requests; it generated only 6.2% of tokens while remote decoders handled 93.8%.
- Compared with conventional cross-cluster PD separation, P90 TTFT fell from 18.3 to 9.8 seconds; versus same-datacenter homogeneous deployment, goodput rose 27.8% with 3.5%–7.1% lower cost.
Sources (1)
“接力跑”盘活全国算力，PD分离终于破局：延迟砍半、成本直降近40%！
Public signals
N/A
TL;DR - Infinigence introduced PDD, a three-stage cross-cluster LLM inference architecture that uses a local relay decoder to hide wide-area KV-cache transfer latency. Tests on agentic workloads showed lower latency and up to a 37.5% improvement in cost efficiency.
- PDD splits inference into Prefill, local RelayDecode, and remote MainDecode stages across heterogeneous clusters.
- The relay begins generating tokens while KV cache transfers remotely, then hands off using token IDs and local KV-cache recomputation.
- Routing reserves the relay for low-cache-hit requests; it generated only 6.2% of tokens while remote decoders handled 93.8%.
- Compared with conventional cross-cluster PD separation, P90 TTFT fell from 18.3 to 9.8 seconds; versus same-datacenter homogeneous deployment, goodput rose 27.8% with 3.5%–7.1% lower cost.