DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving

TL;DR - DreamStream is a generative closed-loop driving simulator designed to preserve policy-relevant scene features rather than optimize photorealism alone. It enables more faithful testing and reveals policy failures missed by prior benchmarks.

• Distills a pretrained autoregressive video model using traffic-layout guidance to vary appearance while preserving scene geometry and dynamic-object consistency.
• Introduces FDπ, a Fréchet-distance metric based on scene-context features from public end-to-end driving policies, addressing shortcomings of metrics such as FID.
• Improves FDπ over the strongest prior closed-loop simulator by 1.6× on nuScenes and 4.7× on NAVSIM.
• Builds Navhard-CL, an interactive benchmark with adversarial driving and weather variations that exposes scorer bias and weak recovery behavior.