港大&字节重磅StereoWorld：无需深度估计，直接生成几何一致双目视频，破解单目几何幻觉，3倍加速5%一致性提升！
Ranking
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - StereoWorld (HKU + ByteDance) is a camera-conditioned stereo world model that generates geometrically consistent binocular video end-to-end in RGB space, without depth estimation or inpainting, grounding 3D structure directly in disparity. It matters because it removes the fragile monocular-depth + warp + inpaint pipeline used for VR/AR rendering and embodied spatial reasoning.
- Unified camera-frame RoPE: camera pose is injected by expanding the token/feature dimension with a camera-aware rotary encoding rather than reparameterizing the backbone's RoPE (as PRoPE does) or concatenating absolute Plücker rays, preserving pretrained video priors; "copy init" from temporal attention weights converges faster and gives better camera accuracy than zero init.
- Stereo-aware attention decomposition: full 4D spatio-temporal-cross-view attention is split into 3D intra-view attention plus horizontal-line cross-view attention, exploiting the epipolar prior of rectified pairs to cut cost from O((2N)²) to roughly O(N²)-scale while keeping disparity-aligned correspondence.
- Results: vs. SOTA "monocular generation + StereoCrafter stereo conversion" pipelines, >3× faster generation and ~5% better view consistency, plus higher camera pose accuracy (VGGT-estimated) and cleaner disparity — notably trained with no depth supervision, only binocular signal.
- Setup and applications: built on Wan2.2-T12V-5B, 49-frame clips, 20k steps on 24× H20 GPUs over Stereo4D/TartanAir/DynamicReplica/VKitti; enables direct binocular VR rendering, metric-scale depth for embodied policy learning (DROID fine-tune), and 4-step causal distillation lifting throughput from 0.49 to 5.6 FPS for 10-second stereo video.
Sources (1)
港大&字节重磅StereoWorld：无需深度估计，直接生成几何一致双目视频，破解单目几何幻觉，3倍加速5%一致性提升！
TL;DR - StereoWorld (HKU + ByteDance) is a camera-conditioned stereo world model that generates geometrically consistent binocular video end-to-end in RGB space, without depth estimation or inpainting, grounding 3D structure directly in disparity. It matters because it removes the fragile monocular-depth + warp + inpaint pipeline used for VR/AR rendering and embodied spatial reasoning.
- Unified camera-frame RoPE: camera pose is injected by expanding the token/feature dimension with a camera-aware rotary encoding rather than reparameterizing the backbone's RoPE (as PRoPE does) or concatenating absolute Plücker rays, preserving pretrained video priors; "copy init" from temporal attention weights converges faster and gives better camera accuracy than zero init.
- Stereo-aware attention decomposition: full 4D spatio-temporal-cross-view attention is split into 3D intra-view attention plus horizontal-line cross-view attention, exploiting the epipolar prior of rectified pairs to cut cost from O((2N)²) to roughly O(N²)-scale while keeping disparity-aligned correspondence.
- Results: vs. SOTA "monocular generation + StereoCrafter stereo conversion" pipelines, >3× faster generation and ~5% better view consistency, plus higher camera pose accuracy (VGGT-estimated) and cleaner disparity — notably trained with no depth supervision, only binocular signal.
- Setup and applications: built on Wan2.2-T12V-5B, 49-frame clips, 20k steps on 24× H20 GPUs over Stereo4D/TartanAir/DynamicReplica/VKitti; enables direct binocular VR rendering, metric-scale depth for embodied policy learning (DROID fine-tune), and 4-step causal distillation lifting throughput from 0.49 to 5.6 FPS for 10-second stereo video.