🛰️ Daily AI Frontier
‹ back to 2026-08-17

港大&腾讯重磅开源:用相机射线重构Attention位置编码,给世界模型换上「3D坐标系」!单图WASD复杂轨迹漫游,FVD降43%

WeChat: 3D视觉工坊 Multimodal & Generative 2026-08-16
Representative image for 港大&腾讯重磅开源:用相机射线重构Attention位置编码,给世界模型换上「3D坐标系」!单图WASD复杂轨迹漫游,FVD降43%

TL;DR - SCoPE injects camera-ray geometry into video diffusion Transformer attention, enabling more accurate camera-controlled world exploration without explicit 3D reconstruction. On Wan2.2-14B, it cuts rotation error by 29% and FVD by 43% while adding under 0.1% parameters.

  • Represents each video token’s viewing ray with Plücker coordinates and injects projected ray features into attention queries and keys.
  • Jointly scores visual similarity and geometric compatibility while retaining the pretrained model’s existing 3D RoPE and content projections.
  • Normalize-Gate-Inject separates ray direction and scale to accommodate camera poses from calibration, SfM, or SLAM.
  • Improves complex WASD trajectories and closed-loop revisits across Wan2.2 5B and 14B models.

view merged work →