李飞飞、Yilun Du罕见联手：别给机器人建大脑了，直接偷视频模型的｜GAIR Paper 115
Ranking
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - A media deep-dive on the arXiv preprint Masked Visual Actions for Unified World Modeling (MVA), co-authored by Fei-Fei Li, Yilun Du, Jiajun Wu, Lvmin Zhang, Gordon Wetzstein and others, which argues robots don't need a bespoke action foundation model — actions can be re-encoded as pixel-space mask trajectories that off-the-shelf video generators already understand.
- Interface, not a new brain: SAM segments the robot arm and manipulated object per frame, turning an action into two colored mask trajectories — no joint angles, coordinates, or numeric action tokens.
- Masked-completion duality: Masking the object trajectory makes the model a forward world simulator; masking the arm trajectory makes it an inverse action generator — same model, same weights, switched only by which mask is hidden.
- Cheap adaptation: Built on Alibaba's open-source Wan2.2 (14B diffusion transformer) via LoRA with ~15 hours of robot data; reported LPIPS 0.0945 vs 0.362 for Ctrl-World, and zero-shot transfer from a single-arm training robot to an unseen bimanual robot.
- Cross-embodiment via URDF: The pipeline predicts where the end-effector should go, then uses each robot's URDF with FK/IK to solve joint angles — this indirection is credited (by industry CTO Li Linxin) as the real source of cross-embodiment generalization, and also constrains the video model to kinematically feasible poses.
Sources (1)
李飞飞、Yilun Du罕见联手：别给机器人建大脑了，直接偷视频模型的｜GAIR Paper 115
TL;DR - A media deep-dive on the arXiv preprint Masked Visual Actions for Unified World Modeling (MVA), co-authored by Fei-Fei Li, Yilun Du, Jiajun Wu, Lvmin Zhang, Gordon Wetzstein and others, which argues robots don't need a bespoke action foundation model — actions can be re-encoded as pixel-space mask trajectories that off-the-shelf video generators already understand.
- Interface, not a new brain: SAM segments the robot arm and manipulated object per frame, turning an action into two colored mask trajectories — no joint angles, coordinates, or numeric action tokens.
- Masked-completion duality: Masking the object trajectory makes the model a forward world simulator; masking the arm trajectory makes it an inverse action generator — same model, same weights, switched only by which mask is hidden.
- Cheap adaptation: Built on Alibaba's open-source Wan2.2 (14B diffusion transformer) via LoRA with ~15 hours of robot data; reported LPIPS 0.0945 vs 0.362 for Ctrl-World, and zero-shot transfer from a single-arm training robot to an unseen bimanual robot.
- Cross-embodiment via URDF: The pipeline predicts where the end-effector should go, then uses each robot's URDF with FK/IK to solve joint angles — this indirection is credited (by industry CTO Li Linxin) as the real source of cross-embodiment generalization, and also constrains the video model to kinematically feasible poses.