PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics
Ranking
Overall
83
Content
100
Popularity
43
Observed public metrics from 1 member.
Merged summary
TL;DR - PointZero learns transferable 3D dynamics by completing point trajectories from RGB-D observations and sparse partial tracks, avoiding the need for robot action labels during pre-training. This enables broader data use and improves downstream 3D prediction and robot manipulation.
- Pre-training uses 2.9 million synthetic frames spanning deformable, articulated, and rigid objects.
- A transformer predicts future 3D trajectories for all observed points from a single RGB-D frame and sparse tracks.
- After action-conditioned fine-tuning, PointZero outperforms baselines on the PGND 3D dynamics benchmark.
- For imitation learning, it matches or exceeds baselines on 6 of 7 simulated and real-world manipulation tasks.
Sources (1)
PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics
Public signals
Hugging Face upvotes 0
TL;DR - PointZero learns transferable 3D dynamics by completing point trajectories from RGB-D observations and sparse partial tracks, avoiding the need for robot action labels during pre-training. This enables broader data use and improves downstream 3D prediction and robot manipulation.
- Pre-training uses 2.9 million synthetic frames spanning deformable, articulated, and rigid objects.
- A transformer predicts future 3D trajectories for all observed points from a single RGB-D frame and sparse tracks.
- After action-conditioned fine-tuning, PointZero outperforms baselines on the PGND 3D dynamics benchmark.
- For imitation learning, it matches or exceeds baselines on 6 of 7 simulated and real-world manipulation tasks.