ECCV 2026 专访：让大模型「忘掉XYZ」，RoboTracer 用 3D 空间感知与度量推理重塑机器人轨迹追踪
Ranking
Overall
78
Content
85
Popularity
63
Observed public metrics from 1 member.
Merged summary
TL;DR - RoboTracer enables vision-language models to generate robot-agnostic 3D trajectories as pixel coordinates plus metric depth, then delegates execution to conventional motion planners. This explicit spatial representation improves metric reasoning and allows the same model to control different robot embodiments without retraining.
- RoboTracer predicts trajectories as
(u, v, d)rather than world-space(x, y, z), combining VLM-friendly image coordinates with absolute depth and avoiding direct learning of camera geometry. - Its TraceSpatial dataset contains roughly 4.5 million samples and 30 million question-answer pairs synthesized from web images, 3D scans, and robot-operation videos.
- A scale-regression decoder and rule-based, metric-sensitive process rewards teach absolute scale and reward valid intermediate spatial evidence during multi-step planning.
- On TraceSpatial-Bench, RoboTracer achieved 39% success from RGB and 45% with camera intrinsics and depth, versus 3% for Gemini-2.5-Pro; reported real-robot success was 60% on UR5 tasks and 30% on Unitree G1 tasks.
Sources (1)
ECCV 2026 专访：让大模型「忘掉XYZ」，RoboTracer 用 3D 空间感知与度量推理重塑机器人轨迹追踪
Public signals
Hugging Face upvotes 37
TL;DR - RoboTracer enables vision-language models to generate robot-agnostic 3D trajectories as pixel coordinates plus metric depth, then delegates execution to conventional motion planners. This explicit spatial representation improves metric reasoning and allows the same model to control different robot embodiments without retraining.
- RoboTracer predicts trajectories as
(u, v, d)rather than world-space(x, y, z), combining VLM-friendly image coordinates with absolute depth and avoiding direct learning of camera geometry. - Its TraceSpatial dataset contains roughly 4.5 million samples and 30 million question-answer pairs synthesized from web images, 3D scans, and robot-operation videos.
- A scale-regression decoder and rule-based, metric-sensitive process rewards teach absolute scale and reward valid intermediate spatial evidence during multi-step planning.
- On TraceSpatial-Bench, RoboTracer achieved 39% success from RGB and 45% with camera intrinsics and depth, versus 3% for Gemini-2.5-Pro; reported real-robot success was 60% on UR5 tasks and 30% on Unitree G1 tasks.