把 Gemini 和 GPT 放到《我的世界》当教练，谁能看懂机器人的微操？
Ranking
Overall
68
Content
85
Popularity
30
Observed public metrics from 1 member.
Merged summary
TL;DR - Tsinghua University and NVIDIA’s VLM-AR3L framework uses vision-language models to generate absolute and relative reward labels for embodied reinforcement learning, then distills them into a lightweight local network. This produces denser, more stable feedback while reducing VLM queries by roughly 20×.
- Gemini-2.0-Flash exceeded 70% progress-judgment accuracy across all 10 simulated tasks; GPT-4.1-nano generally reached 60–66%, while some small open models excelled on specific tasks.
- VLM-AR3L combines absolute goal proximity with pairwise temporal progress, addressing reward drift and cyclic tasks where fixed absolute state rankings fail.
- Offline VLM labeling trains a compact Siamese reward network, avoiding expensive, latency-heavy API calls during policy optimization.
- The method achieved 85% success on Minecraft’s Combat Spider and 95% on Milk Cow, and learned several long-horizon tasks where sparse human-authored rewards yielded 0% success.
Sources (1)
把 Gemini 和 GPT 放到《我的世界》当教练，谁能看懂机器人的微操？
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - Tsinghua University and NVIDIA’s VLM-AR3L framework uses vision-language models to generate absolute and relative reward labels for embodied reinforcement learning, then distills them into a lightweight local network. This produces denser, more stable feedback while reducing VLM queries by roughly 20×.
- Gemini-2.0-Flash exceeded 70% progress-judgment accuracy across all 10 simulated tasks; GPT-4.1-nano generally reached 60–66%, while some small open models excelled on specific tasks.
- VLM-AR3L combines absolute goal proximity with pairwise temporal progress, addressing reward drift and cyclic tasks where fixed absolute state rankings fail.
- Offline VLM labeling trains a compact Siamese reward network, avoiding expensive, latency-heavy API calls during policy optimization.
- The method achieved 85% success on Minecraft’s Combat Spider and 95% on Milk Cow, and learned several long-horizon tasks where sparse human-authored rewards yielded 0% success.