‹ back to 2026-07-24

3D-Aware VLMs with Implicit and Explicit Geometries

arXiv cs.CV Multimodal & Generative Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Ran Xu, Shijian Lu, Gongjie Zhang 2026-07-23

TL;DR - VLM-IE3D improves vision-language models’ 3D spatial understanding by combining implicit geometric priors and explicit reconstructed geometry from RGB video. It handles multiple 3D tasks without requiring dedicated 3D inputs.

Implicit Geometry Tokens capture high-level geometric priors from videos.
Explicit Geometry Tokens encode detailed structures from reconstructed 3D attributes.
A 3D-aware adapter fuses both token types with conventional 2D visual cues.
Experiments report consistently superior performance across 3D detection, grounding, dense captioning, and spatial reasoning tasks.

view merged work →