Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding

TL;DR - ViPS is a framework that fuses spatial priors from multiple pre-trained visual foundation models into MLLMs, claiming new state-of-the-art results on spatial reasoning and 3D understanding benchmarks.

• Observes that different foundation models supply complementary spatial priors, each benefiting different tasks—motivating a multi-model rather than single-expert approach.
• Introduces an Efficient Prior Proxy to generate multiple foundational priors with minimal added inference overhead.
• Adds a Dynamic Prior Fusion mechanism for context-aware, "harmonious" fusion and injection of these priors into the MLLM.
• Reports state-of-the-art performance across multiple complex spatial reasoning and 3D spatial understanding benchmarks; specific metrics not provided in the abstract.