🛰️ Daily AI Frontier
‹ back to 2026-08-10

让生成式模型「画」出空间智能,而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架

WeChat: 机器之心 Multimodal & Generative 2026-08-09
Representative image for 让生成式模型「画」出空间智能,而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架

TL;DR - Zhejiang University's OmniAI team proposes ProVisE, a framework that lets image-generation models answer spatial questions by drawing on the image (markers, depth maps, masks, trajectories) instead of emitting text coordinates, plus SpatialGen-Bench for evaluating spatial cognition. It matters because it opens spatial-reasoning benchmarks to generative models and exposes complementary strengths versus text-output VLMs.

  • ProVisE pairs each task with a "visual protocol": a guidance prompt telling the model how to draw its answer, and a parser that converts pixels back into labels, points, masks, states, or trajectories scorable by the original benchmark.
  • An Agentic Protocol Construction builder scans each task's inputs, answer structure, and scoring rules, then picks Reuse, Build, or Fallback (auxiliary VLM parsing); protocols are validated then frozen for all evaluated models.
  • SpatialGen-Bench spans 14 subtasks across four levels — perception, understanding, reasoning, interaction — used to compare 20 text-output VLMs against 11 image-generation models.
  • Findings: generative models show stronger "spatial intuition" on depth/marking tasks and solved 37% of items GPT-5.4 missed, while VLMs lead by 17.6 points on spatial reasoning; 88.03% of generative failures produced parseable images with wrong spatial content, so the bottleneck is accuracy, not generation or parsing.

view merged work →