🛰️ Daily AI Frontier
‹ back to 2026-08-10

让生成式模型「画」出空间智能,而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架

WeChat: 机器之心 Multimodal & Generative 2026-08-08
Representative image for 让生成式模型「画」出空间智能,而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架

TL;DR - Zhejiang University's OmniAI team proposes ProVisE, an evaluation framework that lets image-generation models answer spatial questions by drawing directly in pixel space (marks, depth maps, masks, trajectories) instead of forcing text/coordinate outputs, plus SpatialGen-Bench, a 14-task spatial cognition benchmark. It matters because it opens spatial-intelligence evaluation to generative models and reveals complementary strengths versus text-output VLMs.

  • ProVisE (Protocolized Visual Evaluation) pairs each task with a "visual protocol": a guidance prompt telling the model how to draw its answer, and a parser converting colors/positions/regions back into structured predictions (labels, points, masks, states, trajectories) scoreable by the original benchmark metrics.
  • An Agentic Protocol Construction builder scans each task's data, answer structure, and scoring rules, then picks Reuse (existing protocol), Build (assemble from registered parsing components), or Fallback (auxiliary VLM parses the generated image); protocols are validated then frozen for all evaluations.
  • SpatialGen-Bench spans 14 subtasks across four levels — perception (counting, relative depth, orientation, size), understanding (grounding, relations, viewpoint, scene modeling), reasoning (multi-step, state prediction, geometric feasibility), and interaction (affordance grounding, navigation, trajectory planning) — evaluated on 20 text-output VLMs and 11 image-generation models.
  • Findings: generation models are competitive where depth maps or spatial markings suffice and solved 37% of items GPT-5.4 got wrong, while VLMs lead by 17.6 points on average at the spatial reasoning level; 88.03% of generative failures produced valid parsed predictions that were simply spatially wrong, not generation/parsing failures.

view merged work →