让生成式模型「画」出空间智能，而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR — Zhejiang University's OmniAI team proposes ProVisE (Protocolized Visual Evaluation), a framework that lets image-generation models answer spatial questions by drawing their answers in pixel space (markers, depth maps, masks, trajectories) rather than emitting text coordinates, together with SpatialGen-Bench, a 14-task spatial cognition benchmark. It matters because it opens spatial-intelligence evaluation to generative models and reveals strengths complementary to text-output VLMs.
- Visual protocols: each task is paired with a guidance prompt telling the model how to draw its answer, plus a parser that converts colors, positions, and regions back into structured predictions (labels, points, masks, states, trajectories) scoreable by the original benchmark metrics.
- Agentic Protocol Construction: a builder scans each task's data/inputs, answer structure, and scoring rules, then chooses Reuse (an existing protocol), Build (assemble from registered parsing components), or Fallback (an auxiliary VLM parses the generated image); protocols are validated, then frozen across all evaluated models.
- SpatialGen-Bench: 14 subtasks across four levels — perception (counting, relative depth, orientation, size), understanding (grounding, relations, viewpoint, scene modeling), reasoning (multi-step, state prediction, geometric feasibility), and interaction (affordance grounding, navigation, trajectory planning).
- Scale and results: 20 text-output VLMs vs. 11 image-generation models; generative models are competitive — showing stronger "spatial intuition" — where depth maps or spatial markings suffice, solving 37% of items GPT-5.4 got wrong, while VLMs lead by 17.6 points on average at the spatial reasoning level.
- Failure analysis: 88.03% of generative failures still produced valid, parseable predictions that were simply spatially wrong, indicating the bottleneck is spatial accuracy rather than image generation or protocol parsing.
Note: The two sources are near-identical in substance; the first gives finer detail on the per-level subtask breakdown and parser outputs, while the second frames the findings more around "spatial intuition" and the accuracy bottleneck.
Sources (2)
让生成式模型「画」出空间智能，而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架
TL;DR - Zhejiang University's OmniAI team proposes ProVisE, an evaluation framework that lets image-generation models answer spatial questions by drawing directly in pixel space (marks, depth maps, masks, trajectories) instead of forcing text/coordinate outputs, plus SpatialGen-Bench, a 14-task spatial cognition benchmark. It matters because it opens spatial-intelligence evaluation to generative models and reveals complementary strengths versus text-output VLMs.
- ProVisE (Protocolized Visual Evaluation) pairs each task with a "visual protocol": a guidance prompt telling the model how to draw its answer, and a parser converting colors/positions/regions back into structured predictions (labels, points, masks, states, trajectories) scoreable by the original benchmark metrics.
- An Agentic Protocol Construction builder scans each task's data, answer structure, and scoring rules, then picks Reuse (existing protocol), Build (assemble from registered parsing components), or Fallback (auxiliary VLM parses the generated image); protocols are validated then frozen for all evaluations.
- SpatialGen-Bench spans 14 subtasks across four levels — perception (counting, relative depth, orientation, size), understanding (grounding, relations, viewpoint, scene modeling), reasoning (multi-step, state prediction, geometric feasibility), and interaction (affordance grounding, navigation, trajectory planning) — evaluated on 20 text-output VLMs and 11 image-generation models.
- Findings: generation models are competitive where depth maps or spatial markings suffice and solved 37% of items GPT-5.4 got wrong, while VLMs lead by 17.6 points on average at the spatial reasoning level; 88.03% of generative failures produced valid parsed predictions that were simply spatially wrong, not generation/parsing failures.
让生成式模型「画」出空间智能，而非强迫LLM输出「坐标」! 浙大提出Agentic空间认知评估框架
TL;DR - Zhejiang University's OmniAI team proposes ProVisE, a framework that lets image-generation models answer spatial questions by drawing on the image (markers, depth maps, masks, trajectories) instead of emitting text coordinates, plus SpatialGen-Bench for evaluating spatial cognition. It matters because it opens spatial-reasoning benchmarks to generative models and exposes complementary strengths versus text-output VLMs.
- ProVisE pairs each task with a "visual protocol": a guidance prompt telling the model how to draw its answer, and a parser that converts pixels back into labels, points, masks, states, or trajectories scorable by the original benchmark.
- An Agentic Protocol Construction builder scans each task's inputs, answer structure, and scoring rules, then picks Reuse, Build, or Fallback (auxiliary VLM parsing); protocols are validated then frozen for all evaluated models.
- SpatialGen-Bench spans 14 subtasks across four levels — perception, understanding, reasoning, interaction — used to compare 20 text-output VLMs against 11 image-generation models.
- Findings: generative models show stronger "spatial intuition" on depth/marking tasks and solved 37% of items GPT-5.4 missed, while VLMs lead by 17.6 points on spatial reasoning; 88.03% of generative failures produced parseable images with wrong spatial content, so the bottleneck is accuracy, not generation or parsing.