CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Ranking
Overall
72
Content
85
Popularity
42
Observed public metrics from 1 member.
Merged summary
TL;DR - CRAG-MM-Diagnostics is a stage-wise benchmark for diagnosing failures in knowledge-intensive visual question answering. It identifies retrieval and reasoning as the main bottleneck and shows that grounding objects before retrieval can substantially improve accuracy.
- Separately evaluates visual grounding, object identification, and knowledge retrieval/reasoning.
- Adds diagnostic metadata including target regions, entity names, and visual complexity scores.
- Finds object identification weaknesses and difficulty incorporating textual cues into image retrieval.
- A grounded bimodal RAG pipeline improves GPT-5 and Qwen accuracy by 13.3 and 8.5 percentage points, respectively.
Sources (1)
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - CRAG-MM-Diagnostics is a stage-wise benchmark for diagnosing failures in knowledge-intensive visual question answering. It identifies retrieval and reasoning as the main bottleneck and shows that grounding objects before retrieval can substantially improve accuracy.
- Separately evaluates visual grounding, object identification, and knowledge retrieval/reasoning.
- Adds diagnostic metadata including target regions, entity names, and visual complexity scores.
- Finds object identification weaknesses and difficulty incorporating textual cues into image retrieval.
- A grounded bimodal RAG pipeline improves GPT-5 and Qwen accuracy by 13.3 and 8.5 percentage points, respectively.