🛰️ Daily AI Frontier
‹ back to 2026-08-07

ACM MM 2026 | DualG-MRAG:解耦宏观推理与微观匹配的多模态检索增强生成

WeChat: 专知 LLM Agents 2026-08-06
Representative image for ACM MM 2026 | DualG-MRAG:解耦宏观推理与微观匹配的多模态检索增强生成

TL;DR - DualG-MRAG (ACM MM 2026, Beihang University) is a multimodal RAG framework that splits graph-augmented retrieval into a macro reasoning graph for cross-document multi-hop routing and a micro matching graph for fine-grained visual/table evidence, avoiding the graph-explosion vs. lost-detail tradeoff. It matters because it turns fragmented retrieved chunks into explicit structured reasoning chains for downstream MLLMs, cutting hallucination on knowledge-intensive QA.

  • Two-tier decoupled graphs: macro graph handles global entity/document relations and multi-hop topological routing; micro graph covers image regions, table cells and local text spans, linked via cross-layer alignment so macro paths ground to concrete multimodal evidence.
  • Query-driven GNN + path decoding: relevance propagation on the macro graph is conditioned on the query (dynamic, not static topology), and dynamic programming extracts the top-scoring node sequence from forward-pass layers into an explicit reasoning path fed to the MLLM.
  • Results: with Qwen3-VL-8B, MMQA EM 46.00 / F1 51.19; ScienceQA average accuracy 90.99 (IMG subset 91.52); retrieval R@5 of 61.9 on MMQA and 58.2 on WebQA at ~0.44s average query latency.
  • Ablations show both tiers are essential: removing the micro graph drops WebQA R@5 from 58.2 to 40.0; removing the macro graph drops MMQA R@5 from 61.9 to 21.8. Removing path injection leaves retrieval unchanged but hurts generation — more so for the 4B model than the 8B, suggesting smaller models rely more on explicit structure.

view merged work →