Does More Retrieved Evidence Help Visual Retrieval-Augmented Generation with Diffusion Language Models?

TL;DR - An arXiv paper showing that in visual RAG with diffusion language models (DLMs), feeding all retrieved pages to the generator hurts accuracy despite higher answer-page recall, and proposing a training-free filter to admit only helpful evidence. It matters because it overturns the "more context is better" default for the emerging DLM-RAG setting.

• Diagnoses the failure as semantic conflict / source-coherence loss in parallel denoising: position-wise proposals can fuse incompatible visual sources into unsupported answers.
• The interference is detectable in the first-step answer-block distribution, enabling pre-decoding assessment of candidate evidence.
• Proposes Entropy-Based Candidate Filter (ECF): multi-granularity evidence units plus blank-controlled block confidence and retrieval rank to decide whether and which candidates enter the final context; no training required.
• Reports +2.62 pp average accuracy over the strongest fixed top-k input across three multimodal DLMs and five visual QA benchmarks, and +2.37 pp with LLaDA2.0-Uni over the best competing training-free baseline per dataset; code released.