Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination

TL;DR - An arXiv cs.CV preprint that reframes LVLM object hallucination as a problem of what the model attends to rather than how much, using Logit Lens to decode high-attention visual regions and then applying training-free fixes. It matters because it offers a diagnostic signal plus targeted mitigation without retraining.

• Counter-evidence to the prevailing "insufficient visual attention" explanation: real and hallucinated objects receive equally strong visual attention in mid-to-late layers.
• Logit Lens decoding of high-attention regions separates the two cases — real-object regions decode to the target object tokens, hallucinated ones do not.
• Two identified mechanisms: visual uncertainty (confusable/semantically similar regions; masking removes the hallucination) and contextual prior (co-occurrence-driven; hallucination persists after masking and attention drifts elsewhere).
• Proposed training-free Detect-Mitigate framework: a Logit-Lens Consistency Check for detection, plus HARM (High-Attention Regions Masking) and VEED (Visual Evidence Enhanced Decoding), reported as state-of-the-art on multiple hallucination benchmarks; code promised but not yet released.