Blind Men and the Elephant: Probing the Epistemic Myopia of LLMs under Long-Tail Divergent Knowledge

TL;DR - ElephantBench evaluates whether LLMs retain multiple conflicting accounts of long-tail facts rather than only a dominant answer. Across 32 models, even the strongest recovered both accounts for just 52.4% of questions, revealing persistent incompleteness in parametric memory.

• The benchmark contains 1,094 closed-book QA questions built from naturally divergent accounts found through an auditable, graph-based pipeline.
• Answers are traceable to source documents, checked against authoritative public sources, and reviewed by human annotators.
• Larger models and inference-time reasoning improve recall but do not eliminate the tendency to omit one account.
• More balanced corpus exposure correlates with more complete recall, while exposure imbalance favors the dominant account.