🛰️ Daily AI Frontier
‹ back to 2026-09-02

LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

arXiv cs.CL LLMs & Foundation Models Muhammed Saeed, Simon Razniewski 2026-09-01
Representative image for LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

TL;DR - LLMPEDIA is an open, browsable audit of encyclopedic knowledge generated from the parametric memory of three LLM families without retrieval. Its sampled factuality rate of 68.4%—over 21 percentage points below MMLU scores—highlights major knowledge gaps hidden by fixed benchmarks.

  • The authors recursively generated roughly 1.3 million articles using GPT-5-mini, DeepSeek-V3.2, and Llama-3.3-70B.
  • Atomic claims were checked against Wikipedia and a curated web stack, then labeled supported, refuted, or insufficiently evidenced.
  • A uniform random sample found 68.4% of claims true and 30.5% insufficient, encompassing both long-tail knowledge and possible hallucinations.
  • The live site supports claim-level inspection, link traversal, cross-model and political-persona comparisons, and guided topic exploration.

view merged work →