Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models
Merged summary
TL;DR - This paper tests whether LLM in-context estimates obey the law of total probability, finding widespread violations of statistical self-consistency—a reference-free way to evaluate frontier models.
- Uses binary trees to recursively partition populations into finer subpopulations, prompts LLMs with verbalized subpopulation descriptions, then aggregates estimates back to population level and compares across partition granularities.
- Across multiple domains and state-of-the-art models, LLM estimates broadly violate basic probabilistic identities (prior-weighted conditionals should aggregate into marginals).
- Identifies a "macro fallacy": estimates reconstructed from fine-grained subpopulation responses often align better with human reference data than direct population-level estimates—models hold subpopulation knowledge but fail to propagate it into aggregates.
- The effect persists across tree structures and tasks and is partially recoverable via implicit prompting, positioning self-consistency as an unsaturated, reference-free evaluation criterion.
Sources (1)
Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models
TL;DR - This paper tests whether LLM in-context estimates obey the law of total probability, finding widespread violations of statistical self-consistency—a reference-free way to evaluate frontier models.
- Uses binary trees to recursively partition populations into finer subpopulations, prompts LLMs with verbalized subpopulation descriptions, then aggregates estimates back to population level and compares across partition granularities.
- Across multiple domains and state-of-the-art models, LLM estimates broadly violate basic probabilistic identities (prior-weighted conditionals should aggregate into marginals).
- Identifies a "macro fallacy": estimates reconstructed from fine-grained subpopulation responses often align better with human reference data than direct population-level estimates—models hold subpopulation knowledge but fail to propagate it into aggregates.
- The effect persists across tree structures and tasks and is partially recoverable via implicit prompting, positioning self-consistency as an unsaturated, reference-free evaluation criterion.