PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
Ranking
Overall
90
Content
100
Popularity
68
Observed public metrics from 1 member.
Merged summary
TL;DR - PAWBench evaluates whether video generators used as world models reproduce the probability distribution of valid physical outcomes, rather than merely generating plausible individual trajectories. Across 50 scenarios and 11 systems, no model consistently recovered both reference probabilities and the full range of valid behaviors.
- Formalizes “probabilistic alignment” as a distribution-level criterion for stochastic world models.
- Introduces PAWEval, which converts repeated video rollouts into empirical distributions over physical outcomes.
- Tests whether language prompts, initial-noise sampling, or model training can reshape predicted outcome distributions.
- Reveals a significant gap between plausible video generation and distributionally accurate world modeling.
Sources (1)
PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
Public signals
Hugging Face upvotes 75
TL;DR - PAWBench evaluates whether video generators used as world models reproduce the probability distribution of valid physical outcomes, rather than merely generating plausible individual trajectories. Across 50 scenarios and 11 systems, no model consistently recovered both reference probabilities and the full range of valid behaviors.
- Formalizes “probabilistic alignment” as a distribution-level criterion for stochastic world models.
- Introduces PAWEval, which converts repeated video rollouts into empirical distributions over physical outcomes.
- Tests whether language prompts, initial-noise sampling, or model training can reshape predicted outcome distributions.
- Reveals a significant gap between plausible video generation and distributionally accurate world modeling.