From Pixels to States: Rethinking Interactive World Models as Game Engines
Merged summary
TL;DR — A survey/position paper that reframes interactive "world models" built on video generative models through the lens of the classic game-engine action-state-observation loop, and releases a gameplay dataset to support state-aware world modeling. It matters because video generation is increasingly pitched as a next-generation game engine, and this work clarifies what's still missing.
- Organizes interactive game-world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation; for each it groups existing methods into families and weighs trade-offs.
- Argues genuine interactivity requires rule-following outcomes over evolving state, consequences that persist over long horizons, and a real-time generation loop—contrasting data-driven video prediction with conventional engines' explicit-state rendering.
- Contributes a scalable data engine for Black Myth: Wukong: 90+ hours of gameplay with frame-aligned player actions, ground-truth game states, visual observations, plus structured/semantic annotations.
- Content is primarily a framework/survey plus a dataset; it presents no new model benchmark results, so takeaways are conceptual and resource-oriented.
Sources (1)
From Pixels to States: Rethinking Interactive World Models as Game Engines
TL;DR — A survey/position paper that reframes interactive "world models" built on video generative models through the lens of the classic game-engine action-state-observation loop, and releases a gameplay dataset to support state-aware world modeling. It matters because video generation is increasingly pitched as a next-generation game engine, and this work clarifies what's still missing.
- Organizes interactive game-world modeling along four dimensions: player action control, game state dynamics, state-observation persistence, and real-time interactive generation; for each it groups existing methods into families and weighs trade-offs.
- Argues genuine interactivity requires rule-following outcomes over evolving state, consequences that persist over long horizons, and a real-time generation loop—contrasting data-driven video prediction with conventional engines' explicit-state rendering.
- Contributes a scalable data engine for Black Myth: Wukong: 90+ hours of gameplay with frame-aligned player actions, ground-truth game states, visual observations, plus structured/semantic annotations.
- Content is primarily a framework/survey plus a dataset; it presents no new model benchmark results, so takeaways are conceptual and resource-oriented.