Schrödinger's Cat: Probabilistic Representation and Prediction of Potential Scene Kinematics

TL;DR - GARFIELD models a distribution of possible future scene motions from an image and optional sparse constraints, rather than predicting one trajectory. It supports fast, uncertainty-aware motion planning and interactive refinement.

• Uses a structured spatiotemporal latent representation to jointly sample scene trajectories.
• A deterministic density decoder localizes motion uncertainty by scene element and timestep.
• Additional constraints progressively refine the predicted future-motion distribution.
• Achieves competitive planning performance while sampling trajectories 97× faster than large video-generation models and estimating densities roughly 100× faster than Monte Carlo sampling.