Reinforced Planning with Latent World Models

TL;DR - Reinforced Planning trains a neural planner to evaluate imagined outcomes and directly improve multi-step plans using offline rollouts from latent world models. Its RP1 implementation outperforms hand-designed search across navigation and robotics tasks while requiring far fewer rollouts and less inference time.

• RP1 jointly learns a critic for evaluating imagined outcomes and an optimizer for revising plans.
• Training is fully offline and uses imagined rollouts rather than environment interactions.
• The planner can be trained independently and attached to different pretrained latent world models.
• Across two world-model backbones, RP1 used 1,000× fewer rollouts and was up to 67× faster than the strongest alternative under concurrent inference.