RT by @ylecun: Using optimization at inference time is a foundational concept of Energy-Based…
TL;DR - The post argues that inference-time optimization is central to energy-based and objective-driven AI, especially for planning with world models. Continuous variables make gradient-based optimization a natural planning mechanism.
- EBMs infer outputs by optimizing an energy or objective at inference time.
- Objective-driven architectures can apply the same principle to action planning.
- World models provide a predictive objective over possible plans.
- Gradient-based search is particularly suitable when inferred variables are continuous.