Belief-State Engine: Augmenting LLMs for Principled Planning Under Partial Observability

TL;DR - The Belief-State Engine augments LLM agents with an external Bayesian posterior over hidden environment states, enabling principled planning under partial observability. This addresses premature commitments, policy drift, and miscalibrated uncertainty without exposing the LLM to raw interaction history.

• Models decision-making as a belief MDP and proves that the BSE-LLM combination forms a sound Markov policy under four belief-consistency axioms.
• Supplies only the current posterior to the LLM, separating probabilistic state inference from action selection.
• Outperforms six baselines—including Chain-of-Thought, ReAct, QMDP, and POMCP—on Tiger POMDP and red-team attack-graph tasks in return, calibration, and consistency.
• Ten ablations test the architectural choices and indicate that improvements are not tied to a single LLM.