LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers

TL;DR - LogicTrack is a neuro-symbolic framework that uses formal logic solvers to audit each step in an LLM’s chain of thought. It aims to improve both reasoning-chain validity and final-answer accuracy, addressing cases where models reach correct conclusions through flawed logic.

• Automatically formalizes intermediate reasoning steps into symbolic representations and checks them with theorem provers.
• Introduces Solver-Based Backtracking Reward, a step-level score that guides backtracking tree search during inference.
• Generates supervised fine-tuning data containing backtracking traces, helping models internalize step-wise auditing.
• Experiments spanning eight reasoning benchmarks and seven LLMs report improvements in reasoning verifiability and final-answer pass rate.