VICT: Verifier-Instrumented Credit Tracing for Long-Horizon LLM Agent Reinforcement Learning

TL;DR - VICT improves credit assignment for long-horizon LLM agent reinforcement learning by tracing a terminal verifier’s structured checks back to the actions that supported them. This enables finer-grained training signals without learned critics, process labels, extra rollouts, or inference-time verifier access.

• Exposes executable or evidence-backed verifier “atoms” and links them to actions through dependency-valid proof edges.
• Redistributes group-relative advantage only along supported edges while preserving the original terminal reward and abstaining when evidence is ambiguous.
• Modifies only the training-time advantage tensor, avoiding additional inference-time requirements.
• On ALFWorld and WebShop, it substantially outperforms outcome-only training and performs competitively with recent fine-grained credit-assignment methods.