‹ back to 2026-09-24

ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning

arXiv cs.LG LLM Agents Ming Ma, Yi Zhu, Yiran Zhong, Feida Zhu, Chonghan Liu, Pengkun Jiao, Qichao Wang, Yanhao Jia, Tianming Yang, Steven Hoi 2026-09-23

TL;DR - ProCredit improves reinforcement learning for long-horizon agents by rerunning acceptance checks after each tool-use turn and rewarding verified progress where it occurs. This provides denser, more precise credit than final outcome rewards alone.

Measures each turn’s reward as the change in task progress, distinguishing productive actions from queries and near-successes from weaker failures.
Assigns credit both across attempts for the same task and across turns within each trajectory.
On AppWorld, it outperformed outcome- and progress-based baselines across three Qwen3.5 model scales and two test sets.
At 4B parameters, it exceeded the strongest outcome-reward baseline by 4.1 percentage points; ablations attribute the gain to turn-level credit assignment rather than trajectory-level progress scoring.

view merged work →