🛰️ Daily AI Frontier
‹ back to 2026-09-23

PACT: From Credit Assignment to Critic Alignment

Research LLM Agents

Ranking

Overall 85
Content 95
Popularity 61

Observed public metrics from 1 member.

Merged summary

TL;DR - PACT formalizes token-level credit assignment for LLM reinforcement learning and introduces an actor-then-critic update that better aligns the critic with the updated policy. It substantially improves agentic mathematical reasoning and software-engineering benchmark performance over several established RL methods.

  • Three regularity conditions—Completeness, Prefix Consistency, and Neutrality—uniquely characterize token-level credit.
  • The analysis connects token-level credit to On-Policy Distillation and explains why response-level RLOO can produce the same expected policy-gradient contribution.
  • PACT uses importance-sampling correction during critic training to address policy mismatch and critic errors associated with GAE.
  • PACT reaches 72.87% average accuracy across four math benchmarks and 67.4% on SWE-bench Verified, outperforming the reported PPO, GRPO, and SAO baselines.

Sources (1)

PACT: From Credit Assignment to Critic Alignment

arXiv cs.LG Jiayan Fu, Hang Xu, Yong Zhang, Zhaokai Luo, Yao Hu, Dongyan Zhao, Mu Chuan 2026-09-22 arXiv:2609.26355
Public signals Hugging Face upvotes 16
Providers: Hugging Face · Upvotes 16 OpenAlex · N/A Publisher · N/A Semantic Scholar · N/A X · N/A Fetched 2026-09-25 14:16:48.248540 UTC

TL;DR - PACT formalizes token-level credit assignment for LLM reinforcement learning and introduces an actor-then-critic update that better aligns the critic with the updated policy. It substantially improves agentic mathematical reasoning and software-engineering benchmark performance over several established RL methods.

  • Three regularity conditions—Completeness, Prefix Consistency, and Neutrality—uniquely characterize token-level credit.
  • The analysis connects token-level credit to On-Policy Distillation and explains why response-level RLOO can produce the same expected policy-gradient contribution.
  • PACT uses importance-sampling correction during critic training to address policy mismatch and critic errors associated with GAE.
  • PACT reaches 72.87% average accuracy across four math benchmarks and 67.4% on SWE-bench Verified, outperforming the reported PPO, GRPO, and SAO baselines.
item →