🛰️ Daily AI Frontier
‹ back to 2026-08-28

TTPO: Test-Time Policy Optimization

Research LLMs & Foundation Models

Ranking

Overall 87
Content 95
Popularity 68

Observed public metrics from 1 member.

Merged summary

TL;DR - TTPO is a label-free test-time policy optimization method that combines self-distillation on majority-agreeing rollouts with grouped reinforcement learning on disagreeing ones. It matches label-supervised OPSD across five competition-level mathematical reasoning benchmarks while improving Qwen3-1.7B test-time accuracy from 38.0% to 45.2%.

  • Uses an asymmetric objective designed to remain useful even when majority-vote pseudo-labels are incorrect.
  • Distills agreeing rollouts via on-policy self-distillation and penalizes disagreeing rollouts through grouped reinforcement learning.
  • Applies token-level selection to down-weight converged positions and penalize only confident errors.
  • Reports strong label-free gains, including 25.2%–36.4% improvements without thinking and cross-task generalization.

Sources (1)

TTPO: Test-Time Policy Optimization

arXiv cs.CL Aozhe Wang, Zhengxi Lu, Jianze Wang, Shangke Lv, Ying Liu, Weiming Lu, Jun Xiao, Yueting Zhuang, Hua Yang, Qianglong Chen, Yongliang Shen 2026-08-27 arXiv:2608.27448
Public signals Hugging Face upvotes 83
Providers: Hugging Face · Upvotes 83 OpenAlex · N/A Publisher · N/A Semantic Scholar · N/A X · N/A Fetched 2026-09-25 14:27:05.942665 UTC

TL;DR - TTPO is a label-free test-time policy optimization method that combines self-distillation on majority-agreeing rollouts with grouped reinforcement learning on disagreeing ones. It matches label-supervised OPSD across five competition-level mathematical reasoning benchmarks while improving Qwen3-1.7B test-time accuracy from 38.0% to 45.2%.

  • Uses an asymmetric objective designed to remain useful even when majority-vote pseudo-labels are incorrect.
  • Distills agreeing rollouts via on-policy self-distillation and penalizes disagreeing rollouts through grouped reinforcement learning.
  • Applies token-level selection to down-weight converged positions and penalize only confident errors.
  • Reports strong label-free gains, including 25.2%–36.4% improvements without thinking and cross-task generalization.
item →