Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Ranking
Overall
90
Content
100
Popularity
67
Observed public metrics from 1 member.
Merged summary
TL;DR - This paper unifies reverse-trajectory and forward-matching reinforcement learning methods for diffusion models under one path-space policy-gradient framework. It argues that their empirical differences primarily arise from variance reduction and derives a recipe that improves prior diffusion-RL baselines.
- Derives an explicit trajectory-space policy gradient using importance sampling between sampling SDEs.
- Connects Flow-GRPO-style updates with the forward-matching structure of AWM and DiffusionNFT.
- Organizes diffusion-RL design around value-gradient estimation, weighting functions, and sampling choices.
- Proposes a rollout-reusing KDE value-gradient estimator and scale-bounded weights, validated on SD3.5-M and Qwen-Image.
Sources (1)
Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Public signals
Semantic Scholar citations 1 · Semantic Scholar influential citations 0
TL;DR - This paper unifies reverse-trajectory and forward-matching reinforcement learning methods for diffusion models under one path-space policy-gradient framework. It argues that their empirical differences primarily arise from variance reduction and derives a recipe that improves prior diffusion-RL baselines.
- Derives an explicit trajectory-space policy gradient using importance sampling between sampling SDEs.
- Connects Flow-GRPO-style updates with the forward-matching structure of AWM and DiffusionNFT.
- Organizes diffusion-RL design around value-gradient estimation, weighting functions, and sampling choices.
- Proposes a rollout-reusing KDE value-gradient estimator and scale-bounded weights, validated on SD3.5-M and Qwen-Image.