‹ back to 2026-08-05

SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs

Research LLMs & Foundation Models

Ranking

Overall 90

Content 100

Popularity 68

Observed public metrics from 1 member.

Merged summary

TL;DR - This paper argues that reinforcement learning handles multi-task LLM reasoning more effectively than supervised fine-tuning by producing sparse, nearly orthogonal task updates. It introduces Parallel-RL to improve training efficiency and flexibility.

Multi-stage SFT exhibits severe gradient conflicts across diverse tasks, while RL supports stable coexistence.
SFT interference scales with absolute gradient magnitude; RL interference is bounded by gradient variance.
Advantage normalization and on-policy optimization help keep RL gradient variance small and task directions near-orthogonal.
Parallel-RL decouples multi-task training based on these findings.

Sources (1)

SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs

arXiv cs.CL Kejian Zhu, Zhuoran Jin, Shangqing Tu, Hongbang Yuan, Yushi Bai, Kang Liu, Juanzi Li, Jun Zhao 2026-08-04 arXiv:2608.03573

Public signals Hugging Face upvotes 60

Providers: Hugging Face · Upvotes 60 OpenAlex · N/A Publisher · N/A Semantic Scholar · N/A X · N/A Fetched 2026-09-03 14:32:54.102911 UTC

Multi-stage SFT exhibits severe gradient conflicts across diverse tasks, while RL supports stable coexistence.
SFT interference scales with absolute gradient magnitude; RL interference is bounded by gradient variance.
Advantage normalization and on-policy optimization help keep RL gradient variance small and task directions near-orthogonal.
Parallel-RL decouples multi-task training based on these findings.

item →