‹ back to 2026-07-19

Understanding Reasoning from Pretraining to Post-Training

arXiv cs.LG LLMs & Foundation Models Jingyan Shen, Ang Li, Salman Rahman, Yifan Sun, Micah Goldblum, Matus Telgarsky, Pavel Izmailov 2026-07-17

TL;DR - A controlled study using chess (and later math) as a testbed to disentangle how pretraining choices shape the returns to RL post-training in LLMs, showing pretraining loss predicts post-RL reasoning performance. It matters because it quantifies the often-ignored pretraining→RL interface that drives reasoning gains.

Trains LLMs (5M–1B params) on human chess games, applies SFT on synthetic reasoning traces, then RL on chess puzzles with verifiable rewards—mirroring the standard LLM pipeline in a controlled setting.
Post-RL performance at a given RL compute level is well-predicted by pretraining loss, and the slope of RL reward curves improves roughly linearly with pretraining tokens.
RL doesn't merely sharpen the SFT policy: on easy puzzles it amplifies already-preferred correct moves, while on hard puzzles it surfaces correct moves nearly absent under SFT.
Findings transfer beyond chess—a 1B model trained on math-domain text shows the same pattern: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL.

view merged work →