博士论文 | 后训练如何损害大模型生成多样性？SimpleStrat与Stylus
Merged summary
TL;DR - A UC Berkeley doctoral thesis examines how SFT and RLHF can collapse generative diversity, then proposes training-free methods to restore structured exploration in language and diffusion models without substantially sacrificing quality.
- SimpleStrat partitions answer spaces into semantic strata and samples across them, improving valid-answer coverage beyond temperature scaling alone.
- On CoverageQA, it improves recall for open and closed models with only small precision losses; in browser-agent task generation, difficult tasks rose from about 20% to as high as 60%.
- Stylus retrieves and composes LoRA adapters to diversify image generation while improving the quality–alignment tradeoff.
- The thesis argues that post-training objectives are mode-seeking and motivates diversity-preserving RL and sampling without replacement.
Sources (1)
博士论文 | 后训练如何损害大模型生成多样性？SimpleStrat与Stylus
TL;DR - A UC Berkeley doctoral thesis examines how SFT and RLHF can collapse generative diversity, then proposes training-free methods to restore structured exploration in language and diffusion models without substantially sacrificing quality.
- SimpleStrat partitions answer spaces into semantic strata and samples across them, improving valid-answer coverage beyond temperature scaling alone.
- On CoverageQA, it improves recall for open and closed models with only small precision losses; in browser-agent task generation, difficult tasks rose from about 20% to as high as 60%.
- Stylus retrieves and composes LoRA adapters to diversify image generation while improving the quality–alignment tradeoff.
- The thesis argues that post-training objectives are mode-seeking and motivates diversity-preserving RL and sampling without replacement.