Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
Ranking
Overall
77
Content
95
Popularity
35
Observed public metrics from 1 member.
Merged summary
TL;DR - A controlled GPT-2 pre-training study directly measures the effect of replacing one batch row with a single 194-token passage. The passage is briefly learned and later forgotten, while its injection leaves a substantial weight displacement that remains within the same loss basin.
- Across eight seeds, injected models predicted the passage 0.039–0.044 nats better after 50 steps, but no significant advantage was detected at training’s end.
- The study trained 32 GPT-2 124M models on OpenWebText, comparing real-subject prose, gradient-matched fabricated-subject prose, random characters, and uninjected twins.
- Final models showed no detectable condition differences in interpolation loss barriers, held-out cross-entropy, or per-layer centered kernel alignment.
- Weight displacement reached 44.1% of seed-to-seed Euclidean distance, yet the loss barrier was only 3.0% of its seed-to-seed counterpart, suggesting relocation within—not escape from—the existing basin.
Sources (1)
Learned, Then Lost: A Measured Single-Example Counterfactual in Pre-training
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - A controlled GPT-2 pre-training study directly measures the effect of replacing one batch row with a single 194-token passage. The passage is briefly learned and later forgotten, while its injection leaves a substantial weight displacement that remains within the same loss basin.
- Across eight seeds, injected models predicted the passage 0.039–0.044 nats better after 50 steps, but no significant advantage was detected at training’s end.
- The study trained 32 GPT-2 124M models on OpenWebText, comparing real-subject prose, gradient-matched fabricated-subject prose, random characters, and uninjected twins.
- Final models showed no detectable condition differences in interpolation loss barriers, held-out cross-entropy, or per-layer centered kernel alignment.
- Weight displacement reached 44.1% of seed-to-seed Euclidean distance, yet the loss barrier was only 3.0% of its seed-to-seed counterpart, suggesting relocation within—not escape from—the existing basin.