拟合接近满分，外推却会跑偏：Meta 给缩放律加了一个指数
Ranking
Overall
75
Content
85
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - Meta FAIR’s Skaling scaling law adds a coupling exponent between model size and training data, substantially improving extrapolation over Chinchilla on key pretraining grids. Its L-shaped sampling strategy also reduces the compute needed to fit scaling laws.
- Skaling reduced extrapolation error by roughly 1.5–3× in major experiments; on the full SK-Grid, far-extrapolation MAPE fell from 5.17% to 0.70%.
- L-shaped sampling estimates scaling behavior from low-cost model-size and data boundaries, cutting fitting-grid compute by about 4.8–9.8×—not final pretraining cost.
- The coupling exponent captures model–data interactions that Chinchilla’s additive formulation omits while reverting to Chinchilla when the exponent equals one.
- Results vary across datasets, so the coupling strength and compute-optimal token-to-parameter allocation are not yet universal.
Sources (1)
拟合接近满分，外推却会跑偏：Meta 给缩放律加了一个指数
Public signals
N/A
TL;DR - Meta FAIR’s Skaling scaling law adds a coupling exponent between model size and training data, substantially improving extrapolation over Chinchilla on key pretraining grids. Its L-shaped sampling strategy also reduces the compute needed to fit scaling laws.
- Skaling reduced extrapolation error by roughly 1.5–3× in major experiments; on the full SK-Grid, far-extrapolation MAPE fell from 5.17% to 0.70%.
- L-shaped sampling estimates scaling behavior from low-cost model-size and data boundaries, cutting fitting-grid compute by about 4.8–9.8×—not final pretraining cost.
- The coupling exponent captures model–data interactions that Chinchilla’s additive formulation omits while reverting to Chinchilla when the exponent equals one.
- Results vary across datasets, so the coupling strength and compute-optimal token-to-parameter allocation are not yet universal.