Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - A controlled synthetic study finds that adding more interventional data to pretraining does not teach LMs correct causal direction; instead, the evidence type present in the inference-time context determines whether causal knowledge is used. It matters because it challenges the assumption that interventional pretraining data alone buys causal reasoning.
- In Simpson's-paradox worlds, raising the interventional fraction in pretraining grows the magnitude of the model's do()-response monotonically, but its sign is copied from the observational context.
- Under an identical training recipe, context type drives outcomes: purely observational contexts cause sign reversal in 29/50 worlds, mixed in 19/50, while aligned interventional probes alone give 41/50 correct; erasing observational evidence releases suppressed causal interpolation (ratio_true = +0.56), and a four-state content manipulation shows the switch is content-mediated and graded.
- Effects are stable across seeds (11/11 strong reversals on a matched-protocol second seed) and persist as a rate at 0.93B parameters (31.8% vs. 6% reversals in the probe-only arm), though absolute gains shrink four-fold; activation patching localizes the switch to middle-layer observational rows.
- A CLadder audit reveals a learned positive-effect prior with two-layer structure (sign-randomized retraining removes it in-distribution but not out-of-distribution); the authors also quantify probe-based evaluation's sampling noise floor and an evidence-averaging protocol cutting sign errors from 26% to 9%.
Sources (1)
Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
TL;DR - A controlled synthetic study finds that adding more interventional data to pretraining does not teach LMs correct causal direction; instead, the evidence type present in the inference-time context determines whether causal knowledge is used. It matters because it challenges the assumption that interventional pretraining data alone buys causal reasoning.
- In Simpson's-paradox worlds, raising the interventional fraction in pretraining grows the magnitude of the model's do()-response monotonically, but its sign is copied from the observational context.
- Under an identical training recipe, context type drives outcomes: purely observational contexts cause sign reversal in 29/50 worlds, mixed in 19/50, while aligned interventional probes alone give 41/50 correct; erasing observational evidence releases suppressed causal interpolation (ratio_true = +0.56), and a four-state content manipulation shows the switch is content-mediated and graded.
- Effects are stable across seeds (11/11 strong reversals on a matched-protocol second seed) and persist as a rate at 0.93B parameters (31.8% vs. 6% reversals in the probe-only arm), though absolute gains shrink four-fold; activation patching localizes the switch to middle-layer observational rows.
- A CLadder audit reveals a learned positive-effect prior with two-layer structure (sign-randomized retraining removes it in-distribution but not out-of-distribution); the authors also quantify probe-based evaluation's sampling noise floor and an evidence-averaging protocol cutting sign errors from 26% to 9%.