Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit
TL;DR - Circuit Condensation post-trains models so a target behavior is carried by a smaller causal circuit, making mechanistic explanations easier to inspect and verify. Across eight models and four behaviors, it reduced circuits by 8.1Ă— on average versus the strongest frozen-model baseline while preserving task performance and general capabilities.
- Each round prunes low-attribution edges and trains a low-rank adapter to reproduce the original behavior through the remaining graph.
- Condensed circuits were smaller in 30 of 32 settings, with reductions reaching 316Ă—; control searches without weight updates produced larger circuits in 29 of 32 settings.
- Exhaustive subset testing found 11 of 19 circuits irreducible, while pair ablations showed that edge effects can depend on one another.
- For indirect object identification, condensation isolated 24 attention heads, including 17 with documented roles, versus 61 heads in the matched frozen circuit.