Through the Looking Glass: Directly Reading and Writing Transformers
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - This paper introduces a training-free method for reading and directly editing transformer components using only model parameters and activations. It finds that predictions depend critically on small component subsets and demonstrates targeted, low-cost installation and control of internal associations.
- Although thousands of components contribute to logits, only 2–16 components can independently suffice for a prediction across tested 124M–7B parameter models.
- Tracing a prediction backward implicates just 1–3% of the model, a fraction that does not increase with model size.
- Parameter-derived descriptions identify what components write and read, with read-side identification reaching 58.9% above chance for the eight strongest inputs.
- New associations and context-dependent edits can be installed in spare units or attention-head/unit circuits at substantially lower held-out-loss cost than rank-one updates.
Sources (1)
Through the Looking Glass: Directly Reading and Writing Transformers
Public signals
N/A
TL;DR - This paper introduces a training-free method for reading and directly editing transformer components using only model parameters and activations. It finds that predictions depend critically on small component subsets and demonstrates targeted, low-cost installation and control of internal associations.
- Although thousands of components contribute to logits, only 2–16 components can independently suffice for a prediction across tested 124M–7B parameter models.
- Tracing a prediction backward implicates just 1–3% of the model, a fraction that does not increase with model size.
- Parameter-derived descriptions identify what components write and read, with read-side identification reaching 58.9% above chance for the eight strongest inputs.
- New associations and context-dependent edits can be installed in spare units or attention-head/unit circuits at substantially lower held-out-loss cost than rank-one updates.