Through the Looking Glass: Directly Reading and Writing Transformers

TL;DR - This paper introduces a training-free method for reading and directly editing transformer components using only model parameters and activations. It finds that predictions depend critically on small component subsets and demonstrates targeted, low-cost installation and control of internal associations.

• Although thousands of components contribute to logits, only 2–16 components can independently suffice for a prediction across tested 124M–7B parameter models.
• Tracing a prediction backward implicates just 1–3% of the model, a fraction that does not increase with model size.
• Parameter-derived descriptions identify what components write and read, with read-side identification reaching 58.9% above chance for the eight strongest inputs.
• New associations and context-dependent edits can be installed in spare units or attention-head/unit circuits at substantially lower held-out-loss cost than rank-one updates.