One Adapter Pair per Model: A Universal Activation Interface for Language Models
Ranking
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - A "Universal Activation Bus" learns a shared dense activation space plus one lightweight linear encoder–decoder adapter pair per language model, so interpretability tools built for one model can be reused on others. It matters because probes, SAEs, and natural-language interpreters currently must be rebuilt from scratch for every new model.
- Trained on a small set of source models to induce a shared space; once frozen, a new model joins by fitting only its adapter pair on unlabeled matched text — no retraining of the interface or the tools.
- Enables cross-model reuse of probes, sparse autoencoder features, and an NLA (natural-language interpreter) originally trained for a different model.
- Evaluated across five models: semantically related texts form consistent neighborhoods in the shared space, and an onboarded model reuses existing tools effectively.
- Demonstrates activation transplantation — an intermediate activation from one model can be fed into another model's frozen upper layers to produce predictions, suggesting a stable model-wise "activation contract."
Sources (1)
One Adapter Pair per Model: A Universal Activation Interface for Language Models
TL;DR - A "Universal Activation Bus" learns a shared dense activation space plus one lightweight linear encoder–decoder adapter pair per language model, so interpretability tools built for one model can be reused on others. It matters because probes, SAEs, and natural-language interpreters currently must be rebuilt from scratch for every new model.
- Trained on a small set of source models to induce a shared space; once frozen, a new model joins by fitting only its adapter pair on unlabeled matched text — no retraining of the interface or the tools.
- Enables cross-model reuse of probes, sparse autoencoder features, and an NLA (natural-language interpreter) originally trained for a different model.
- Evaluated across five models: semantically related texts form consistent neighborhoods in the shared space, and an onboarded model reuses existing tools effectively.
- Demonstrates activation transplantation — an intermediate activation from one model can be fed into another model's frozen upper layers to produce predictions, suggesting a stable model-wise "activation contract."