‹ back to 2026-07-24

How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

Research LLMs & Foundation Models

Ranking

Overall 79

Content 95

Popularity 42

Observed public metrics from 1 member.

Merged summary

TL;DR - This paper quantifies how much information LoRA adapters memorize when fine-tuning frozen models. It finds that capacity and privacy leakage depend more on adapter placement and base-model structure than raw parameter count.

Adapters store a few bits per trainable parameter, less than full fine-tuning.
Moving an equal parameter budget from attention to MLP layers nearly doubles capacity.
Capacity largely disappears when the frozen base model’s structure is removed.
In Qwen2.5 experiments, leakage tracks written bits; supervised tuning memorizes secrets verbatim, while training on verifiable rewards does not.

Sources (1)

How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

arXiv cs.LG Kaizhen Tan, Heqing Du, Yang Feng 2026-07-23 arXiv:2607.21351

Public signals Semantic Scholar citations 0 · Semantic Scholar influential citations 0

Providers: Hugging Face · N/A OpenAlex · N/A Publisher · N/A Semantic Scholar · Citations 0 · Influential citations 0 X · N/A Fetched 2026-08-21 14:36:43.571135 UTC

Adapters store a few bits per trainable parameter, less than full fine-tuning.
Moving an equal parameter budget from attention to MLP layers nearly doubles capacity.
Capacity largely disappears when the frozen base model’s structure is removed.
In Qwen2.5 experiments, leakage tracks written bits; supervised tuning memorizes secrets verbatim, while training on verifiable rewards does not.

item →