Statistical Inference for Rank Allocation in Low-Rank Adaptation
Ranking
Overall
76
Content
90
Popularity
43
Observed public metrics from 1 member.
Merged summary
TL;DR - StatLoRA frames rank allocation for parameter-efficient fine-tuning as statistical hypothesis testing, using estimated p-values to retain or prune LoRA components. It matches or outperforms several LoRA baselines under equal rank budgets.
- Establishes asymptotic normality for stochastic optimizer trajectories, including AdamW.
- Derives distributions for component-level test statistics used in rank allocation.
- Evaluates DeBERTaV3-base, BART-Large, and Qwen2.5-7B across understanding, generation, and question-answering tasks.
- Reports stable allocation behavior and empirical support for the proposed asymptotic theory.
Sources (1)
Statistical Inference for Rank Allocation in Low-Rank Adaptation
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - StatLoRA frames rank allocation for parameter-efficient fine-tuning as statistical hypothesis testing, using estimated p-values to retain or prune LoRA components. It matches or outperforms several LoRA baselines under equal rank budgets.
- Establishes asymptotic normality for stochastic optimizer trajectories, including AdamW.
- Derives distributions for component-level test statistics used in rank allocation.
- Evaluates DeBERTaV3-base, BART-Large, and Qwen2.5-7B across understanding, generation, and question-answering tasks.
- Reports stable allocation behavior and empirical support for the proposed asymptotic theory.