The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
Ranking
Overall
65
Content
75
Popularity
43
Observed public metrics from 1 member.
Merged summary
TL;DR - PortLLM’s LoRA patches remain effective across 10 continual-pretraining updates on multiple model families. The paper attributes this temporal portability to near-orthogonality in high-dimensional parameter spaces.
- Evaluates Mistral, Gemma, and Qwen across 10 continual-pretraining steps.
- Finds that repeated fine-tuning may be unnecessary after periodic base-model updates.
- Provides two theoretical analyses explaining PortLLM’s competitive performance.
- Uses loss-landscape geometry to compare model-adaptation options.
Sources (1)
The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - PortLLM’s LoRA patches remain effective across 10 continual-pretraining updates on multiple model families. The paper attributes this temporal portability to near-orthogonality in high-dimensional parameter spaces.
- Evaluates Mistral, Gemma, and Qwen across 10 continual-pretraining steps.
- Finds that repeated fine-tuning may be unnecessary after periodic base-model updates.
- Provides two theoretical analyses explaining PortLLM’s competitive performance.
- Uses loss-landscape geometry to compare model-adaptation options.