Towards Privacy-Preserving Federated Prompt Tuning under Data Heterogeneity: A Subspace-Decomposed Expert Approach
Merged summary
TL;DR - FedSEPT is a privacy-preserving federated prompt-tuning method for vision-language models that represents multiple prompt experts in compact shared subspaces. It aims to improve personalization and global generalization under data heterogeneity without expanding differential-privacy noise and communication costs.
- Shared low-rank factors, a fixed public basis, and private residuals decompose prompt experts into compact components.
- Only the compact factor space is communicated and perturbed for local differential privacy, enabling direct server aggregation.
- On-device routing adaptively combines complementary experts, with cached text features supporting efficient logit-level fusion.
- Across 11 heterogeneous benchmarks, FedSEPT reportedly outperforms strong baselines on the local-versus-global trade-off under equal privacy constraints.
Sources (1)
Towards Privacy-Preserving Federated Prompt Tuning under Data Heterogeneity: A Subspace-Decomposed Expert Approach
TL;DR - FedSEPT is a privacy-preserving federated prompt-tuning method for vision-language models that represents multiple prompt experts in compact shared subspaces. It aims to improve personalization and global generalization under data heterogeneity without expanding differential-privacy noise and communication costs.
- Shared low-rank factors, a fixed public basis, and private residuals decompose prompt experts into compact components.
- Only the compact factor space is communicated and perturbed for local differential privacy, enabling direct server aggregation.
- On-device routing adaptively combines complementary experts, with cached text features supporting efficient logit-level fusion.
- Across 11 heterogeneous benchmarks, FedSEPT reportedly outperforms strong baselines on the local-versus-global trade-off under equal privacy constraints.