Towards Privacy-Preserving Federated Prompt Tuning under Data Heterogeneity: A Subspace-Decomposed Expert Approach

TL;DR - FedSEPT is a privacy-preserving federated prompt-tuning method for vision-language models that represents multiple prompt experts in compact shared subspaces. It aims to improve personalization and global generalization under data heterogeneity without expanding differential-privacy noise and communication costs.

• Shared low-rank factors, a fixed public basis, and private residuals decompose prompt experts into compact components.
• Only the compact factor space is communicated and perturbed for local differential privacy, enabling direct server aggregation.
• On-device routing adaptively combines complementary experts, with cached text features supporting efficient logit-level fusion.
• Across 11 heterogeneous benchmarks, FedSEPT reportedly outperforms strong baselines on the local-versus-global trade-off under equal privacy constraints.