AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

TL;DR - AE-PSL is a split-learning framework that uses a lightweight autoencoder to reduce activation and gradient communication when fine-tuning foundation models across constrained edge devices. Its two-stage alignment aims to preserve compatibility with pretrained and client-specific feature distributions.

• Keeps only a few model layers on clients while offloading most computation to a server.
• Compresses intermediate activations and gradients at the model’s split layer.
• Aligns the autoencoder with the pretrained model’s feature manifold before distributed fine-tuning.
• Further adapts compression to client-specific feature distributions to mitigate representation misalignment.