MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities

TL;DR - MOSAIC is a federated framework for weakly supervised tumor segmentation when hospitals have different missing imaging modalities. It approaches fully supervised performance using only image-level labels while preserving decentralized data.

• A modality-agnostic alignment module maps each client’s available imaging channels into a shared latent space without knowing modality identities.
• Spectral prototype alignment uses compact, non-invertible frequency-domain statistics to reduce cross-client distribution shift.
• A federated refinement network converts noisy class activation map pseudo-labels into more accurate segmentation masks.
• Across three brain tumor benchmarks, MOSAIC outperformed image-, box-, and point-supervised baselines, reaching 0.84 Dice on FeTS2022; new institutions joined without federation-wide retraining and performed within 0.01–0.04 Dice.