‹ back to 2026-07-21

PRiSM: Prototype Regularization for Few-Shot VLMs

Research Multimodal & Generative

Merged summary

TL;DR - PRiSM is a plug-and-play prototype regularization method for training-free few-shot adaptation of vision-language models. It improves robustness when labeled examples are class-imbalanced or span many classes.

Introduces a Dirichlet-sampled benchmark varying class balance and effective class count.
Finds substantial degradation in existing methods under realistic imbalance, sometimes worsening with more labeled samples.
Optimizes prototypes using inter-class separation, support-feature alignment, and baseline-fidelity terms.
Uses an efficient block Majorize-Minimize optimizer with Lipschitz bounds derived via the Gershgorin circle theorem.

Sources (1)

PRiSM: Prototype Regularization for Few-Shot VLMs

arXiv cs.CV Ghassen Baklouti, Omprakash Chakraborty, Jose Dolz, Ismail Ben Ayed 2026-07-20 arXiv:2607.17820

Introduces a Dirichlet-sampled benchmark varying class balance and effective class count.
Finds substantial degradation in existing methods under realistic imbalance, sometimes worsening with more labeled samples.
Optimizes prototypes using inter-class separation, support-feature alignment, and baseline-fidelity terms.
Uses an efficient block Majorize-Minimize optimizer with Lipschitz bounds derived via the Gershgorin circle theorem.

item →