🛰️ Daily AI Frontier
‹ back to 2026-08-17

别再乱调图像塔了！浙大 IJCAI 论文揭露 VLM 非对称性真相，给 CLIP 微调「踩刹车」

Research Multimodal & Generative

Ranking

Overall 75
Content 85
Popularity N/A

No observed public metrics; popularity remains neutral/archived.

Representative image for 别再乱调图像塔了！浙大 IJCAI 论文揭露 VLM 非对称性真相，给 CLIP 微调「踩刹车」

Merged summary

TL;DR - An IJCAI paper proposes an adaptive asymmetric adapter for few-shot CLIP tuning that freely adapts the text encoder while suppressing image-encoder updates on uncertain inputs. This preserves pretrained visual generalization under distribution shift while retaining in-domain adaptation.

  • Experiments across 11 datasets find text-side tuning generally more beneficial, while aggressive image-side tuning harms out-of-distribution performance.
  • A confidence-driven regularizer pushes image-adapter corrections toward zero for low-confidence samples, effectively reverting to pretrained CLIP visual features.
  • The adapter uses a shared down-projection, multiple routed up-projection experts, and load balancing to prevent expert collapse.
  • It reportedly outperforms 11 baselines across base-to-novel, cross-dataset, and domain-generalization evaluations, with lightweight ViT and ResNet-50 support.

Sources (1)

别再乱调图像塔了！浙大 IJCAI 论文揭露 VLM 非对称性真相，给 CLIP 微调「踩刹车」

雷峰网 (AI科技评论) 2026-08-17
Public signals N/A
Providers: Hugging Face · N/A OpenAlex · N/A Publisher · N/A Semantic Scholar · N/A X · N/A Fetched 2026-09-16 14:20:06.276057 UTC

TL;DR - An IJCAI paper proposes an adaptive asymmetric adapter for few-shot CLIP tuning that freely adapts the text encoder while suppressing image-encoder updates on uncertain inputs. This preserves pretrained visual generalization under distribution shift while retaining in-domain adaptation.

  • Experiments across 11 datasets find text-side tuning generally more beneficial, while aggressive image-side tuning harms out-of-distribution performance.
  • A confidence-driven regularizer pushes image-adapter corrections toward zero for low-confidence samples, effectively reverting to pretrained CLIP visual features.
  • The adapter uses a shared down-projection, multiple routed up-projection experts, and load balancing to prevent expert collapse.
  • It reportedly outperforms 11 baselines across base-to-novel, cross-dataset, and domain-generalization evaluations, with lightweight ViT and ResNet-50 support.
item →