OrthKD: Extracting Generalized Clinical Knowledge from Heterogeneous Teachers for Lightweight Deployment
Ranking
Overall
77
Content
90
Popularity
45
Observed public metrics from 1 member.
Merged summary
TL;DR - OrthKD selectively distills complementary knowledge from heterogeneous teachers into a lightweight diabetic-retinopathy screening model, improving edge deployment and robustness under domain shift.
- Uses full supervision from a stronger EfficientNet-B3 teacher but only feature-level supervision from a weaker Swin-Base teacher.
- Orthogonal student projections encourage teacher-specific features to contribute complementary local and global evidence.
- The 5.4M-parameter MobileNetV3 student achieves 0.885 QWK on EyePACS.
- Zero-shot Messidor-2 performance rises from 0.507 to 0.728 QWK, alongside strong referral AUC and calibration.
Sources (1)
OrthKD: Extracting Generalized Clinical Knowledge from Heterogeneous Teachers for Lightweight Deployment
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - OrthKD selectively distills complementary knowledge from heterogeneous teachers into a lightweight diabetic-retinopathy screening model, improving edge deployment and robustness under domain shift.
- Uses full supervision from a stronger EfficientNet-B3 teacher but only feature-level supervision from a weaker Swin-Base teacher.
- Orthogonal student projections encourage teacher-specific features to contribute complementary local and global evidence.
- The 5.4M-parameter MobileNetV3 student achieves 0.885 QWK on EyePACS.
- Zero-shot Messidor-2 performance rises from 0.507 to 0.728 QWK, alongside strong referral AUC and calibration.