Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
Merged summary
TL;DR - MseaCL is a multimodal contrastive learning framework for 3D pediatric brain MRI and radiology reports that reduces false negatives by using semantic similarity between reports as a guiding signal, yielding large downstream gains in clinical tasks.
- Targets a known flaw in standard contrastive learning: treating all non-paired samples as negatives creates false negatives when samples share high-level semantic attributes, common in medical data.
- Uses semantic similarity between radiology reports to guide alignment during pretraining, rather than assuming strict instance-based correspondence within a batch.
- Trained on a pediatric cohort of 3D brain MRI scans paired with radiology reports.
- Reports at least a 22.6% AUC increase on pediatric brain tumor molecular classification when used as a pretraining stage.
Sources (1)
Multimodal Semantic-Aware Contrastive Learning For False Negative Mitigation in 3D Medical Imaging
TL;DR - MseaCL is a multimodal contrastive learning framework for 3D pediatric brain MRI and radiology reports that reduces false negatives by using semantic similarity between reports as a guiding signal, yielding large downstream gains in clinical tasks.
- Targets a known flaw in standard contrastive learning: treating all non-paired samples as negatives creates false negatives when samples share high-level semantic attributes, common in medical data.
- Uses semantic similarity between radiology reports to guide alignment during pretraining, rather than assuming strict instance-based correspondence within a batch.
- Trained on a pediatric cohort of 3D brain MRI scans paired with radiology reports.
- Reports at least a 22.6% AUC increase on pediatric brain tumor molecular classification when used as a pretraining stage.