Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
Merged summary
TL;DR — OCP-CT is a vision-language pretraining framework that aligns 3D CT volumes with radiology reports at the fine-grained level of "organ-conditioned radiological patterns," improving zero-shot abnormality diagnosis over prior methods.
- Motivation: global scan-report contrast obscures per-organ evidence, while direct organ-level alignment is too coarse since one anatomy can present multiple distinct radiological appearances.
- Method: keeps a stable global CT-report contrastive branch plus an "organ pattern interface" — sparse Mixture-of-Experts routes image/text tokens by latent radiological pattern, learnable slots query them into continuous pattern tokens, and paired token contrast aligns image-text pattern tokens using soft targets from report-derived clinical similarity.
- Results: zero-shot abnormality diagnosis average AUROC of 84.5% on CT-RATE and 69.9% on RAD-ChestCT, absolute gains of +6.7 and +0.8 points over the strongest prior reported results.
- Note: cross-modal (3D CT imaging + text) medical VLM, evaluated on the public CT-RATE and RAD-ChestCT benchmarks.
Sources (1)
Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
TL;DR — OCP-CT is a vision-language pretraining framework that aligns 3D CT volumes with radiology reports at the fine-grained level of "organ-conditioned radiological patterns," improving zero-shot abnormality diagnosis over prior methods.
- Motivation: global scan-report contrast obscures per-organ evidence, while direct organ-level alignment is too coarse since one anatomy can present multiple distinct radiological appearances.
- Method: keeps a stable global CT-report contrastive branch plus an "organ pattern interface" — sparse Mixture-of-Experts routes image/text tokens by latent radiological pattern, learnable slots query them into continuous pattern tokens, and paired token contrast aligns image-text pattern tokens using soft targets from report-derived clinical similarity.
- Results: zero-shot abnormality diagnosis average AUROC of 84.5% on CT-RATE and 69.9% on RAD-ChestCT, absolute gains of +6.7 and +0.8 points over the strongest prior reported results.
- Note: cross-modal (3D CT imaging + text) medical VLM, evaluated on the public CT-RATE and RAD-ChestCT benchmarks.