ConfTriage: A Calibration-Aware LLM Triage Framework for Pulmonary Nodule Malignancy with Selective Specialist Deferral

TL;DR - ConfTriage is a calibration-aware framework where a generalist LLM reads structured natural-language descriptions of pulmonary nodules to triage malignancy risk, deferring only low-confidence cases to a specialist image-trained DL model. It matters because it shows clinically useful diagnostic signal can be extracted from text alone, reducing reliance on heavily annotated imaging pipelines.

• Three design pillars: language as the input modality, confidence calibration as the safety mechanism, and a selective specialist DL backstop for uncertain cases.
• Two theoretical guarantees: a finite-sample combined-error bound giving a per-threshold operational certificate, and an oracle inequality bounding excess risk over the Bayes-optimal deferral classifier by the LLM's L1 calibration error.
• On LIDC-IDRI: F1 of 88.22% and AUC of 0.92, with 76.5% of cases resolved by zero-shot LLM inference alone.
• A seven-way input ablation across five frontier LLMs found natural-language attribute descriptions carry the diagnostic signal, while low-level image statistics were essentially uninformative; code is released publicly.