Medical AI Encodes a "Feeling of Error": Verifying Cancer Segmentation via Internal Concepts

TL;DR - This paper finds that cancer segmentation models encode internal activation patterns that distinguish successful predictions from failures. Detecting this latent “feeling of error” could flag unreliable masks without sacrificing segmentation quality.

• Sparse autoencoders decompose internal neural activations into human-interpretable concepts.
• Failed segmentations exhibit fewer active concepts and lower activation magnitudes than successful cases.
• A classifier trained on these concept activations detects failures and provides explanations tied to internal model concepts.
• Across prostate, pancreatic, and brain cancer segmentation, the method outperforms output-based failure detection approaches while preserving segmentation quality.