MedPixel: A Unified Pixel-Language Model for Medical Reasoning and Segmentation

TL;DR - MedPixel is a unified medical pixel-language model that couples clinical language reasoning with pixel-level segmentation through a shared language–mask interface, closing the gap between medical VLMs that can't localize and segmenters that need explicit categories or spatial prompts.

• Addresses a supervision mismatch: segmentation data has masks but little language; medical VL data has language but few dense annotations.
• Introduces MedPLG-440K, ~440K pixel-language task samples built via a clinically motivated synthesis process with no external LLM annotation.
• Training is joint multi-task SFT followed by Pixel-Level Preference Optimization, using ground-truth masks as offline verifiers to derive response preferences from mask quality.
• Covers explicit grounding, implicit reasoning, spatial interaction, grounded explanation, and medical VQA; reports zero-shot transfer to external grounding benchmarks and robustness to imperfect spatial prompts. Code/checkpoints promised at github.com/yhy-whu/Medpixel.