通用模型竟然比医疗专用模型更懂医疗？一篇 ACL 论文的两个反直觉发现 | GAIR Paper 123

TL;DR - MedErrBench is the first multilingual benchmark for detecting, locating, and correcting medical errors across native English, Chinese, and Arabic data. Its ACL 2026 Findings results challenge assumptions that medical-specialized models or English-language inputs necessarily perform best.

• The benchmark covers 10 clinical error types and evaluates models on error detection, localization, and correction, with clinician-reviewed annotations for key terms, difficulty, and reasoning type.
• General-purpose models outperformed medical-specialized models such as MedGemma and HuatuoGPT; Doubao-1.5-Thinking-Pro led the English evaluation with 0.779 detection and 0.774 localization accuracy.
• Doubao and DeepSeek generally surpassed the tested GPT-4o and Gemini versions across the three languages, suggesting broad reasoning and instruction-following capabilities are critical alongside medical knowledge.
• Models performed better on native Chinese data than translated variants, indicating that translation can lose clinically important context; the benchmark still lacks error-severity labels and has limited Arabic data.