Medical AI has a measurement problem

TL;DR - Nature highlights a measurement problem exposed by two medical AI assistants: capabilities are advancing faster than reliable evaluation methods. The limited item content frames the challenge but provides no specific results.

• Existing assessments might not adequately establish which medical AI systems work.
• Evaluation must keep pace with rapidly evolving assistant capabilities.
• Trustworthy adoption depends on defining robust measures of real-world effectiveness.