MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places

TL;DR - MAP is a benchmark for evaluating whether multimodal AI assistants can reliably help users plan real-world visits around accessibility requirements. It matters because accessibility details and place information change over time and require verifiable, location-specific evidence.

• Evaluates both verification and recommendation of points of interest that meet requested accessibility features.
• Tests whether systems can determine if accessibility claims are supported and identify qualifying places.
• Measures retrieval of relevant visual evidence for the specified place and accessibility requirement.
• Supports scheduled ground-truth refreshes, automated scoring, and human review of a subset of responses.