Multi-Agent AI System for Radiology Report Structuring and Quality Assurance with Independent Radiologist Evaluation

TL;DR - A locally deployed multi-agent system used rules and LLMs to structure 638 radiology reports and flag potential quality issues. Independent radiologists found generally favorable performance, suggesting utility for report standardization and clinical QA.

• Structured 22,270 sentences from CT reports into predefined anatomical sections while retaining the original content.
• Flagged 90 reports (14.1%), primarily for mismatches involving Findings and Impression sections.
• Both reviewers judged 31 of 45 sampled reports correctly restructured and agreed that none omitted important information or introduced fabricated content.
• Reviewers rated overall QA performance “excellent” or “good” for 84% of the evaluated reports.