Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records

TL;DR - BERT-LER is an explainable BERT-style model for structured EHR timelines, pretrained on de-identified records from 75 million patients. It combines percentile-binned laboratory values with token-level Integrated Gradients explanations, delivering competitive clinical predictions and clinically plausible attributions.

• Represents quantitative laboratory results as discrete tokens while preserving graded information through percentile-based binning.
• Evaluated on the public EHRShot benchmark and a real-world asthma severity progression study.
• Matches benchmark models overall and often outperforms them on laboratory-related tasks.
• Produces input-event attributions that align with established clinical risk factors.