Pretraining EHR Foundation Models with Patient-Aware Sampling
Merged summary
TL;DR - This paper proposes patient-aware sampling for pretraining autoregressive EHR foundation models, avoiding mixed-patient windows and controlling each patient’s training contribution. It improves downstream clinical performance over conventional global-stream sampling.
- Standard token-stream sampling can overrepresent patients with longer records.
- Stochastic Patient Sampling supports controllable weighting of training signals across patients.
- It improves Macro AUROC and AUPRC on clinical tasks using MIMIC-IV v2.2 and v3.1.
- Sequence construction is identified as an important, underexplored EHR pretraining design choice.
Sources (1)
Pretraining EHR Foundation Models with Patient-Aware Sampling
TL;DR - This paper proposes patient-aware sampling for pretraining autoregressive EHR foundation models, avoiding mixed-patient windows and controlling each patient’s training contribution. It improves downstream clinical performance over conventional global-stream sampling.
- Standard token-stream sampling can overrepresent patients with longer records.
- Stochastic Patient Sampling supports controllable weighting of training signals across patients.
- It improves Macro AUROC and AUPRC on clinical tasks using MIMIC-IV v2.2 and v3.1.
- Sequence construction is identified as an important, underexplored EHR pretraining design choice.