BEHRT: Transformer for Electronic Health Records

• https://www.nature.com/articles/s41598-020-62922-y
• Language models for health records

About 1.6M patients using UK’s Clinical Practice Research Datalink (CPRD) dataset and HES (hospitalisations, outpatient visits, accident and emergency for all admissions to National Health Service (NHS) hospitals in England) dataset.

They kept those with at least 5 visits.

Only considered 301 codes for diagnoses, not going into the detailed codes, by using Caliber codes (an expert checked mapping dictionary from University College London).