01 - BD2K ThinkTank on EHR data semantic harmonization, definition, content, ontologies: OHDSI, CDRNs, NLP

Objective: To evaluate the continuity and completeness of electronic health record (EHR) data, and the concordance of select clinical outcomes and baseline comorbidities between EHR and linked claims data, from three healthcare delivery systems in Taiwan. Methods: We identified oral hypoglycemic agent (OHA) users from the Integrated Medical Database of National Taiwan University Hospital (NTUH-iMD), which was linked to the National Health Insurance Research Database (NHIRD), from June 2011 to December 2016. A secondary evaluation involved two additional EHR databases. We created consecutive 90-day periods before and after the first recorded OHA prescription and defined patients as having continuous EHR data if there was at least one encounter or prescription in a 90-day interval. EHR data completeness was measured by dividing the number of encounters in the NTUH-iMD by the number of encounters in the NHIRD. We assessed the concordance between EHR and claims data on three clinical outcomes (cardiovascular events, nephropathy-related events, and heart failure admission). We used individual comorbidities that comprised the Charlson comorbidity index to examine the concordance of select baseline comorbidities between EHRs and claims. Results: We identified 39,268 OHA users in the NTUH-iMD. Thirty-one percent (n = 12,296) of these users contributed to the analysis that examined data continuity during the 6-month baseline and 24-month follow-up period; 31% (n = 3,845) of the 12,296 users had continuous data during this 30-month period and EHR data completeness was 52%. The concordance of major cardiovascular events, nephropathy-related events, and heart failure admission was moderate, with the NTU-iMD capturing 49–55% of the outcome events recorded in the NHIRD. The concordance of comorbidities was considerably different between the NTUH-iMD and NHIRD, with an absolute standardized difference >0.1 for most comorbidities examined. Across the three EHR databases studied, 29–55% of the OHA users had continuous records during the 6-month baseline and 24-month follow-up period. Conclusion: EHR data continuity and data completeness may be suboptimal. A thorough evaluation of data continuity and completeness is recommended before conducting clinical and translational research using EHR data in Taiwan.

Electronic health record (EHR) data are widely used in public health research, including in HIV-related studies, but are limited by potential bias due to incomplete and inaccurate information, lack of generalizability, and lack of representativeness. This study explores how workflow processes among HIV health care providers (HCPs), data scientists, and state health department professionals may potentially introduce or minimize bias within EHR data. One focus group with 3 health department professionals working in HIV surveillance and 16 in-depth interviews (ie, 5 people with HIV, 5 HCPs, 5 data scientists, and 1 health department professional providing retention-in-care services) were conducted with participants purposively sampled in South Carolina from August 2023 to April 2024. All interviews were transcribed verbatim and analyzed using a constructivist grounded theory approach, where transcripts were first coded and then focused, axial, and theoretically coded. The EHR data lifecycle originates with people with HIV and HCPs in the clinical setting. Data scientists then curate EHR data and health department professionals manage and use the data for surveillance and policy decision-making. Throughout this lifecycle, the three primary stakeholders (ie, HCPs, data scientists, and health department professionals) identified challenges with EHR processes and provided their recommendations and accommodations in addressing the related challenges. HCPs reported the influence of socio-structural biases on their inquiry, interpretation, and documentation of social determinants of health (SDOH) information of people living with HIV, the influence of which is proposed to be mitigated through people living with HIV access to their EHRs. Data scientists identified limited data availability and representativeness as biasing the data they manage. Health department professionals face challenges with delayed and incomplete data, which may be addressed statistically but require consideration of the data's limitations. Overall, bias within the EHR data lifecycle persists because workflows are not intentionally structured to minimize bias and there is a diffusion of responsibility for data quality between the various stakeholders. From the perspective of various stakeholders, this study describes the EHR data lifecycle and its associated challenges as well as stakeholders' accommodations and recommendations for mitigating and eliminating bias in EHR data. Based upon these findings, studies reliant on EHR data should adequately consider its challenges and limitations. Throughout the EHR data lifecycle, bias could be reduced through an inclusive, supportive health care environment, people living with HIV verification of SDOH information, the customization of data collection systems, and EHR data inspection for completeness, accuracy, and timeliness. Future research is needed to further identify instances where bias is introduced and how it can best be mitigated and eliminated across the EHR data lifecycle. Systematic changes are necessary to reduce instances of bias between data workflows and stakeholders.

We assessed the suitability of pooled electronic health record (EHR) data from clinical research networks (CRNs) of the patient-centered outcomes research network to conduct studies of the association between tumor necrosis factor inhibitors (TNFi) and infections. EHR data from patients with one of seven autoimmune diseases were obtained from three CRNs and pooled. Person-level linkage of CRN data and Centers for Medicare and Medicaid Services (CMS) fee-for-service claims data was performed where possible. Using filled prescriptions from CMS claims data as the gold standard, we assessed the misclassification of EHR-based new (incident) user definitions. Among new users of TNFi, we assessed subsequent rates of hospitalized infection in EHR and CMS data. The study included 45 483 new users of TNFi, of whom 1416 were successfully linked to their CMS claims. Overall, 44% of new EHR TNFi prescriptions were not associated with medication claims. Our most specific new user definition had a misclassification rate of 3.5%-16.4% for prevalent use, depending on the medication. Greater than 80% of CRN prescriptions had either zero refills or missing refill data. Compared to using EHR data alone, there was a 2- to 8-fold increase in hospitalized infection rates when CMS claims data were added to the analysis. EHR data substantially misclassified TNFi exposure and underestimated the incidence of hospitalized infections compared to claims data. EHR-based new user definitions were reasonably accurate. Overall, using CRN data for pharmacoepidemiology studies is challenging, especially for biologics, and would benefit from supplementation by other sources.

IntroductionThe ability to utilize the electronic health record (EHR) without compromising the doctor-patient relationship (DPR) is an essential skill of all physicians and trainees, yet little time is spent on educating or assessing learners on needed techniques. To address this gap, we developed a conventional OSCE station coupled with a simulated patient chart within the Epic program in order to assess our students' skills utilizing the EHR during a patient encounter.MethodsOf third-year medical students, 119 were given full access to the patient's simulated chart 24 hours in advance of their OSCE to review clinical data. During an in-person OSCE with a standardized patient (SP), students performed a focused history and physical, using the EHR to verify allergies and medications. Students completed an electronic patient note graded by faculty. SPs evaluated the students on communication and interpersonal skills with specific rubric elements. Faculty graded the students' notes to evaluate their expression of clinical reasoning in the assessment and plan.ResultsTraining SPs and faculty to assess students on EHR skills was feasible. After implementation of a comprehensive curriculum focused on EHR and DPR, there was a significant difference on EHR-related communication skills (M = 76.4, SD = 17.6) versus (M = 37, SD = 28.9) before curriculum enhancement t (117.9) = −12.4, p <.001.DiscussionThe EHR OSCE station provided a standardized method of assessing students' EHR skills during a patient encounter. Challenges still exist in the technological requirements to develop and deliver cases in today's EHR platform.

Multicenter research of geriatric emergency department (GED) care remains limited. Our objectives were to: 1. Prospectively collect data prioritized by the Geriatric Emergency care Applied Research (GEAR) network, a transdisciplinary taskforce for GED care, and create a multicenter GED research repository of prospective and electronic health record (EHR) data, 2. Assess concordance between prospective and EHR data. The GEAR Standardization Study (GEARSS) is a multicenter, prospective study of older emergency department (ED) patients (65+) focusing on the 4Ms of age-friendly care (mobility, medication safety, mentation, what matters) and elder mistreatment. Demographic and clinical data were collected via interviews by trained research assistants (RA) on Days 0, 4, 30, and 90 and linked to EHR. Prevalence of chronic comorbidities and incident delirium were measured and reported using descriptive statistics. Prospective and EHR data concordance was assessed with Cohen's Kappa. 999 participants were recruited from 5 EDs (3/25/2021-6/30/2022) across 3 institutions: Grady Health System, Northwestern Memorial Hospital, and Yale New Haven Health. The cohort was 57.0% female, 55.2% White, 39.1% Black, and 3.4% Hispanic, and the mean age was 75.1 years. For rheumatologic disease, peptic ulcer disease, diabetes, renal disease, and cancer, prevalence differed between prospective and EHR data by > 10%. About two-thirds of participants were at risk for falls. Concordance between prospective and EHR data was good for ethnicity (K = 0.73); excellent for sex (K = 1.00), age (K = 1.00), and race (K = 0.98); fair for disposition (K = 0.53); slight for ED observation status (K = 0.33) and dementia diagnosis (K = 0.24); poor for delirium presence (K = 0.07). In GEARSS, demographic variables aligned strongly between prospective and EHR data, while diagnosis, disposition, and mentation factors did not. This multicenter data source provides preliminary findings for common geriatric syndromes and conditions. Choice of measures using these data should be driven by GED research questions.

BackgroundHealthcare providers are often targeted as research participants, especially for implementation science studies evaluating provider- or system-level issues. Frequently, provider eligibility is based on both provider and patient factors. Manual chart review and self-report are common provider screening strategies but require substantial time, effort, and resources. The automated use of electronic health record (EHR) data may streamline provider identification for implementation science research. Here, we describe an approach to provider screening for a Veterans Health Administration (VHA)-funded study focused on implementing risk-aligned surveillance for bladder cancer patients.MethodsOur goal was to identify providers at 6 pre-specified facilities who performed ≥10 surveillance cystoscopy procedures among bladder cancer patients in the 12 months prior to recruitment start on January 16, 2020, and who were currently practicing at 1 of 6 pre-specified facilities. Using VHA EHR data (using CPT, ICD10 procedure, and ICD10 diagnosis codes), we identified cystoscopy procedures performed after an initial bladder cancer diagnosis (i.e., surveillance procedures). Procedures were linked to VHA staff data to determine the provider of record, the number of cystoscopies they performed, and their current location of practice. To validate this approach, we performed a chart review of 105 procedures performed by a random sample of identified providers. The proportion of correctly identified procedures was calculated (Positive Predictive Value (PPV)), along with binomial 95% confidence intervals (CI).FindingsWe identified 1,917,856 cystoscopies performed on 703,324 patients from October 1, 1999—January 16, 2020, across the nationwide VHA. Of those procedures, 40% were done on patients who had a prior record of bladder cancer and were completed by 15,065 distinct providers. Of those, 61 performed ≥ 10 procedures and were currently practicing at 1 of the 6 facilities of interest in the 1 year prior to study recruitment. The random chart review of 7 providers found 101 of 105 procedures (PPV: 96%; 95% CI: 91% to 99%) were surveillance procedures and were performed by the selected provider on the recorded date.ImplicationsThese results show that EHR data can be used for accurate identification of healthcare providers as research participants when inclusion criteria consist of both patient- (temporal relationship between diagnosis and procedure) and provider-level (frequency of procedure and location of current practice) factors. As administrative codes and provider identifiers are collected in most, if not all, EHRs for billing purposes this approach can be translated from provider recruitment in VHA to other healthcare systems. Implementation studies should consider this method of screening providers.

Electronic health record (EHR) data are increasingly used to develop prediction models that guide clinical decision making at the point of care. These include algorithms that use high-frequency data, like in sepsis prediction, as well as simpler equations, such as the Pooled Cohort Equations for cardiovascular outcome prediction. Although EHR data used in prediction models are often highly granular and more current than other data, there is systematic and nonsystematic missingness in EHR data as there is with most data. Despite growing use for clinical decisions, algorithms implemented in EHRs are mostly unregulated and are often opaque to the user. Guidelines about the development, validation, implementation, and reporting on clinical prediction models are sparse in their recommendations regarding missing data. This article characterizes missingness in EHR data, summarizes methods for attending to missing data when developing prediction models, makes recommendations about validation and implementation of models in practice when data are missing, and identifies research needs in this field.

Although hypothesized to be the root cause of the pulse oximetry disparities, skin tone and its use for improving medical therapies have yet to be extensively studied. Studies previously used self-reported race as a proxy variable for skin tone. However, this approach cannot account for skin tone variability within race groups and also risks the potential to be confounded by other non-biological factors when modeling data. Therefore, to better evaluate health disparities associated with pulse oximetry, this study aimed to create a unique baseline dataset that included skin tone and electronic health record (EHR) data. Patients admitted to Duke University Hospital were eligible if they had at least one pulse oximetry value recorded within 5 minutes before an arterial blood gas (ABG) value. We collected skin tone data at 16 different body locations using multiple devices, including administered visual scales, colorimetric, spectrophotometric, and photography via mobile phone cameras. All patients' data were linked in Duke's Protected Analytics Computational Environment (PACE), converted into a common data model, and then de-identified before publication in PhysioNet. Skin tone data were collected from 128 patients. We assessed 167 features per skin location on each patient. We also collected over 2000 images from mobile phones measured in the same controlled environment. Skin tone data are linked with patients' EHR data, such as laboratory data, vital sign recordings, and demographic information. Measuring different aspects of skin tone for each of the sixteen body locations and linking them with patients' EHR data could assist in the development of a more equitable AI model to combat disparities in healthcare associated with skin tone. A common data model format enables easy data federation with similar data from other sources, facilitating multicenter research on skin tone in healthcare. A prospectively collected EHR-linked skin tone measurements database in a common data model with emphasis on pulse oximetry disparities.

With the worldwide digitalisation of medical records, electronic health records (EHRs) have become an increasingly important source of real-world data (RWD). RWD can complement traditional study designs because it captures...

Background Defining clinical conditions from electronic health record (EHR) data underpins population health activities, clinical decision support, and analytics. In an EHR, defining a condition commonly employs a diagnosis value set or “grouper.” For constructing value sets, Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) offers high clinical fidelity, a hierarchical ontology, and wide implementation in EHRs as the standard interoperability vocabulary for problems. Objective This article demonstrates a practical approach to defining conditions with combinations of SNOMED CT concept hierarchies, and evaluates sharing of definitions for clinical and analytic uses. Methods We constructed diagnosis value sets for EHR patient registries using SNOMED CT concept hierarchies combined with Boolean logic, and shared them for clinical decision support, reporting, and analytic purposes. Results A total of 125 condition-defining “standard” SNOMED CT diagnosis value sets were created within our EHR. The median number of SNOMED CT concept hierarchies needed was only 2 (25th–75th percentiles: 1–5). Each value set, when compiled as an EHR diagnosis grouper, was associated with a median of 22 International Classification of Diseases (ICD)-9 and ICD-10 codes (25th–75th percentiles: 8–85) and yielded a median of 155 clinical terms available for selection by clinicians in the EHR (25th–75th percentiles: 63–976). Sharing of standard groupers for population health, clinical decision support, and analytic uses was high, including 57 patient registries (with 362 uses of standard groupers), 132 clinical decision support records, 190 rules, 124 EHR reports, 125 diagnosis dimension slicers for self-service analytics, and 111 clinical quality measure calculations. Identical SNOMED CT definitions were created in an EHR-agnostic tool enabling application across disparate organizations and EHRs. Conclusion SNOMED CT-based diagnosis value sets are simple to develop, concise, understandable to clinicians, useful in the EHR and for analytics, and shareable. Developing curated SNOMED CT hierarchy-based condition definitions for public use could accelerate cross-organizational population health efforts, “smarter” EHR feature configuration, and clinical–translational research employing EHR-derived data.

Established classifications exist to confirm Sjögren’s Disease (SD) (previously referred as Sjögren’s Syndrome) and recruit patients for research. However, no established classification exists for diagnosis in clinical settings causing delayed diagnosis. SD patients experience a huge dental disease burden impairing their quality of life. This study established criteria to characterize Indiana University School of Dentistry (IUSD) patients’ SD based on symptoms and signs in the electronic health record (EHR) data available through the state-wide Indiana health information exchange (IHIE). Association between SD diagnosis, and comorbidities including other autoimmune conditions, and documentation of SD diagnosis in electronic dental record (EDR) were also determined. The IUSD patients’ EDR were linked with their EHR data in the IHIE and queried for SD diagnostic ICD9/10 codes. The resulting cohorts’ EHR clinical findings were characterized and classified using diagnostic criteria based on clinical experts’ recommendations. Descriptive statistics were performed, and Chi-square tests determined the association between the different SD presentations and comorbidities including other autoimmune conditions. Eighty-three percent of IUSD patients had an EHR of which 377 patients had a SD diagnosis. They were characterized as positive (24%), uncertain (20%) and negative (56%) based on EHR clinical findings. Dry eyes and mouth were reported for 51% and positive Anti-Ro/SSA antibodies and anti-nuclear antibody (ANA) for 17% of this study cohort. One comorbidity was present in 98% and other autoimmune condition/s were present in 53% respectively. Significant differences were observed between the three SD clinical characteristics/classifications and certain medical and autoimmune conditions (p<0.05). Sixty-nine percent of patients’ EDR did not mention SD, highlighting the huge gap in reporting SD during dental care. This study of SD patients diagnosed in community practices characterized three different SD clinical presentations, which can be used to generate SD study cohorts for longitudinal studies using EHR data. The results emphasize the heterogenous SD clinical presentations and the need for further research to diagnose SD early in community practice settings where most people seek care.

Background: Effective and efficient methods are needed to identify naloxone administrations within electronic health record (EHR) data to conduct overdose surveillance and research. The objective of this study was to develop and validate a text-mining tool to identify naloxone administrations in EHR data. Methods: Clinical notes stored in databases between January 2017 and March 2018 were used to iteratively develop a text-mining tool to identify naloxone administrations. The first iteration of the tool used broad search terms. Then, after reviewing clinical notes of overdose encounters, we developed a list of phrases that described naloxone administrations to inform iteration two. While validating iteration two, additional phrases were found, which were then added to inform the final iteration. The comparator was an administrative code query extracted from the EHR. Medical record review was used to identify true positives. The primary outcome was the positive predictive values (PPV) of the second iteration, final iteration, and administrative code query. Results: Iteration two, the final iteration, and the administrative code had PPVs of 84.3% (95% confidence interval [CI] 78.6–89.0%), 83.8% (95% CI 78.6–88.2%), and 57.1% (95% CI 47.1–66.8%), respectively. Both iterations of the tool had a significantly higher PPV than the administrative code (p < 0.001). Conclusions: A text-mining tool improved the identification of naloxone administrations in EHR data from less than 60% with the administrative code to greater than 80% with both versions of the tool. Text-mining tools can inform the use of more sophisticated informatics methods, which often require significant time, resource, and expertise investment.

The increase in hospitalizations of patients with noncommunicable diseases remains a key challenge for healthcare systems. A significant portion of these hospitalizations is potentially preventable with timely outpatient monitoring and effective primary care. Predictive models based on data from health information systems (HIS) and electronic health records (EHRs) make it possible to identify patients at increased risk of hospitalization and improve their management strategies. The article systematizes and summarizes international experience in using hospitalization risk prognostic models, developed on the basis of HIS and EHR data. We made a systematic review of publications presented in Russian and foreign databases (Scopus, PubMed, BMC Health Services Research, BMJ Research, Google Scholar, Elibrary, Oxford Academic, Wiley Online Library) for the period 1993-2023. The analysis included studies that used EHR and HIS data to develop and/or validate prognostic models of hospitalization risk. The information is systematized by following predictor categories: diseases and comorbidities, prescriptions, interaction with healthcare organizations, sociodemographic and laboratory parameters. The final analysis included 14 studies describing 28 models for predicting the hospitalization risk. Most studies used logistic regression. Machine learning methods (gradient boosting, random forest, and Bayesian ensemble models) were used in some studies. The categories with the highest prognostic value were "diseases and comorbidities" (42,3%), "medication prescriptions" (23,6%), and "interaction with health organizations" (19,8%). Including temporal characteristics (frequency and recency of hospitalizations, visits to primary care facilities, and laboratory tests) increased the model accuracy. Predictive models based on EHR and HIS data demonstrate high discriminatory power and enable the assessment of hospitalization risk at the primary care level. Machine learning methods in some studies demonstrated the possibility of a more in-depth analysis of the relationships between predictors and improved prediction accuracy. A promising area of research is the further development, external validation, and adaptation of models using expanded EHR and HIS data sets for its use in outpatient monitoring and preventing noncommunicable diseases.

As T1D treatments improve and the US population ages, there is a growing number of older adults (≥65 years) with T1D. This patient population is distinct from their T2D counterparts yet remains understudied. Analyzing routine health care may yield real-world evidence to augment trial data and inform best practices. Yet, it is essential to properly classify patients with T1D vs. T2D. We aimed to assess the extent to which codes for T1D versus T2D overlap in EHR data among older adults with diabetes and assess the implications for identifying older adults with T1D, specifically. We analyzed electronic health record (EHR) data from a large, public health care system in the southeast US, including 12 affiliated hospitals and over 200 academic and community-based practices. We included data from adults ages 65+ years that had at least one code for T1D or T2D. Of the 7,436 patients withT1D or T2D listed among their medical conditions, 7,253 (97.5%) had a diagnostic code for T2D and 895 (12.0%) had a diagnostic code for T1D. Of those with a T1D code, 787 (87.9%) also had at least one T2D code. Among those with a code for T1D, the mean percentage of codes that indicate T1D is 49.4%. There is a high degree of co-occurring T1D and T2D diagnosis codes in EHR profiles of older adults, with implications for clinical care and research. Cohort classification in diabetes-oriented RWE studies warrants further research. Disclosure J.M.Weinstein: Research Support; Dexcom, Inc. R.Muthukkumar: None. L.A.Young: Research Support; Novo Nordisk, Rhythm Pharmaceuticals, Inc., Eli Lilly and Company, Jaeb Center for Health Research, Sanofi, Boehringer-Ingelheim. A.R.Kahkoska: None. Funding National Institutes of Health (KL2TR002490, UL1TR002489)

ObjectiveThe ability to reproduce the work of others is an essential part of the scientific disciplines. Replicating observational studies using electronic health record (EHR) data can be challenging due to complexities in data access, variations in EHR systems across institutions, and the potential for unaccounted confounding variables. Our aim is to identify the barriers to methods reproducibility for replication studies using EHR data.MethodsWe replicated a study that examined the risk of hospitalisation following a positive COVID-19 test in individuals with diabetes. Using EHR data from the NHS England’s Secure Data Environment (SDE) covering the whole of England, UK (population 57m), we sought to replicate findings from the original study, which used data from Greater Manchester (a large urban region in the UK, population 2.9m). Both analyses were conducted in Trusted Research Environments (TREs) or SDEs, containing linked primary and secondary care data, however methods reproducibility was not straightforward. Differences between the environments that contributed to the difficulties were documented, categorized into themes, and converted into a list of recommendations for TRE/SDEs.ResultsSmall differences between the environments and the data sources led to several challenges in methods reproducibility. Our recommendations of TRE/SDEs should facilitate future replication studies. The recommendations include: a need for improved machine-readable metadata for EHR data; standardization of governance processes to facilitate federated analysis; mandating of code sharing; and for environments to have a support structure for data engineers and analysts. We also propose a new theme for research, “data reproducibility”, as the ability to prepare, extract and clean data from a different database for a replication study.ConclusionEven with perfect code sharing, data reproducibility remains a challenge. Our recommendations have the potential to reduce the barriers to replication studies and therefore enhance the potential of observational studies using EHR data.