Using pooled electronic health records data to conduct pharmacoepidemiology safety studies: Challenges and lessons learned.

Objective: To evaluate the continuity and completeness of electronic health record (EHR) data, and the concordance of select clinical outcomes and baseline comorbidities between EHR and linked claims data, from three healthcare delivery systems in Taiwan. Methods: We identified oral hypoglycemic agent (OHA) users from the Integrated Medical Database of National Taiwan University Hospital (NTUH-iMD), which was linked to the National Health Insurance Research Database (NHIRD), from June 2011 to December 2016. A secondary evaluation involved two additional EHR databases. We created consecutive 90-day periods before and after the first recorded OHA prescription and defined patients as having continuous EHR data if there was at least one encounter or prescription in a 90-day interval. EHR data completeness was measured by dividing the number of encounters in the NTUH-iMD by the number of encounters in the NHIRD. We assessed the concordance between EHR and claims data on three clinical outcomes (cardiovascular events, nephropathy-related events, and heart failure admission). We used individual comorbidities that comprised the Charlson comorbidity index to examine the concordance of select baseline comorbidities between EHRs and claims. Results: We identified 39,268 OHA users in the NTUH-iMD. Thirty-one percent (n = 12,296) of these users contributed to the analysis that examined data continuity during the 6-month baseline and 24-month follow-up period; 31% (n = 3,845) of the 12,296 users had continuous data during this 30-month period and EHR data completeness was 52%. The concordance of major cardiovascular events, nephropathy-related events, and heart failure admission was moderate, with the NTU-iMD capturing 49–55% of the outcome events recorded in the NHIRD. The concordance of comorbidities was considerably different between the NTUH-iMD and NHIRD, with an absolute standardized difference >0.1 for most comorbidities examined. Across the three EHR databases studied, 29–55% of the OHA users had continuous records during the 6-month baseline and 24-month follow-up period. Conclusion: EHR data continuity and data completeness may be suboptimal. A thorough evaluation of data continuity and completeness is recommended before conducting clinical and translational research using EHR data in Taiwan.

Despite the extensive use of real-world data for retrospective, observational clinical research, our understanding of how real-world data might increase the efficiency of data collection in patient-level randomized clinical trials is largely unknown. The structure of real-world data is inherently heterogeneous, with each source electronic health record and claims database different from the next. Their fitness-for-use as data sources for multisite trials in the United States has not been established. For a subset of participants in the HARMONY Outcomes Trial, we obtained electronic health record data from recruiting sites or Medicare claims data from the Centers for Medicare & Medicaid Services. For baseline characteristics and follow-up events, we assessed the level of agreement between these real-world data and data documented in the trial database. Real-world data-derived demographic information tended to agree with trial-reported demographic information, although real-world data were less accurate in identifying medical history. The ability of real-world data to identify baseline medication usage differed by real-world data source, with claims data demonstrating substantially better performance than electronic health record data. The limited number of lab results in the collected electronic health record data matched closely with values in the trial database. There were not enough follow-up events in the ancillary study population to draw meaningful conclusions about the performance of real-world data for identification of events. Based on the conduct of this ancillary study, the challenges and opportunities of using real-world data within clinical trials are discussed. Based on a subset of participants from the HARMONY Outcomes Trial, our results suggest that electronic health record or claims data, as currently available, are unlikely to be a complete substitute for trial data collection of medical history or baseline lab results, but that Medicare claims were able to identify most medications. The limited size of the study population prevents us from drawing strong conclusions based on these results, and other studies are clearly needed to confirm or refute these findings.

Electronic health record (EHR) data are widely used in public health research, including in HIV-related studies, but are limited by potential bias due to incomplete and inaccurate information, lack of generalizability, and lack of representativeness. This study explores how workflow processes among HIV health care providers (HCPs), data scientists, and state health department professionals may potentially introduce or minimize bias within EHR data. One focus group with 3 health department professionals working in HIV surveillance and 16 in-depth interviews (ie, 5 people with HIV, 5 HCPs, 5 data scientists, and 1 health department professional providing retention-in-care services) were conducted with participants purposively sampled in South Carolina from August 2023 to April 2024. All interviews were transcribed verbatim and analyzed using a constructivist grounded theory approach, where transcripts were first coded and then focused, axial, and theoretically coded. The EHR data lifecycle originates with people with HIV and HCPs in the clinical setting. Data scientists then curate EHR data and health department professionals manage and use the data for surveillance and policy decision-making. Throughout this lifecycle, the three primary stakeholders (ie, HCPs, data scientists, and health department professionals) identified challenges with EHR processes and provided their recommendations and accommodations in addressing the related challenges. HCPs reported the influence of socio-structural biases on their inquiry, interpretation, and documentation of social determinants of health (SDOH) information of people living with HIV, the influence of which is proposed to be mitigated through people living with HIV access to their EHRs. Data scientists identified limited data availability and representativeness as biasing the data they manage. Health department professionals face challenges with delayed and incomplete data, which may be addressed statistically but require consideration of the data's limitations. Overall, bias within the EHR data lifecycle persists because workflows are not intentionally structured to minimize bias and there is a diffusion of responsibility for data quality between the various stakeholders. From the perspective of various stakeholders, this study describes the EHR data lifecycle and its associated challenges as well as stakeholders' accommodations and recommendations for mitigating and eliminating bias in EHR data. Based upon these findings, studies reliant on EHR data should adequately consider its challenges and limitations. Throughout the EHR data lifecycle, bias could be reduced through an inclusive, supportive health care environment, people living with HIV verification of SDOH information, the customization of data collection systems, and EHR data inspection for completeness, accuracy, and timeliness. Future research is needed to further identify instances where bias is introduced and how it can best be mitigated and eliminated across the EHR data lifecycle. Systematic changes are necessary to reduce instances of bias between data workflows and stakeholders.

Multicenter research of geriatric emergency department (GED) care remains limited. Our objectives were to: 1. Prospectively collect data prioritized by the Geriatric Emergency care Applied Research (GEAR) network, a transdisciplinary taskforce for GED care, and create a multicenter GED research repository of prospective and electronic health record (EHR) data, 2. Assess concordance between prospective and EHR data. The GEAR Standardization Study (GEARSS) is a multicenter, prospective study of older emergency department (ED) patients (65+) focusing on the 4Ms of age-friendly care (mobility, medication safety, mentation, what matters) and elder mistreatment. Demographic and clinical data were collected via interviews by trained research assistants (RA) on Days 0, 4, 30, and 90 and linked to EHR. Prevalence of chronic comorbidities and incident delirium were measured and reported using descriptive statistics. Prospective and EHR data concordance was assessed with Cohen's Kappa. 999 participants were recruited from 5 EDs (3/25/2021-6/30/2022) across 3 institutions: Grady Health System, Northwestern Memorial Hospital, and Yale New Haven Health. The cohort was 57.0% female, 55.2% White, 39.1% Black, and 3.4% Hispanic, and the mean age was 75.1 years. For rheumatologic disease, peptic ulcer disease, diabetes, renal disease, and cancer, prevalence differed between prospective and EHR data by > 10%. About two-thirds of participants were at risk for falls. Concordance between prospective and EHR data was good for ethnicity (K = 0.73); excellent for sex (K = 1.00), age (K = 1.00), and race (K = 0.98); fair for disposition (K = 0.53); slight for ED observation status (K = 0.33) and dementia diagnosis (K = 0.24); poor for delirium presence (K = 0.07). In GEARSS, demographic variables aligned strongly between prospective and EHR data, while diagnosis, disposition, and mentation factors did not. This multicenter data source provides preliminary findings for common geriatric syndromes and conditions. Choice of measures using these data should be driven by GED research questions.

Electronic health record (EHR) data are increasingly used to develop prediction models that guide clinical decision making at the point of care. These include algorithms that use high-frequency data, like in sepsis prediction, as well as simpler equations, such as the Pooled Cohort Equations for cardiovascular outcome prediction. Although EHR data used in prediction models are often highly granular and more current than other data, there is systematic and nonsystematic missingness in EHR data as there is with most data. Despite growing use for clinical decisions, algorithms implemented in EHRs are mostly unregulated and are often opaque to the user. Guidelines about the development, validation, implementation, and reporting on clinical prediction models are sparse in their recommendations regarding missing data. This article characterizes missingness in EHR data, summarizes methods for attending to missing data when developing prediction models, makes recommendations about validation and implementation of models in practice when data are missing, and identifies research needs in this field.

Although hypothesized to be the root cause of the pulse oximetry disparities, skin tone and its use for improving medical therapies have yet to be extensively studied. Studies previously used self-reported race as a proxy variable for skin tone. However, this approach cannot account for skin tone variability within race groups and also risks the potential to be confounded by other non-biological factors when modeling data. Therefore, to better evaluate health disparities associated with pulse oximetry, this study aimed to create a unique baseline dataset that included skin tone and electronic health record (EHR) data. Patients admitted to Duke University Hospital were eligible if they had at least one pulse oximetry value recorded within 5 minutes before an arterial blood gas (ABG) value. We collected skin tone data at 16 different body locations using multiple devices, including administered visual scales, colorimetric, spectrophotometric, and photography via mobile phone cameras. All patients' data were linked in Duke's Protected Analytics Computational Environment (PACE), converted into a common data model, and then de-identified before publication in PhysioNet. Skin tone data were collected from 128 patients. We assessed 167 features per skin location on each patient. We also collected over 2000 images from mobile phones measured in the same controlled environment. Skin tone data are linked with patients' EHR data, such as laboratory data, vital sign recordings, and demographic information. Measuring different aspects of skin tone for each of the sixteen body locations and linking them with patients' EHR data could assist in the development of a more equitable AI model to combat disparities in healthcare associated with skin tone. A common data model format enables easy data federation with similar data from other sources, facilitating multicenter research on skin tone in healthcare. A prospectively collected EHR-linked skin tone measurements database in a common data model with emphasis on pulse oximetry disparities.

To examine the value of unstructured electronic health record (EHR) data (free-text notes) in identifying a set of geriatric syndromes. Retrospective analysis of unstructured EHR notes using a natural language processing (NLP) algorithm. Large multispecialty group. Older adults (N=18,341; average age 75.9, 58.9% female). We compared the number of geriatric syndrome cases identified using structured claims and structured and unstructured EHR data. We also calculated these rates using a population-level claims database as a reference and identified comparable epidemiological rates in peer-reviewed literature as a benchmark. Using insurance claims data resulted in a geriatric syndrome prevalence ranging from 0.03% for lack of social support to 8.3% for walking difficulty. Using structured EHR data resulted in similar prevalence rates, ranging from 0.03% for malnutrition to 7.85% for walking difficulty. Incorporating unstructured EHR notes, enabled by applying the NLP algorithm, identified considerably higher rates of geriatric syndromes: absence of fecal control (2.1%, 2.3 times as much as structured claims and EHR data combined), decubitus ulcer (1.4%, 1.7 times as much), dementia (6.7%, 1.5 times as much), falls (23.6%, 3.2 times as much), malnutrition (2.5%, 18.0 times as much), lack of social support (29.8%, 455.9 times as much), urinary retention (4.2%, 3.9 times as much), vision impairment (6.2%, 7.4 times as much), weight loss (19.2%, 2.9 as much), and walking difficulty (36.34%, 3.4 as much). The geriatric syndrome rates extracted from structured data were substantially lower than published epidemiological rates, although adding the NLP results considerably closed this gap. Claims and structured EHR data give an incomplete picture of burden related to geriatric syndromes. Geriatric syndromes are likely to be missed if unstructured data are not analyzed. Pragmatic NLP algorithms can assist with identifying individuals at high risk of experiencing geriatric syndromes and improving coordination of care for older adults.

Established classifications exist to confirm Sjögren’s Disease (SD) (previously referred as Sjögren’s Syndrome) and recruit patients for research. However, no established classification exists for diagnosis in clinical settings causing delayed diagnosis. SD patients experience a huge dental disease burden impairing their quality of life. This study established criteria to characterize Indiana University School of Dentistry (IUSD) patients’ SD based on symptoms and signs in the electronic health record (EHR) data available through the state-wide Indiana health information exchange (IHIE). Association between SD diagnosis, and comorbidities including other autoimmune conditions, and documentation of SD diagnosis in electronic dental record (EDR) were also determined. The IUSD patients’ EDR were linked with their EHR data in the IHIE and queried for SD diagnostic ICD9/10 codes. The resulting cohorts’ EHR clinical findings were characterized and classified using diagnostic criteria based on clinical experts’ recommendations. Descriptive statistics were performed, and Chi-square tests determined the association between the different SD presentations and comorbidities including other autoimmune conditions. Eighty-three percent of IUSD patients had an EHR of which 377 patients had a SD diagnosis. They were characterized as positive (24%), uncertain (20%) and negative (56%) based on EHR clinical findings. Dry eyes and mouth were reported for 51% and positive Anti-Ro/SSA antibodies and anti-nuclear antibody (ANA) for 17% of this study cohort. One comorbidity was present in 98% and other autoimmune condition/s were present in 53% respectively. Significant differences were observed between the three SD clinical characteristics/classifications and certain medical and autoimmune conditions (p<0.05). Sixty-nine percent of patients’ EDR did not mention SD, highlighting the huge gap in reporting SD during dental care. This study of SD patients diagnosed in community practices characterized three different SD clinical presentations, which can be used to generate SD study cohorts for longitudinal studies using EHR data. The results emphasize the heterogenous SD clinical presentations and the need for further research to diagnose SD early in community practice settings where most people seek care.

Background: Effective and efficient methods are needed to identify naloxone administrations within electronic health record (EHR) data to conduct overdose surveillance and research. The objective of this study was to develop and validate a text-mining tool to identify naloxone administrations in EHR data. Methods: Clinical notes stored in databases between January 2017 and March 2018 were used to iteratively develop a text-mining tool to identify naloxone administrations. The first iteration of the tool used broad search terms. Then, after reviewing clinical notes of overdose encounters, we developed a list of phrases that described naloxone administrations to inform iteration two. While validating iteration two, additional phrases were found, which were then added to inform the final iteration. The comparator was an administrative code query extracted from the EHR. Medical record review was used to identify true positives. The primary outcome was the positive predictive values (PPV) of the second iteration, final iteration, and administrative code query. Results: Iteration two, the final iteration, and the administrative code had PPVs of 84.3% (95% confidence interval [CI] 78.6–89.0%), 83.8% (95% CI 78.6–88.2%), and 57.1% (95% CI 47.1–66.8%), respectively. Both iterations of the tool had a significantly higher PPV than the administrative code (p < 0.001). Conclusions: A text-mining tool improved the identification of naloxone administrations in EHR data from less than 60% with the administrative code to greater than 80% with both versions of the tool. Text-mining tools can inform the use of more sophisticated informatics methods, which often require significant time, resource, and expertise investment.

This December 11, 2014 think tank discussed issues around the methodologies for optimizing the robustness and use of Electronic Health Records (EHR) data for a variety of clinical research purposes. Experts in both accessing EHR data and study design and analysis methodology using EHR data for research highlighted the challenges, solutions, and needs in this area. Given the potential broad scope of this topic, participants were asked to focus primarily on issues related to the use of EHR on the back end (i.e. the imperfect data as currently collected), rather than strategies to improve the quality of EHR data collected on the 'front end' (e.g., data entered by clinicians).

As T1D treatments improve and the US population ages, there is a growing number of older adults (≥65 years) with T1D. This patient population is distinct from their T2D counterparts yet remains understudied. Analyzing routine health care may yield real-world evidence to augment trial data and inform best practices. Yet, it is essential to properly classify patients with T1D vs. T2D. We aimed to assess the extent to which codes for T1D versus T2D overlap in EHR data among older adults with diabetes and assess the implications for identifying older adults with T1D, specifically. We analyzed electronic health record (EHR) data from a large, public health care system in the southeast US, including 12 affiliated hospitals and over 200 academic and community-based practices. We included data from adults ages 65+ years that had at least one code for T1D or T2D. Of the 7,436 patients withT1D or T2D listed among their medical conditions, 7,253 (97.5%) had a diagnostic code for T2D and 895 (12.0%) had a diagnostic code for T1D. Of those with a T1D code, 787 (87.9%) also had at least one T2D code. Among those with a code for T1D, the mean percentage of codes that indicate T1D is 49.4%. There is a high degree of co-occurring T1D and T2D diagnosis codes in EHR profiles of older adults, with implications for clinical care and research. Cohort classification in diabetes-oriented RWE studies warrants further research. Disclosure J.M.Weinstein: Research Support; Dexcom, Inc. R.Muthukkumar: None. L.A.Young: Research Support; Novo Nordisk, Rhythm Pharmaceuticals, Inc., Eli Lilly and Company, Jaeb Center for Health Research, Sanofi, Boehringer-Ingelheim. A.R.Kahkoska: None. Funding National Institutes of Health (KL2TR002490, UL1TR002489)

Our practice-based research network (PBRN) is conducting an outreach intervention to increase health insurance coverage for patients seen in the network. To assist with outreach site selection, we sought an understandable way to use electronic health record (EHR) data to locate uninsured patients. Health insurance information was displayed within a web-based mapping platform to demonstrate the feasibility of using geographic information systems (GIS) to visualize EHR data. This study used EHR data from 52 clinics in the OCHIN PBRN. We included cross-sectional coverage data for patients aged 0 to 64 years with at least 1 visit to a study clinic during 2011 (n = 228,284). Our PBRN was successful in using GIS to identify intervention sites. Through use of the maps, we found geographic variation in insurance rates of patients seeking care in OCHIN PBRN clinics. Insurance rates also varied by age: The percentage of adults without insurance ranged from 13.2% to 86.8%; rates of children lacking insurance ranged from 1.1% to 71.7%. GIS also showed some areas of households with median incomes that had low insurance rates. EHR data can be imported into a web-based GIS mapping tool to visualize patient information. Using EHR data, we were able to observe smaller areas than could be seen using only publicly available data. Using this information, we identified appropriate OCHIN PBRN clinics for dissemination of an EHR-based insurance outreach intervention. GIS could also be used by clinics to visualize other patient-level characteristics to target clinic outreach efforts or interventions.

ObjectiveThe ability to reproduce the work of others is an essential part of the scientific disciplines. Replicating observational studies using electronic health record (EHR) data can be challenging due to complexities in data access, variations in EHR systems across institutions, and the potential for unaccounted confounding variables. Our aim is to identify the barriers to methods reproducibility for replication studies using EHR data.MethodsWe replicated a study that examined the risk of hospitalisation following a positive COVID-19 test in individuals with diabetes. Using EHR data from the NHS England’s Secure Data Environment (SDE) covering the whole of England, UK (population 57m), we sought to replicate findings from the original study, which used data from Greater Manchester (a large urban region in the UK, population 2.9m). Both analyses were conducted in Trusted Research Environments (TREs) or SDEs, containing linked primary and secondary care data, however methods reproducibility was not straightforward. Differences between the environments that contributed to the difficulties were documented, categorized into themes, and converted into a list of recommendations for TRE/SDEs.ResultsSmall differences between the environments and the data sources led to several challenges in methods reproducibility. Our recommendations of TRE/SDEs should facilitate future replication studies. The recommendations include: a need for improved machine-readable metadata for EHR data; standardization of governance processes to facilitate federated analysis; mandating of code sharing; and for environments to have a support structure for data engineers and analysts. We also propose a new theme for research, “data reproducibility”, as the ability to prepare, extract and clean data from a different database for a replication study.ConclusionEven with perfect code sharing, data reproducibility remains a challenge. Our recommendations have the potential to reduce the barriers to replication studies and therefore enhance the potential of observational studies using EHR data.

The increase in hospitalizations of patients with noncommunicable diseases remains a key challenge for healthcare systems. A significant portion of these hospitalizations is potentially preventable with timely outpatient monitoring and effective primary care. Predictive models based on data from health information systems (HIS) and electronic health records (EHRs) make it possible to identify patients at increased risk of hospitalization and improve their management strategies. The article systematizes and summarizes international experience in using hospitalization risk prognostic models, developed on the basis of HIS and EHR data. We made a systematic review of publications presented in Russian and foreign databases (Scopus, PubMed, BMC Health Services Research, BMJ Research, Google Scholar, Elibrary, Oxford Academic, Wiley Online Library) for the period 1993-2023. The analysis included studies that used EHR and HIS data to develop and/or validate prognostic models of hospitalization risk. The information is systematized by following predictor categories: diseases and comorbidities, prescriptions, interaction with healthcare organizations, sociodemographic and laboratory parameters. The final analysis included 14 studies describing 28 models for predicting the hospitalization risk. Most studies used logistic regression. Machine learning methods (gradient boosting, random forest, and Bayesian ensemble models) were used in some studies. The categories with the highest prognostic value were "diseases and comorbidities" (42,3%), "medication prescriptions" (23,6%), and "interaction with health organizations" (19,8%). Including temporal characteristics (frequency and recency of hospitalizations, visits to primary care facilities, and laboratory tests) increased the model accuracy. Predictive models based on EHR and HIS data demonstrate high discriminatory power and enable the assessment of hospitalization risk at the primary care level. Machine learning methods in some studies demonstrated the possibility of a more in-depth analysis of the relationships between predictors and improved prediction accuracy. A promising area of research is the further development, external validation, and adaptation of models using expanded EHR and HIS data sets for its use in outpatient monitoring and preventing noncommunicable diseases.

Background Defining clinical conditions from electronic health record (EHR) data underpins population health activities, clinical decision support, and analytics. In an EHR, defining a condition commonly employs a diagnosis value set or “grouper.” For constructing value sets, Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) offers high clinical fidelity, a hierarchical ontology, and wide implementation in EHRs as the standard interoperability vocabulary for problems. Objective This article demonstrates a practical approach to defining conditions with combinations of SNOMED CT concept hierarchies, and evaluates sharing of definitions for clinical and analytic uses. Methods We constructed diagnosis value sets for EHR patient registries using SNOMED CT concept hierarchies combined with Boolean logic, and shared them for clinical decision support, reporting, and analytic purposes. Results A total of 125 condition-defining “standard” SNOMED CT diagnosis value sets were created within our EHR. The median number of SNOMED CT concept hierarchies needed was only 2 (25th–75th percentiles: 1–5). Each value set, when compiled as an EHR diagnosis grouper, was associated with a median of 22 International Classification of Diseases (ICD)-9 and ICD-10 codes (25th–75th percentiles: 8–85) and yielded a median of 155 clinical terms available for selection by clinicians in the EHR (25th–75th percentiles: 63–976). Sharing of standard groupers for population health, clinical decision support, and analytic uses was high, including 57 patient registries (with 362 uses of standard groupers), 132 clinical decision support records, 190 rules, 124 EHR reports, 125 diagnosis dimension slicers for self-service analytics, and 111 clinical quality measure calculations. Identical SNOMED CT definitions were created in an EHR-agnostic tool enabling application across disparate organizations and EHRs. Conclusion SNOMED CT-based diagnosis value sets are simple to develop, concise, understandable to clinicians, useful in the EHR and for analytics, and shareable. Developing curated SNOMED CT hierarchy-based condition definitions for public use could accelerate cross-organizational population health efforts, “smarter” EHR feature configuration, and clinical–translational research employing EHR-derived data.