Identifying naloxone administrations in electronic health record data using a text-mining tool

BackgroundHealthcare providers are often targeted as research participants, especially for implementation science studies evaluating provider- or system-level issues. Frequently, provider eligibility is based on both provider and patient factors. Manual chart review and self-report are common provider screening strategies but require substantial time, effort, and resources. The automated use of electronic health record (EHR) data may streamline provider identification for implementation science research. Here, we describe an approach to provider screening for a Veterans Health Administration (VHA)-funded study focused on implementing risk-aligned surveillance for bladder cancer patients.MethodsOur goal was to identify providers at 6 pre-specified facilities who performed ≥10 surveillance cystoscopy procedures among bladder cancer patients in the 12 months prior to recruitment start on January 16, 2020, and who were currently practicing at 1 of 6 pre-specified facilities. Using VHA EHR data (using CPT, ICD10 procedure, and ICD10 diagnosis codes), we identified cystoscopy procedures performed after an initial bladder cancer diagnosis (i.e., surveillance procedures). Procedures were linked to VHA staff data to determine the provider of record, the number of cystoscopies they performed, and their current location of practice. To validate this approach, we performed a chart review of 105 procedures performed by a random sample of identified providers. The proportion of correctly identified procedures was calculated (Positive Predictive Value (PPV)), along with binomial 95% confidence intervals (CI).FindingsWe identified 1,917,856 cystoscopies performed on 703,324 patients from October 1, 1999—January 16, 2020, across the nationwide VHA. Of those procedures, 40% were done on patients who had a prior record of bladder cancer and were completed by 15,065 distinct providers. Of those, 61 performed ≥ 10 procedures and were currently practicing at 1 of the 6 facilities of interest in the 1 year prior to study recruitment. The random chart review of 7 providers found 101 of 105 procedures (PPV: 96%; 95% CI: 91% to 99%) were surveillance procedures and were performed by the selected provider on the recorded date.ImplicationsThese results show that EHR data can be used for accurate identification of healthcare providers as research participants when inclusion criteria consist of both patient- (temporal relationship between diagnosis and procedure) and provider-level (frequency of procedure and location of current practice) factors. As administrative codes and provider identifiers are collected in most, if not all, EHRs for billing purposes this approach can be translated from provider recruitment in VHA to other healthcare systems. Implementation studies should consider this method of screening providers.

Objective: To evaluate the continuity and completeness of electronic health record (EHR) data, and the concordance of select clinical outcomes and baseline comorbidities between EHR and linked claims data, from three healthcare delivery systems in Taiwan. Methods: We identified oral hypoglycemic agent (OHA) users from the Integrated Medical Database of National Taiwan University Hospital (NTUH-iMD), which was linked to the National Health Insurance Research Database (NHIRD), from June 2011 to December 2016. A secondary evaluation involved two additional EHR databases. We created consecutive 90-day periods before and after the first recorded OHA prescription and defined patients as having continuous EHR data if there was at least one encounter or prescription in a 90-day interval. EHR data completeness was measured by dividing the number of encounters in the NTUH-iMD by the number of encounters in the NHIRD. We assessed the concordance between EHR and claims data on three clinical outcomes (cardiovascular events, nephropathy-related events, and heart failure admission). We used individual comorbidities that comprised the Charlson comorbidity index to examine the concordance of select baseline comorbidities between EHRs and claims. Results: We identified 39,268 OHA users in the NTUH-iMD. Thirty-one percent (n = 12,296) of these users contributed to the analysis that examined data continuity during the 6-month baseline and 24-month follow-up period; 31% (n = 3,845) of the 12,296 users had continuous data during this 30-month period and EHR data completeness was 52%. The concordance of major cardiovascular events, nephropathy-related events, and heart failure admission was moderate, with the NTU-iMD capturing 49–55% of the outcome events recorded in the NHIRD. The concordance of comorbidities was considerably different between the NTUH-iMD and NHIRD, with an absolute standardized difference >0.1 for most comorbidities examined. Across the three EHR databases studied, 29–55% of the OHA users had continuous records during the 6-month baseline and 24-month follow-up period. Conclusion: EHR data continuity and data completeness may be suboptimal. A thorough evaluation of data continuity and completeness is recommended before conducting clinical and translational research using EHR data in Taiwan.

Electronic health record (EHR) data are widely used in public health research, including in HIV-related studies, but are limited by potential bias due to incomplete and inaccurate information, lack of generalizability, and lack of representativeness. This study explores how workflow processes among HIV health care providers (HCPs), data scientists, and state health department professionals may potentially introduce or minimize bias within EHR data. One focus group with 3 health department professionals working in HIV surveillance and 16 in-depth interviews (ie, 5 people with HIV, 5 HCPs, 5 data scientists, and 1 health department professional providing retention-in-care services) were conducted with participants purposively sampled in South Carolina from August 2023 to April 2024. All interviews were transcribed verbatim and analyzed using a constructivist grounded theory approach, where transcripts were first coded and then focused, axial, and theoretically coded. The EHR data lifecycle originates with people with HIV and HCPs in the clinical setting. Data scientists then curate EHR data and health department professionals manage and use the data for surveillance and policy decision-making. Throughout this lifecycle, the three primary stakeholders (ie, HCPs, data scientists, and health department professionals) identified challenges with EHR processes and provided their recommendations and accommodations in addressing the related challenges. HCPs reported the influence of socio-structural biases on their inquiry, interpretation, and documentation of social determinants of health (SDOH) information of people living with HIV, the influence of which is proposed to be mitigated through people living with HIV access to their EHRs. Data scientists identified limited data availability and representativeness as biasing the data they manage. Health department professionals face challenges with delayed and incomplete data, which may be addressed statistically but require consideration of the data's limitations. Overall, bias within the EHR data lifecycle persists because workflows are not intentionally structured to minimize bias and there is a diffusion of responsibility for data quality between the various stakeholders. From the perspective of various stakeholders, this study describes the EHR data lifecycle and its associated challenges as well as stakeholders' accommodations and recommendations for mitigating and eliminating bias in EHR data. Based upon these findings, studies reliant on EHR data should adequately consider its challenges and limitations. Throughout the EHR data lifecycle, bias could be reduced through an inclusive, supportive health care environment, people living with HIV verification of SDOH information, the customization of data collection systems, and EHR data inspection for completeness, accuracy, and timeliness. Future research is needed to further identify instances where bias is introduced and how it can best be mitigated and eliminated across the EHR data lifecycle. Systematic changes are necessary to reduce instances of bias between data workflows and stakeholders.

We assessed the suitability of pooled electronic health record (EHR) data from clinical research networks (CRNs) of the patient-centered outcomes research network to conduct studies of the association between tumor necrosis factor inhibitors (TNFi) and infections. EHR data from patients with one of seven autoimmune diseases were obtained from three CRNs and pooled. Person-level linkage of CRN data and Centers for Medicare and Medicaid Services (CMS) fee-for-service claims data was performed where possible. Using filled prescriptions from CMS claims data as the gold standard, we assessed the misclassification of EHR-based new (incident) user definitions. Among new users of TNFi, we assessed subsequent rates of hospitalized infection in EHR and CMS data. The study included 45 483 new users of TNFi, of whom 1416 were successfully linked to their CMS claims. Overall, 44% of new EHR TNFi prescriptions were not associated with medication claims. Our most specific new user definition had a misclassification rate of 3.5%-16.4% for prevalent use, depending on the medication. Greater than 80% of CRN prescriptions had either zero refills or missing refill data. Compared to using EHR data alone, there was a 2- to 8-fold increase in hospitalized infection rates when CMS claims data were added to the analysis. EHR data substantially misclassified TNFi exposure and underestimated the incidence of hospitalized infections compared to claims data. EHR-based new user definitions were reasonably accurate. Overall, using CRN data for pharmacoepidemiology studies is challenging, especially for biologics, and would benefit from supplementation by other sources.

Established classifications exist to confirm Sjögren’s Disease (SD) (previously referred as Sjögren’s Syndrome) and recruit patients for research. However, no established classification exists for diagnosis in clinical settings causing delayed diagnosis. SD patients experience a huge dental disease burden impairing their quality of life. This study established criteria to characterize Indiana University School of Dentistry (IUSD) patients’ SD based on symptoms and signs in the electronic health record (EHR) data available through the state-wide Indiana health information exchange (IHIE). Association between SD diagnosis, and comorbidities including other autoimmune conditions, and documentation of SD diagnosis in electronic dental record (EDR) were also determined. The IUSD patients’ EDR were linked with their EHR data in the IHIE and queried for SD diagnostic ICD9/10 codes. The resulting cohorts’ EHR clinical findings were characterized and classified using diagnostic criteria based on clinical experts’ recommendations. Descriptive statistics were performed, and Chi-square tests determined the association between the different SD presentations and comorbidities including other autoimmune conditions. Eighty-three percent of IUSD patients had an EHR of which 377 patients had a SD diagnosis. They were characterized as positive (24%), uncertain (20%) and negative (56%) based on EHR clinical findings. Dry eyes and mouth were reported for 51% and positive Anti-Ro/SSA antibodies and anti-nuclear antibody (ANA) for 17% of this study cohort. One comorbidity was present in 98% and other autoimmune condition/s were present in 53% respectively. Significant differences were observed between the three SD clinical characteristics/classifications and certain medical and autoimmune conditions (p<0.05). Sixty-nine percent of patients’ EDR did not mention SD, highlighting the huge gap in reporting SD during dental care. This study of SD patients diagnosed in community practices characterized three different SD clinical presentations, which can be used to generate SD study cohorts for longitudinal studies using EHR data. The results emphasize the heterogenous SD clinical presentations and the need for further research to diagnose SD early in community practice settings where most people seek care.

IntroductionAccurate identification of Aged Care residents accessing hospital services is challenging due to multiple hospital Electronic Health Record (EHR) systems and entry points resulting in inconsistent addresses and coding. Improving accuracy and completeness of addresses is crucial for cohort identification and data linkage. ObjectiveDevelop methods to validate cross-sectoral data linkages to improve identification of aged care residents within EHR systems. ApproachAmong individuals aged ≥65 years (or ≥50 identifying as Aboriginal or Torres Strait Islander) between January 2013 to December 2023 within our Data Platform, we first used data items mandated by the Department of Health to identify aged care residency. Data quality was enhanced using a residential address index derived from Australian Geocoded National Address File (G-NAF). We evaluated sensitivity, specificity, and Positive Predictive Values (PPVs) of recorded addresses against the Australian Institute of Health and Welfare (AIHW) aged care services list. ResultsAmong 409,677 addresses extracted, 55.4% (raw) had sufficient data for validation. This increased to 89.9% (enhanced) with G-NAF. Among 79,028 admitted and 88,876 emergency patients, 22.7% and 18.4% respectively were aged care residents. We compared performance of raw vs enhanced comparisons were: sensitivity (admission: 34.1% vs 73.8%), (emergency: 42.6% vs 66.2%); specificity (admission: 96.2% vs 90.9%), emergency (96.8% vs 95.2%); Positive Predictive Value (admission: 61.3% vs 58.8%), (emergency: 64.2% vs 65.3%). Conclusions and ImplicationsOur approach significantly increased sensitivity without sacrificing specificity, enhancing Aged Care resident identification in EHR data. It also improved linkage with external datasets and Statistical Area Level 1 (SA1) mapping.

Multicenter research of geriatric emergency department (GED) care remains limited. Our objectives were to: 1. Prospectively collect data prioritized by the Geriatric Emergency care Applied Research (GEAR) network, a transdisciplinary taskforce for GED care, and create a multicenter GED research repository of prospective and electronic health record (EHR) data, 2. Assess concordance between prospective and EHR data. The GEAR Standardization Study (GEARSS) is a multicenter, prospective study of older emergency department (ED) patients (65+) focusing on the 4Ms of age-friendly care (mobility, medication safety, mentation, what matters) and elder mistreatment. Demographic and clinical data were collected via interviews by trained research assistants (RA) on Days 0, 4, 30, and 90 and linked to EHR. Prevalence of chronic comorbidities and incident delirium were measured and reported using descriptive statistics. Prospective and EHR data concordance was assessed with Cohen's Kappa. 999 participants were recruited from 5 EDs (3/25/2021-6/30/2022) across 3 institutions: Grady Health System, Northwestern Memorial Hospital, and Yale New Haven Health. The cohort was 57.0% female, 55.2% White, 39.1% Black, and 3.4% Hispanic, and the mean age was 75.1 years. For rheumatologic disease, peptic ulcer disease, diabetes, renal disease, and cancer, prevalence differed between prospective and EHR data by > 10%. About two-thirds of participants were at risk for falls. Concordance between prospective and EHR data was good for ethnicity (K = 0.73); excellent for sex (K = 1.00), age (K = 1.00), and race (K = 0.98); fair for disposition (K = 0.53); slight for ED observation status (K = 0.33) and dementia diagnosis (K = 0.24); poor for delirium presence (K = 0.07). In GEARSS, demographic variables aligned strongly between prospective and EHR data, while diagnosis, disposition, and mentation factors did not. This multicenter data source provides preliminary findings for common geriatric syndromes and conditions. Choice of measures using these data should be driven by GED research questions.

Despite the extensive use of real-world data for retrospective, observational clinical research, our understanding of how real-world data might increase the efficiency of data collection in patient-level randomized clinical trials is largely unknown. The structure of real-world data is inherently heterogeneous, with each source electronic health record and claims database different from the next. Their fitness-for-use as data sources for multisite trials in the United States has not been established. For a subset of participants in the HARMONY Outcomes Trial, we obtained electronic health record data from recruiting sites or Medicare claims data from the Centers for Medicare & Medicaid Services. For baseline characteristics and follow-up events, we assessed the level of agreement between these real-world data and data documented in the trial database. Real-world data-derived demographic information tended to agree with trial-reported demographic information, although real-world data were less accurate in identifying medical history. The ability of real-world data to identify baseline medication usage differed by real-world data source, with claims data demonstrating substantially better performance than electronic health record data. The limited number of lab results in the collected electronic health record data matched closely with values in the trial database. There were not enough follow-up events in the ancillary study population to draw meaningful conclusions about the performance of real-world data for identification of events. Based on the conduct of this ancillary study, the challenges and opportunities of using real-world data within clinical trials are discussed. Based on a subset of participants from the HARMONY Outcomes Trial, our results suggest that electronic health record or claims data, as currently available, are unlikely to be a complete substitute for trial data collection of medical history or baseline lab results, but that Medicare claims were able to identify most medications. The limited size of the study population prevents us from drawing strong conclusions based on these results, and other studies are clearly needed to confirm or refute these findings.

ObjectiveThe ability to reproduce the work of others is an essential part of the scientific disciplines. Replicating observational studies using electronic health record (EHR) data can be challenging due to complexities in data access, variations in EHR systems across institutions, and the potential for unaccounted confounding variables. Our aim is to identify the barriers to methods reproducibility for replication studies using EHR data.MethodsWe replicated a study that examined the risk of hospitalisation following a positive COVID-19 test in individuals with diabetes. Using EHR data from the NHS England’s Secure Data Environment (SDE) covering the whole of England, UK (population 57m), we sought to replicate findings from the original study, which used data from Greater Manchester (a large urban region in the UK, population 2.9m). Both analyses were conducted in Trusted Research Environments (TREs) or SDEs, containing linked primary and secondary care data, however methods reproducibility was not straightforward. Differences between the environments that contributed to the difficulties were documented, categorized into themes, and converted into a list of recommendations for TRE/SDEs.ResultsSmall differences between the environments and the data sources led to several challenges in methods reproducibility. Our recommendations of TRE/SDEs should facilitate future replication studies. The recommendations include: a need for improved machine-readable metadata for EHR data; standardization of governance processes to facilitate federated analysis; mandating of code sharing; and for environments to have a support structure for data engineers and analysts. We also propose a new theme for research, “data reproducibility”, as the ability to prepare, extract and clean data from a different database for a replication study.ConclusionEven with perfect code sharing, data reproducibility remains a challenge. Our recommendations have the potential to reduce the barriers to replication studies and therefore enhance the potential of observational studies using EHR data.

A Genotype Validated Bimodal Method for the Large-Scale Identification and Phenotyping of Persons with Sickle Cell Disease Using Electronic Health Record Data

Do recent changes in European Society of Human Reproduction and Embryology (ESHRE) clinical guidelines result in more comprehensive diagnosis of women with endometriosis? The latest shift in clinical guidelines results in diagnosis of more women with endometriosis but current ESHRE diagnostic criteria do not capture a sizable percentage of women with the disease. Historically, laparoscopy was the gold standard for diagnosing endometriosis, a complex gynecological condition marked by a heterogeneous set of symptoms that vary widely among women. More recently, changes in clinical guidelines have shifted to incorporate imaging-based approaches such as transvaginal sonography and magnetic resonance imaging. Retrospective, observational cohort study of women aged 15-49 years diagnosed with endometriosis in the United States (US) between January 1, 2013, and December 31, 2023. Data sources include US insurance claims data from the Merative™ MarketScan® Commercial Database (CCAE), Merative™ MarketScan® Multi-State Medicaid Database (MDCD), Optum® de-identified Electronic Health Record dataset (Optum® EHR), and electronic health record (EHR) data from a large academic medical center in New York City (CUIMC EHR). To examine the potential impact of expanding clinical criteria for the diagnosis of endometriosis, we validated and compared five cohort definitions based on different sets of diagnostic guidelines involving combinations of surgical confirmation, diagnostic imaging, guideline-recognized symptoms, and other symptoms commonly reported among women with endometriosis. We performed pairwise comparisons between cohorts and applied Bonferroni corrections to account for multiple comparisons. We identified 491,048 women with a diagnosis of endometriosis across the CCAE, MDCD, Optum EHR, and CUIMC EHR datasets. Each cohort definition demonstrated strong positive predictive value (0.84-0.96), yet only 15-20% of cases were identified by all 5 cohort definitions. Women diagnosed with endometriosis based on imaging and symptoms were three years younger, on average, than women with a diagnosis based on surgical confirmation (mean age = 35 years [SD = 9] vs 38 years [SD = 8]; p<0.001). Women in cohorts based only on symptoms were two years younger than those based on surgery (36 years [SD = 8] vs 38 years [SD = 8]; p<0.001). More than one-fourth of cases presented with endometriosis-related symptoms but lacked surgical or imaging-related documentation required by ESHRE guideline criteria. Pain was reported among nearly all women with endometriosis. Abdominal pain and pain in the pelvis were the most prevalent (ranging from 69% to 90% of women in each cohort). Among approximately 2-5% of all endometriosis cases (14,795 total), women presented with pelvic and/or abdominal pain but none of the other symptoms noted in clinical guidelines. Our study has potential biases associated with documentation practices and secondary data use of insurance claims and EHR data. Further, the phenotyping algorithms used rely on clinical codes that do not necessarily capture all ESHRE diagnostic criteria for endometriosis and may not be generalizable to women with atypical presentation of endometriosis. High positive predictive value among all five cohort definitions despite poor overlap among cases identified illustrates both the heterogeneous presentation of the disease and importance of expanding diagnostic criteria. For example, cohorts derived from updated guidelines identified younger patients at time of diagnosis. Women diagnosed based on imaging had higher rates of emergency room visits while patients diagnosed via laparoscopy had a larger number of hospitalizations. The substantial number of cases with pelvic and/or abdominal pain but none of the other symptoms noted in clinical guidelines underscores the continued need for improved access to timely and appropriate care, particularly among those with non-classical symptoms, different care-seeking patterns, or lack of available surgical intervention.

Electronic health record (EHR) data are increasingly used to develop prediction models that guide clinical decision making at the point of care. These include algorithms that use high-frequency data, like in sepsis prediction, as well as simpler equations, such as the Pooled Cohort Equations for cardiovascular outcome prediction. Although EHR data used in prediction models are often highly granular and more current than other data, there is systematic and nonsystematic missingness in EHR data as there is with most data. Despite growing use for clinical decisions, algorithms implemented in EHRs are mostly unregulated and are often opaque to the user. Guidelines about the development, validation, implementation, and reporting on clinical prediction models are sparse in their recommendations regarding missing data. This article characterizes missingness in EHR data, summarizes methods for attending to missing data when developing prediction models, makes recommendations about validation and implementation of models in practice when data are missing, and identifies research needs in this field.

Predicting need for advanced illness or palliative care in a primary care population using electronic health record data.

Although hypothesized to be the root cause of the pulse oximetry disparities, skin tone and its use for improving medical therapies have yet to be extensively studied. Studies previously used self-reported race as a proxy variable for skin tone. However, this approach cannot account for skin tone variability within race groups and also risks the potential to be confounded by other non-biological factors when modeling data. Therefore, to better evaluate health disparities associated with pulse oximetry, this study aimed to create a unique baseline dataset that included skin tone and electronic health record (EHR) data. Patients admitted to Duke University Hospital were eligible if they had at least one pulse oximetry value recorded within 5 minutes before an arterial blood gas (ABG) value. We collected skin tone data at 16 different body locations using multiple devices, including administered visual scales, colorimetric, spectrophotometric, and photography via mobile phone cameras. All patients' data were linked in Duke's Protected Analytics Computational Environment (PACE), converted into a common data model, and then de-identified before publication in PhysioNet. Skin tone data were collected from 128 patients. We assessed 167 features per skin location on each patient. We also collected over 2000 images from mobile phones measured in the same controlled environment. Skin tone data are linked with patients' EHR data, such as laboratory data, vital sign recordings, and demographic information. Measuring different aspects of skin tone for each of the sixteen body locations and linking them with patients' EHR data could assist in the development of a more equitable AI model to combat disparities in healthcare associated with skin tone. A common data model format enables easy data federation with similar data from other sources, facilitating multicenter research on skin tone in healthcare. A prospectively collected EHR-linked skin tone measurements database in a common data model with emphasis on pulse oximetry disparities.

This December 11, 2014 think tank discussed issues around the methodologies for optimizing the robustness and use of Electronic Health Records (EHR) data for a variety of clinical research purposes. Experts in both accessing EHR data and study design and analysis methodology using EHR data for research highlighted the challenges, solutions, and needs in this area. Given the potential broad scope of this topic, participants were asked to focus primarily on issues related to the use of EHR on the back end (i.e. the imperfect data as currently collected), rather than strategies to improve the quality of EHR data collected on the 'front end' (e.g., data entered by clinicians).