To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage. Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity. There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients. Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race. Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Remote monitoring of women experiencing hypertensive disorders of pregnancy (HDP) can provide timely life-saving data, particularly if these data are integrated into existing patient and clinical workflows. This pilot intervention of a smartphone application (app) for postpartum monitoring of hypertensive disorders integrates patient-contributed data into electronic health records (EHRs) to support monitoring and clinical decision-making. Results from the evaluation of the pilot highlight the resources needed when implementing the app, challenges for integrating an app into the EHR, and the usability and utility of the HDP monitoring app for patient and clinician users. The implementation team’s key observations included the importance of a local clinical champion, more robust patient involvement and support for the remote patient monitoring program, an impetus for EHR developers to adopt data integration standards, and a need to expand the capabilities of the standards to support interventions using patient-contributed data.

ObjectivesTo define pregnancy episodes and estimate gestational age within electronic health record (EHR) data from the National COVID Cohort Collaborative (N3C).Materials and MethodsWe developed a comprehensive approach, named Hierarchy and rule-based pregnancy episode Inference integrated with Pregnancy Progression Signatures (HIPPS), and applied it to EHR data in the N3C (January 1, 2018–April 7, 2022). HIPPS combines: (1) an extension of a previously published pregnancy episode algorithm, (2) a novel algorithm to detect gestational age-specific signatures of a progressing pregnancy for further episode support, and (3) pregnancy start date inference. Clinicians performed validation of HIPPS on a subset of episodes. We then generated pregnancy cohorts based on gestational age precision and pregnancy outcomes for assessment of accuracy and comparison of COVID-19 and other characteristics.ResultsWe identified 628 165 pregnant persons with 816 471 pregnancy episodes, of which 52.3% were live births, 24.4% were other outcomes (stillbirth, ectopic pregnancy, abortions), and 23.3% had unknown outcomes. Clinician validation agreed 98.8% with HIPPS-identified episodes. We were able to estimate start dates within 1 week of precision for 475 433 (58.2%) episodes. 62 540 (7.7%) episodes had incident COVID-19 during pregnancy.DiscussionHIPPS provides measures of support for pregnancy-related variables such as gestational age and pregnancy outcomes based on N3C data. Gestational age precision allows researchers to find time to events with reasonable confidence.ConclusionWe have developed a novel and robust approach for inferring pregnancy episodes and gestational age that addresses data inconsistency and missingness in EHR data.

The aim of this study was to understand the influence of clinician encouragement and sociodemographic factors on whether patients access online electronic medical records (EMR). We analyzed 3279 responses from the Health Information National Trends Survey 5 cycle 4 survey, a cross-sectional, nationally representative survey administered by the National Cancer Institute. Frequencies and weighted proportions were calculated to compare clinical encouragement and access to their online EMR. Using multivariate logistic regression, we identified factors associated with online EMR use and clinician encouragement. In 2020, an estimated 42% of US adults accessed their online EMR and 51% were encouraged by clinicians to access their online EMR. In multivariate regression, respondents who accessed EMR were more likely to have received clinician encouragement (odds ratio [OR], 10.3; 95% confidence interval [CI], 7.7-14.0), college education or higher (OR, 1.9; 95% CI, 1.4-2.7), history of cancer (OR, 1.5; 95% CI, 1.0-2.3), and history of chronic disease (OR, 2.3; 95% CI, 1.7-3.2). Male and Hispanic respondents were less likely to have accessed EMR than female and non-Hispanic White respondents (OR, 0.6; 95% CI, 0.5-0.8, and OR, 0.5; 95% CI, 0.3-0.8, respectively). Respondents receiving encouragement from clinicians were more likely to be female (OR, 1.7; 95% CI, 1.3-2.3), have college education (OR, 1.5; 95% CI, 1.1-2.0), history of cancer (OR, 1.8; 95% CI, 1.3-2.5), and greater income levels (OR, 1.8-3.6). Clinician encouragement of patient EMR use is strongly associated with patients accessing EMR, and there are disparities in who receives clinician encouragement related to education, income, sex, and ethnicity. Clinicians have an important role to ensure that all patients benefit from online EMR use.

To test the association between the initial red blood cell distribution width (RDW) value in the emergency department (ED) and hospital admission and, among those admitted, in-hospital mortality. We perform a retrospective analysis of 210 930 adult ED visits with complete blood count results from March 2013 to February 2022. Primary outcomes were hospital admission and in-hospital mortality. Variables for each visit included demographics, comorbidities, vital signs, basic metabolic panel, complete blood count, and final diagnosis. The association of each outcome with the initial RDW value was calculated across 3 age groups (<45, 45-65, and >65) as well as across 374 diagnosis categories. Logistic regression (LR) and XGBoost models using all variables excluding final diagnoses were built to test whether RDW was a highly weighted and informative predictor for each outcome. Finally, simplified models using only age, sex, and vital signs were built to test whether RDW had additive predictive value. Compared to that of discharged visits (mean [SD]: 13.8 [2.03]), RDW was significantly elevated in visits that resulted in admission (15.1 [2.72]) and, among admissions, those resulting in intensive care unit stay (15.3 [2.88]) and/or death (16.8 [3.25]). This relationship held across age groups as well as across various diagnosis categories. An RDW >16 achieved 90% specificity for hospital admission, while an RDW >18.5 achieved 90% specificity for in-hospital mortality. LR achieved a test area under the curve (AUC) of 0.77 (95% confidence interval [CI] 0.77-0.78) for hospital admission and 0.85 (95% CI 0.81-0.88) for in-hospital mortality, while XGBoost achieved a test AUC of 0.90 (95% CI 0.89-0.90) for hospital admission and 0.96 (95% CI 0.94-0.97) for in-hospital mortality. RDW had high scaled weights and information gain for both outcomes and had additive value in simplified models predicting hospital admission. Elevated RDW, previously associated with mortality in myocardial infarction, pulmonary embolism, heart failure, sepsis, and COVID-19, is associated with hospital admission and in-hospital mortality across all-cause adult ED visits. Used alone, elevated RDW may be a specific, but not sensitive, test for both outcomes, with multivariate LR and XGBoost models showing significantly improved test characteristics. RDW, a component of the complete blood count panel routinely ordered as the initial workup for the undifferentiated patient, may be a generalizable biomarker for acuity in the ED.

To develop a standardizable, reproducible method for creating drug codelists that incorporates clinical expertise and is adaptable to other studies and databases. We developed methods to generate drug codelists and tested this using the Clinical Practice Research Datalink (CPRD) Aurum database, accounting for missing data in the database. We generated codelists for: (1) cardiovascular disease and (2) inhaled Chronic Obstructive Pulmonary Disease (COPD) therapies, applying them to a sample cohort of 335931 COPD patients. We compared searching all drug dictionary variables (A) against searching only (B) chemical or (C) ontological variables. In Search A, we identified 165150 patients prescribed cardiovascular drugs (49.2% of cohort), and 317963 prescribed COPD inhalers (94.7% of cohort). Evaluating output per search strategy, Search C missed numerous prescriptions, including vasodilator anti-hypertensives (A and B:19696 prescriptions; C:1145) and SAMA inhalers (A and B:35310; C:564). We recommend the full search (A) for comprehensiveness. There are special considerations when generating adaptable and generalizable drug codelists, including fluctuating status, cohort-specific drug indications, underlying hierarchical ontology, and statistical analyses. Methods must have end-to-end clinical input, and be standardizable, reproducible, and understandable to all researchers across data contexts.

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients. Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers. Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ). A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE. We identify challenges and offer lessons from our user-centered design process.

As the recognition of team-based care grows and pharmacists increase in patient care interventions, it is important that tools to track clinical services are easily accessible and well-integrated into workflow for all providers. We describe and discuss feasibility and implementation of data tools in an electronic health record to evaluate a pragmatic clinical pharmacy intervention focused on deprescribing in aged adults delivered at multiple clinical sites in a large academic health system. Of the data tools utilized, we were able to demonstrate clear documentation frequency of certain phrases during the intervention period for 574 patients receiving opioids and 537 patients receiving benzodiazepines. Although clinical decision support and documentation tools exist, they are underutilized or cumbersome to integrate into primary health care and strategies, such as employed, are a solution. This communication incorporates the importance of clinical pharmacy information systems in research design.

When correcting for the "class imbalance" problem in medical data, the effects of resampling applied on classifier algorithms remain unclear. We examined the effect on performance over several combinations of classifiers and resampling ratios. Multiple classification algorithms were trained on 7 resampled datasets: no correction, random undersampling, 4 ratios of Synthetic Minority Oversampling Technique (SMOTE), and random oversampling with the Adaptive Synthetic algorithm (ADASYN). Performance was evaluated in Area Under the Curve (AUC), precision, recall, Brier score, and calibration metrics. A case study on prediction modeling for 30-day unplanned readmissions in previously admitted Urology patients was presented. For most algorithms, using resampled data showed a significant increase in AUC and precision, ranging from 0.74 (CI: 0.69-0.79) to 0.93 (CI: 0.92-0.94), and 0.35 (CI: 0.12-0.58) to 0.86 (CI: 0.81-0.92) respectively. All classification algorithms showed significant increases in recall, and significant decreases in Brier score with distorted calibration overestimating positives. Imbalance correction resulted in an overall improved performance, yet poorly calibrated models. There can still be clinical utility due to a strong discriminating performance, specifically when predicting only low and high risk cases is clinically more relevant. Resampling data resulted in increased performances in classification algorithms, yet produced an overestimation of positive predictions. Based on the findings from our case study, a thoughtful predefinition of the clinical prediction task may guide the use of resampling techniques in future studies aiming to improve clinical decision support tools.

ObjectiveModern healthcare data reflect massive multi-level and multi-scale information collected over many years. The majority of the existing phenotyping algorithms use case–control definitions of disease. This paper aims to study the time to disease onset and progression and identify the time-varying risk factors that drive them.Materials and MethodsWe developed an algorithmic approach to phenotyping the incidence of diseases by consolidating data sources from the UK Biobank (UKB), including primary care electronic health records (EHRs). We focused on defining events, event dates, and their censoring time, including relevant terms and existing phenotypes, excluding generic, rare, or semantically distant terms, forward-mapping terminology terms, and expert review. We applied our approach to phenotyping diabetes complications, including a composite cardiovascular disease (CVD) outcome, diabetic kidney disease (DKD), and diabetic retinopathy (DR), in the UKB study.ResultsWe identified 49 049 participants with diabetes. Among them, 1023 had type 1 diabetes (T1D), and 40 193 had type 2 diabetes (T2D). A total of 23 833 diabetes subjects had linked primary care records. There were 3237, 3113, and 4922 patients with CVD, DKD, and DR events, respectively. The risk prediction performance for each outcome was assessed, and our results are consistent with the prediction area under the ROC (receiver operating characteristic) curve (AUC) of standard risk prediction models using cohort studies.Discussion and ConclusionOur publicly available pipeline and platform enable streamlined curation of incidence events, identification of time-varying risk factors underlying disease progression, and the definition of a relevant cohort for time-to-event analyses. These important steps need to be considered simultaneously to study disease progression.