Formative assessment and design of a complex clinical decision support tool for pulmonary embolism

Clinical Decision Support Tools

554 Background: Primary care providers (PCPs) routinely address health screening during clinical encounters and are ideally positioned to promote awareness among patients who would benefit from lung cancer screening (LCS). Integrating clinical decision support (CDS) tools for LCS into the electronic health record (EHR) within primary care settings can potentially serve as a strategy for the implementation of LCS. This study aimed to (1) identify PCPs’ recommendations for LCS CDS tools and (2) develop LCS CDS tools and test their usability in primary care settings. Methods: User-centered design principles were used to gather information from 23 PCPs about their recommendations for LCS CDS tools. Qualitative research methods, including individual interviews and group meetings, were used to collect data on the desired components of LCS CDS tools that would be useful in primary care. Braun and Clarke’s thematic analysis was used to analyze the qualitative data. Member checking was conducted to validate the accuracy of the recommendations. A set of EHR-based CDS tools was developed based on the recommendations. The usability of the CDS tools was tested using a think-aloud methodology with five PCPs and five patients. Iterative changes were made until no further feedback was provided by the PCPs and patients who reviewed the CDS tools. Results: Four components for LCS CDS tools were identified as important: (1) streamline the assessment process to increase the accuracy of tobacco use data collection, (2) provide shared decision making educational resources for patients before the visit and integrate shared decision making tools into the best practice alerts for use by PCPs, (3) provide non-interruptive best practice advisory alerts that were easy to see within the EHR, and (4) streamline the process for ordering CT scans and enhance communication across departments about results and future testing. The CDS tools that were developed included: (1) a patient-reported tobacco use survey, (2) order set for tobacco treatment, (3) best practice alerts to notify PCPs about LCS eligibility based on the accuracy of information available from the tobacco use survey and the EHR, (4) shared decision-making educational materials, and (5) LCS CT order sets. PCPs provided positive feedback about the CDS tools except for the order set for tobacco treatment. PCPs indicated that the order set for tobacco treatment was difficult to use since it disrupted their workflow. Patients provided positive feedback about the educational resources and stressed the importance of having tobacco use questions framed in a way that minimized stigma associated with tobacco use. Conclusions: PCPs recommended four components for CDS tools to facilitate LCS, with a focus on strategies to streamline clinical workflows. Future testing of the CDS tools is needed to determine whether they facilitate implementation of LCS.

Through our work, we have demonstrated how clinical decision support (CDS) tools integrated into the electronic health record (EHR) assist providers in adopting evidence-based practices. This requires confronting technical challenges that result from relying on the EHR as the foundation for tool development; for example, the individual CDS tools need to be built independently for each different EHR. The objective of our research was to build and implement an EHR-agnostic platform for integrating CDS tools, which would remove the technical constraints inherent in relying on the EHR as the foundation and enable a single set of CDS tools that can work with any EHR. We developed EvidencePoint, a novel, cloud-based, EHR-agnostic CDS platform, and we will describe the development of EvidencePoint and the deployment of its initial CDS tools, which include EHR-integrated applications for clinical use cases such as prediction of hospitalization survival for patients with COVID-19, venous thromboembolism prophylaxis, and pulmonary embolism diagnosis. The results below highlight the adoption of the CDS tools, the International Medical Prevention Registry on Venous Thromboembolism-D-Dimer, the Wells' criteria, and the Northwell COVID-19 Survival (NOCOS), following development, usability testing, and implementation. The International Medical Prevention Registry on Venous Thromboembolism-D-Dimer CDS was used in 5249 patients at the 2 clinical intervention sites. The intervention group tool adoption was 77.8% (4083/5249 possible uses). For the NOCOS tool, which was designed to assist with triaging patients with COVID-19 for hospital admission in the event of constrained hospital resources, the worst-case resourcing scenario never materialized and triaging was never required. As a result, the NOCOS tool was not frequently used, though the EvidencePoint platform's flexibility and customizability enabled the tool to be developed and deployed rapidly under the emergency conditions of the pandemic. Adoption rates for the Wells' criteria tool will be reported in a future publication. The EvidencePoint system successfully demonstrated that a flexible, user-friendly platform for hosting CDS tools outside of a specific EHR is feasible. The forthcoming results of our outcomes analyses will demonstrate the adoption rate of EvidencePoint tools as well as the impact of behavioral economics "nudges" on the adoption rate. Due to the EHR-agnostic nature of EvidencePoint, the development process for additional forms of CDS will be simpler than traditional and cumbersome IT integration approaches and will benefit from the capabilities provided by the core system of EvidencePoint.

To develop and implement a clinical decision support (CDS) tool to improve antibiotic prescribing in neonatal intensive care units (NICUs) and to evaluate user acceptance of the CDS tool. Following sociotechnical analysis of NICU prescribing processes, a CDS tool for empiric and targeted antimicrobial therapy for healthcare-associated infections (HAIs) was developed and incorporated into a commercial electronic health record (EHR) in two NICUs. User logs were reviewed and NICU prescribers were surveyed for their perceptions of the CDS tool. The CDS tool aggregated selected laboratory results, including culture results, to make treatment recommendations for common clinical scenarios. From July 2010 to May 2012, 1,303 CDS activations for 452 patients occurred representing 22% of patients prescribed antibiotics during this period. While NICU clinicians viewed two culture results per tool activation, prescribing recommendations were viewed during only 15% of activations. Most (63%) survey respondents were aware of the CDS tool, but fewer (37%) used it during their most recent NICU rotation. Respondents considered the most useful features to be summarized culture results (43%) and antibiotic recommendations (48%). During the study period, the CDS tool functionality was hindered by EHR upgrades, implementation of a new laboratory information system, and changes to antimicrobial testing methodologies. Loss of functionality may have reduced viewing antibiotic recommendations. In contrast, viewing culture results was frequently performed, likely because this feature was perceived as useful and functionality was preserved. To improve CDS tool visibility and usefulness, we recommend early user and information technology team involvement which would facilitate use and mitigate implementation challenges.

Use of a Commercially Available Clinical Decision Support Tool to Expedite Prior Authorization in Partnership With a Private Payer.

Integrating usability testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical decision support

Provider Attitudes Toward the Implementation of Clinical Decision Support Tools in Dental Practice

BackgroundRetinal toxicity is a rare adverse event related to the use of hydroxychloroquine (HCQ). To address this, in 2016, the American Academy of Ophthalmology (AAO) issued guidelines recommending that HCQ not exceed 5 mg/kg/day. We analyzed HCQ prescribing habits at our institution, compared to these guidelines, and used surveys to determine the opinions on these guidelines. We then introduced, in a prospective and non-controlled study, a clinical decision support (CDS) tool into the electronic medical record (EMR) to study how this intervention might affect adherence with or opinions on these guidelines.MethodsData were collected pre-intervention (June 2017–January 2019) and post-intervention (March 2019–April 2020). In January 2019 we released our CDS tool. Results were analyzed using descriptive statistics for demographic data and Fisher’s exact tests for comparisons of proportions between groups.ResultsPre-intervention, we reviewed 1128 rheumatology charts and 282 dermatology charts. 31.0 and 39.7% respectively (32.8% combined) were prescribed HCQ > 5 .0 mg/kg/day. Post-intervention, we reviewed 1161 rheumatology charts and 110 dermatology charts. 23.0 and 25.5% respectively (23.2% combined) were prescribed HCQ > 5.0 mg/kg/day. Post-intervention, 9.6% fewer patients were prescribed HCQ > 5 mg/kg/day (P < .001). Pre-intervention, we compiled 18 rheumatology surveys and 12 dermatology surveys. Post-intervention, we compiled 16 rheumatology surveys and 12 dermatology surveys. Post-intervention, fewer rheumatologists incorrectly described the AAO weight-based guidelines. Combined, there was an overall reduction but not of statistical significance (P = .47). The majority of providers surveyed believed that the CDS tool was useful (72.2%).ConclusionsAt our academic institution, there remains unfamiliarity with and hesitation to comply with the 2016 AAO guidelines. Prescribed doses often exceed what is recommended in these guidelines. A CDS tool can improve adherence with these guidelines and might improve providers’ familiarity with these guidelines.

BACKGROUND: Atrial fibrillation (AF) is a common arrhythmia that is often first diagnosed in the Emergency Department (ED). AF is associated with an increased risk of stroke that can be reduced by 64% if an oral anticoagulant (OAC) is prescribed appropriately. RESEARCH QUESTION: Will education and implementation of a clinical decision support (CDS) tool in the ED increase OAC prescription for appropriate candidates diagnosed with new-onset or paroxysmal AF (pAF)? METHODS: This is the first phase of a multi-stage multi-center stepped-wedge cluster-randomized clinical trial implementing a CDS tool into the electronic health record (EHR). Step one implements an external browser link to the CDS tool in the EHR with targeted provider education, whereas steps 2 and 3 will motivate provider engagement with triggers. Data were collected from an urban academic tertiary care center and a community hospital. Patients aged greater than 17 years were selected if they had a primary diagnosis of AF/pAF during ED visits from Jan 2020-Apr 2023. The academic and community sites transitioned from the retrospective phase to Step 1 in Jan 2022 and Apr 2022, respectively. Appropriate OAC prescription was determined based on elevated stroke risk and low-moderate bleed risk. A logistic GEE model was used to study the effect of the proposed intervention on OAC prescription. ED providers were treated as a random effect to capture clustering. The model includes the CDS intervention, site, and patient characteristics sex, race, ethnicity, and age. RESULTS: Of 2031 patients, 312 met inclusion criteria and 150 were selected as appropriate OAC candidates, with 72 (52% male, median age 74) in the retrospective step and 78 (52% male, median age 76) in Step 1. Below are odds ratio estimates with corresponding 95% CI’s: CDS intervention 3.12 [1.57, 6.21] and sex female vs male 2.04 [1.04, 4.00]. Non-significant variables site, race, ethnicity, and age were included as adjusting covariates. CONCLUSIONS: Limited CDS tool education and integration is associated with improved OAC prescribing for appropriate AF/pAF patients. Future studies should assess the impact of additional CDS integration on OAC prescribing and patient outcomes.

The effectiveness of an electronic clinical-decision support tool embedded into the patient medical record at the point of care to guide Mohs surgery utilization: A cohort study

Digital clinical decision support (CDS) tools are of growing importance in supporting healthcare professionals in understanding complex clinical problems and arriving at decisions that improve patient outcomes. CDS tools are also increasingly used to improve antimicrobial stewardship (AMS) practices in healthcare settings. However, far fewer CDS tools are available in lowerand middle-income countries (LMICs) and in animal health settings, where their use in improving diagnostic and treatment decision-making is likely to have the greatest impact. The aim of this study was to evaluate digital CDS tools designed as a direct aid to support diagnosis and/or treatment decisionmaking, by reviewing their scope, functions, methodologies, and quality. Recommendations for the development of veterinary CDS tools in LMICs are then provided. The review considered studies and reports published between January 2017 and October 2023 in the English language in peer-reviewed and gray literature. A total of 41 studies and reports detailing CDS tools were included in the final review, with 35 CDS tools designed for human healthcare settings and six tools for animal healthcare settings. Of the tools reviewed, the majority were deployed in high-income countries (80.5%). Support for AMS programs was a feature in 12 (29.3%) of the tools, with 10 tools in human healthcare settings. The capabilities of the CDS tools varied when reviewed against the GUIDES checklist. We recommend a methodological approach for the development of veterinary CDS tools in LMICs predicated on securing sufficient and sustainable funding. Employing a multidisciplinary development team is an important first step. Developing standalone CDS tools using Bayesian algorithms based on local expert knowledge will provide users with rapid and reliable access to quality guidance on diagnoses and treatments. Such tools are likely to contribute to improved disease management on farms and reduce inappropriate antimicrobial use, thus supporting AMS practices in areas of high need.

The purpose of this study was to assess the effects of a clinical decision support (CDS) tool aimed at decreasing the prescribing of glyburide, a potentially inappropriate medication (PIM), in patients 65 years of age and older. Quasi-experimental, pre-post intervention study. Ambulatory care clinics of an academic medical center. The tool appeared to providers when entering new prescriptions or refills for glyburide. Glimepiride, which is a more appropriate sulfonylurea, was suggested as an alternative at order entry. The primary outcome was the prescribing of glyburide orders, measured as a percentage of the total oral diabetic medications ordered in patients 65 years of age and older, during the study period. The secondary outcome measured was the response to the CDS tool (accept versus reject). The CDS tool alerted providers 101 times during the 90-day postimplementation period. When the tool appeared, patients were transitioned off of glyburide 17.8% of the time. Subanalysis found that when physicians viewed the alert, patients were transitioned off of glyburide 46.2% of the time. As a percentage of the total number of oral diabetic medications, glyburide prescribing was significantly decreased from pre- to postimplementation study period (3.3% vs. 1.2%; P < 0.001). A CDS tool can be used in the ambulatory care setting to influence prescribing and provide a safer alternative medication. Additional information is needed to test the use of a CDS tool in conjunction with education to ensure providers are comfortable with and understand implications of a CDS tool.

BackgroundSyncope evaluation and management is associated with testing overuse and unnecessary hospitalizations. The 2017 American College of Cardiology/American Heart Association (ACC/AHA) Syncope Guideline aims to standardize clinical practice and reduce unnecessary services. The use of clinical decision support (CDS) tools offers the potential to successfully implement evidence-based clinical guidelines. However, CDS tools that provide an evidence-based differential diagnosis (DDx) of syncope at the point of care are currently lacking.ObjectiveWith input from diverse health systems, we developed and demonstrated the viability of a mobile app, the Multilevel Implementation Strategy for Syncope optImal care thrOugh eNgagement (MISSION) Syncope, as a CDS tool for syncope diagnosis and prognosis.MethodsDevelopment of the app had three main goals: (1) reliable generation of an accurate DDx, (2) incorporation of an evidence-based clinical risk tool for prognosis, and (3) user-based design and technical development. To generate a DDx that incorporated assessment recommendations, we reviewed guidelines and the literature to determine clinical assessment questions (variables) and likelihood ratios (LHRs) for each variable in predicting etiology. The creation and validation of the app diagnosis occurred through an iterative clinician review and application to actual clinical cases. The review of available risk score calculators focused on identifying an easily applied and valid evidence-based clinical risk stratification tool. The review and decision-making factors included characteristics of the original study, clinical variables, and validation studies. App design and development relied on user-centered design principles. We used observations of the emergency department workflow, storyboard demonstration, multiple mock review sessions, and beta-testing to optimize functionality and usability.ResultsThe MISSION Syncope app is consistent with guideline recommendations on evidence-based practice (EBP), and its user interface (UI) reflects steps in a real-world patient evaluation: assessment, DDx, risk stratification, and recommendations. The app provides flexible clinical decision making, while emphasizing a care continuum; it generates recommendations for diagnosis and prognosis based on user input. The DDx in the app is deemed a pragmatic model that more closely aligns with real-world clinical practice and was validated using actual clinical cases. The beta-testing of the app demonstrated well-accepted functionality and usability of this syncope CDS tool.ConclusionsThe MISSION Syncope app development integrated the current literature and clinical expertise to provide an evidence-based DDx, a prognosis using a validated scoring system, and recommendations based on clinical guidelines. This app demonstrates the importance of using research literature in the development of a CDS tool and applying clinical experience to fill the gaps in available research. It is essential for a successful app to be deliberate in pursuing a practical clinical model instead of striving for a perfect mathematical model, given available published evidence. This hybrid methodology can be applied to similar CDS tool development.

Peanut allergy in children is a population health problem. Evidence suggests early peanut introduction (EPI) for infants can reduce the development of peanut allergy. Primary care settings have not widely adopted guidelines recommending EPI. Peanut allergy prevention depends on primary care providers incorporating EPI guidelines into well-child check (WCC) encounters. We aimed to improve guideline adherence in a primary care setting by implementing a bundle of clinical decision support (CDS) tools. Using quality improvement methodology, the team developed a standardized work protocol and CDS tools within an electronic medical record (EMR) at 4, 6, and 9-month WCC encounters. The team executed changes and modifications through plan-do-study-act cycles and analyzed results with statistical process control charts. We collected data from 445 WCC encounters from baseline through sustainability. EMR documentation of EPI guidance at 4, 6, and 9-month WCCs shifted from 13.9% to 83.5% over 12 months. Provider adoption of smart lists and templates increased from 2% to 73%, the distribution of home peanut introduction handouts increased from 5.2% to 54.1%, and caregiver-reported peanut ingestion increased from 0% to 34.6%. Diphtheria-tetanus-acellular pertussis vaccination rates remained at 100% for 6-month visits, and patient in-room time remained at 65 minutes. Quality improvement methodology improved documentation of EPI guidance and increased reported peanut ingestion at routine WCC encounters without impacting other measures. Broader use of bundled CDS tools and EMR standardization could further improve guideline adherence and increase early peanut introduction to prevent peanut allergy in infants.

Variance Reduction in Neurosurgical Practice: The Case for Analytics-Driven Decision Support in the Era of Big Data