IntroductionMoving evidence-based practices into the hands of practitioners requires the synthesis and translation of research literature. However, the growing pace of scientific publications across disciplines makes it increasingly difficult to stay abreast of research literature. Natural language processing (NLP) methods are emerging as a valuable strategy for conducting content analyses of academic literature. We sought to apply NLP to identify publication trends in the journal Implementation Science, including key topic clusters and the distribution of topics over time. A parallel study objective was to demonstrate how NLP can be used in research synthesis.MethodsWe examined 1711 Implementation Science abstracts published from February 22, 2006, to October 1, 2020. We retrieved the study data using PubMed’s Application Programming Interface (API) to assemble a database. Following standard preprocessing steps, we use topic modeling with Latent Dirichlet allocation (LDA) to cluster the abstracts following a minimization algorithm.ResultsWe examined 30 topics and computed topic model statistics of quality. Analyses revealed that published articles largely reflect (i) characteristics of research, or (ii) domains of practice. Emergent topic clusters encompassed key terms both salient and common to implementation science. HIV and stroke represent the most commonly published clinical areas. Systematic reviews have grown in topic prominence and coherence, whereas articles pertaining to knowledge translation (KT) have dropped in prominence since 2013. Articles on HIV and implementation effectiveness have increased in topic exclusivity over time.DiscussionWe demonstrated how NLP can be used as a synthesis and translation method to identify trends and topics across a large number of (over 1700) articles. With applicability to a variety of research domains, NLP is a promising approach to accelerate the dissemination and uptake of research literature. For future research in implementation science, we encourage the inclusion of more equity-focused studies to expand the impact of implementation science on disadvantaged communities.

BackgroundCalls have been made for greater application of the decision sciences to investigate and improve use of research evidence in mental health policy and practice. This article proposes a novel method, “decision sampling,” to improve the study of decision-making and research evidence use in policy and programmatic innovation. An illustrative case study applies the decision sampling framework to investigate the decisions made by mid-level administrators when developing system-wide interventions to identify and treat the trauma of children entering foster care.MethodsDecision sampling grounds qualitative inquiry in decision analysis to elicit information about the decision-making process. Our case study engaged mid-level managers in public sector agencies (n = 32) from 12 states, anchoring responses on a recent index decision regarding universal trauma screening for children entering foster care. Qualitative semi-structured interviews inquired on questions aligned with key components of decision analysis, systematically collecting information on the index decisions, choices considered, information synthesized, expertise accessed, and ultimately the values expressed when selecting among available alternatives.ResultsFindings resulted in identification of a case-specific decision set, gaps in available evidence across the decision set, and an understanding of the values that guided decision-making. Specifically, respondents described 14 inter-related decision points summarized in five domains for adoption of universal trauma screening protocols, including (1) reach of the screening protocol, (2) content of the screening tool, (3) threshold for referral, (4) resources for screening startup and sustainment, and (5) system capacity to respond to identified needs. Respondents engaged a continuum of information that ranged from anecdote to research evidence, synthesizing multiple types of knowledge with their expertise. Policy, clinical, and delivery system experts were consulted to help address gaps in available information, prioritize specific information, and assess “fit to context.” The role of values was revealed as participants evaluated potential trade-offs and selected among policy alternatives.ConclusionsThe decision sampling framework is a novel methodological approach to investigate the decision-making process and ultimately aims to inform the development of future dissemination and implementation strategies by identifying the evidence gaps and values expressed by the decision-makers, themselves.

BackgroundOsteoarthritis is a leading cause of pain and disability worldwide. Despite research supporting best practice, evidence-based guidelines are often not followed. Little is known about the implementation of non-surgical models of care in routine primary care practice. From a knowledge mobilisation perspective, the aim of this study was to understand the uptake of a clinical innovation for osteoarthritis and explore the journey from a clinical trial to implementation.MethodsThis study used two methods: secondary analysis of focus groups undertaken with general practice staff from the Managing OSteoArthritis in ConsultationS research trial, which investigated the effectiveness of an enhanced osteoarthritis consultation, and interviews with stakeholders from an implementation project which started post-trial following demand from general practices. Data from three focus groups with 21 multi-disciplinary clinical professionals (5–8 participants per group), and 13 interviews with clinical and non-clinical stakeholders, were thematically analysed utilising the Integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework, in a theoretically informative approach. Public contributors were involved in topic guide design and interpretation of results.ResultsIn operationalising implementation of an innovation for osteoarthritis following a trial, the importance of a whole practice approach, including the opportunity for reflection and planning, were identified. The end of a clinical trial provided opportune timing for facilitating implementation planning. In the context of osteoarthritis in primary care, facilitation by an inter-disciplinary knowledge brokering service, nested within an academic institution, was instrumental in supporting ongoing implementation by providing facilitation, infrastructure and resource to support the workload burden. ‘Instinctive facilitation’ may involve individuals who do not adopt formal brokering roles or fully recognise their role in mobilising knowledge for implementation. Public contributors and lay communities were not only recipients of healthcare innovations but also potential powerful facilitators of implementation.ConclusionThis theoretically informed knowledge mobilisation study into the uptake of a clinical innovation for osteoarthritis in primary care has enabled further characterisation of the facilitation and recipient constructs of i-PARIHS by describing optimum timing for facilitation and roles and characteristics of facilitators.

ObjectiveTo assess the quality of recommendations from 161 clinical practice guidelines (CPGs) using AGREE-REX-D (Appraisal of Guidelines REsearch and Evaluation-Recommendations Excellence Draft).DesignCross-sectional studySettingInternational CPG community.ParticipantsThree hundred twenty-two international CPG developers, users, and researchers.InterventionParticipants were assigned to appraise one of 161 CPGs selected for the study using the AGREE-REX-D toolMain outcome measuresAGREE-REX-D scores of 161 CPGs (7-point scale, maximum 7).ResultsRecommendations from 161 CPGs were appraised by 322 participants using the AGREE-REX-D. CPGs were developed by 67 different organizations. The total overall average score of the CPG recommendations was 4.23 (standard deviation (SD) = 1.14). AGREE-REX-D items that scored the highest were (mean; SD): evidence (5.51; 1.14), clinical relevance (5.95; SD 0.8), and patients/population relevance (4.87; SD 1.33), while the lowest scores were observed for the policy values (3.44; SD 1.53), local applicability (3,56; SD 1.47), and resources, tools, and capacity (3.49; SD 1.44) items. CPGs developed by government-supported organizations and developed in the UK and Canada had significantly higher recommendation quality scores with the AGREE-REX-D tool (p < 0.05) than their comparators.ConclusionsWe found that there is significant room for improvement of some CPGs such as the considerations of patient/population values, policy values, local applicability and resources, tools, and capacity. These findings may be considered a baseline upon which to measure future improvements in the quality of CPGs.

BackgroundIn the last decade, there is an increasing focus on detecting and compiling lists of low-value nursing procedures. However, less is known about effective de-implementation strategies for these procedures. Therefore, the aim of this systematic review was to summarize the evidence of effective strategies to de-implement low-value nursing procedures.MethodsPubMed, Embase, Emcare, CINAHL, PsycINFO, Cochrane Central Register of Controlled Trials, Web of Science, and Google Scholar were searched till January 2020. Additionally, reference lists and citations of the included studies were searched. Studies were included that described de-implementation of low-value nursing procedures, i.e., procedures, test, or drug orders by nurses or nurse practitioners. PRISMA guideline was followed, and the ‘Cochrane Effective Practice and Organisation of Care’ (EPOC) taxonomy was used to categorize de-implementation strategies. A meta-analysis was performed for the volume of low-value nursing procedures in controlled studies, and Mantel–Haenszel risk ratios (95% CI) were calculated using a random effects model.ResultsTwenty-seven studies were included in this review. Studies used a (cluster) randomized design (n = 10), controlled before-after design (n = 5), and an uncontrolled before-after design (n = 12). Low-value nursing procedures performed by nurses and/or nurse specialists that were found in this study were restraint use (n = 20), inappropriate antibiotic prescribing (n = 3), indwelling or unnecessary urinary catheters use (n = 2), ordering unnecessary liver function tests (n = 1), and unnecessary antipsychotic prescribing (n = 1). Fourteen studies showed a significant reduction in low-value nursing procedures. Thirteen of these 14 studies included an educational component within their de-implementation strategy. Twelve controlled studies were included in the meta-analysis. Subgroup analyses for study design showed no statistically significant subgroup effect for the volume of low-value nursing procedures (p = 0.20).ConclusionsThe majority of the studies with a positive significant effect used a de-implementation strategy with an educational component. Unfortunately, no conclusions can be drawn about which strategy is most effective for reducing low-value nursing care due to a high level of heterogeneity and a lack of studies. We recommend that future studies better report the effects of de-implementation strategies and perform a process evaluation to determine to which extent the strategy has been used.Trial registrationThe review is registered in Prospero (CRD42018105100).

De-implementing inappropriate health interventions is essential for minimizing patient harm, maximizing efficient use of resources, and improving population health. Research on de-implementation has expanded in recent years as it cuts across types of interventions, patient populations, health conditions, and delivery settings. This commentary explores unique aspects of de-implementing inappropriate interventions that differentiate it from implementing evidence-based interventions, including multi-level factors, types of action, strategies for de-implementation, outcomes, and unintended negative consequences. We highlight opportunities to continue to advance research on the de-implementation of inappropriate interventions in health care and public health.

BackgroundThe use of cardiovascular disease (CVD) prevention guidelines based on absolute risk assessment is poor around the world, including Australia. Behavioural barriers amongst GPs and patients include capability (e.g. difficulty communicating/understanding risk) and motivation (e.g. attitudes towards guidelines/medication). This paper outlines the theory-based development of a website for GP guidelines, and piloting of a new risk calculator/decision aid.MethodsStage 1 involved identifying evidence-based solutions using the Behaviour Change Wheel (BCW) framework, informed by previous research involving 400 GPs and 600 patients/consumers. Stage 2 co-developed website content with GPs. Stage 3 piloted a prototype website at a national GP conference. Stage 4 iteratively improved the website based on “think aloud” interviews with GPs and patients. Stage 5 was a feasibility study to evaluate potential efficacy (guidelines-based recommendations for each risk category), acceptability (intended use) and demand (actual use over 1 month) amongst GPs (n = 98).ResultsStage 1 identified GPs as the target for behaviour change; the need for a new risk calculator/decision aid linked to existing audit and feedback training; and online guidelines as a delivery format. Stage 2-4 iteratively improved content and format based on qualitative feedback from GP and patient user testing over three rounds of website development. Stage 5 suggested potential efficacy with improved identification of hypothetical high risk patients (from 26 to 76%) and recommended medication (from 57 to 86%) after viewing the website (n = 42), but prescribing to low risk patients remained similar (from 19 to 22%; n = 37). Most GPs (89%) indicated they would use the website in the next month, and 72% reported using it again after one month (n = 98). Open feedback identified implementation barriers including a need for integration with medical software, low health literacy resources and pre-consultation assessment.ConclusionsFollowing a theory-based development process and user co-design, the resulting intervention was acceptable to GPs with high intentions for use, improved identification of patient risk categories and more guidelines-based prescribing intentions for high risk but not low risk patients. The effectiveness of linking the intervention to clinical practice more closely to address implementation barriers will be evaluated in future research.

BackgroundIn healthcare settings, system and organization leaders often control the selection and design of implementation strategies even though frontline workers may have the most intimate understanding of the care delivery process, and factors that optimize and constrain evidence-based practice implementation within the local system. Innovation tournaments, a structured participatory design strategy to crowdsource ideas, are a promising approach to participatory design that may increase the effectiveness of implementation strategies by involving end users (i.e., clinicians). We utilized a system-wide innovation tournament to garner ideas from clinicians about how to enhance the use of evidence-based practices (EBPs) within a large public behavioral health system.MethodsOur innovation tournament occurred in three phases. First, we invited over 500 clinicians to share, through a web-based platform, their ideas regarding how their organizations could best support use of EBPs. Clinicians could rate and comment on ideas submitted by others. Second, submissions were judged by an expert panel (including behavioral scientists, system leaders, and payers) based on their rated enthusiasm for the idea. Third, we held a community-facing event during which the six clinicians who submitted winning ideas presented their strategies to 85 attendees representing a cross-section of clinicians and system and organizational leaders.ResultsWe had a high rate of participation (12.3%), more than double the average rate of previous tournaments conducted in other settings (5%). A total of 65 ideas were submitted by 55 participants representing 38 organizations. The most common categories of ideas pertained to training (42%), financing and compensation (26%), clinician support and preparation tools (22%), and EBP-focused supervision (17%). The expert panel and clinicians differed on their ratings of the ideas, highlighting value of seeking input from multiple stakeholder groups when developing implementation strategies.ConclusionsInnovation tournaments are a useful and feasible methodology for engaging end users, system leaders, and behavioral scientists through a structured approach to developing implementation strategies. The process and resultant strategies engendered significant enthusiasm and engagement from participants at all levels of a healthcare system. Research is needed to compare the effectiveness of strategies developed through innovation tournaments to strategies developed through design approaches.

BackgroundThe science of effective teams is well documented; far less is known, however, about how specific team configurations may impact primary care team effectiveness. Further, teams experiencing frequent personnel changes (perhaps as a consequence of poor implementation) may have difficulty delivering effective, continuous, well-coordinated care. This study aims to examine the extent to which primary care clinics in the Veterans Health Administration have implemented team configurations consistent with recommendations based on the Patient-Centered Medical Home model and the extent to which adherence to said recommendations, team stability, and role stability impact healthcare quality. Specifically, we expect to find better clinical outcomes in teams that adhere to recommended team configurations, teams whose membership and configuration are more stable over time, and teams whose clinical manager role is more stable over time.Methods/designWe will employ a combination of social network analysis and multilevel modeling to conduct a database review of variables extracted from the Veterans Health Administration’s Team Assignments Report (TAR) (one of the largest, most diverse existing national samples of primary care teams (nteams > 7000)), as well as other employee and clinical data sources. To ensure the examination of appropriate clinical outcomes, we will enlist a team of subject matter experts to select a concise set of clear, prioritized primary care performance metrics. We will accomplish this using the Productivity Measurement and Enhancement System, an evidence-based methodology for developing and implementing performance measurement.DiscussionWe are unaware of other studies of healthcare teams that consider team size, composition, and configuration longitudinally or with sample sizes of this magnitude. Results from this study can inform primary care team implementation policy and practice in both private- and public-sector clinics, such that teams are configured optimally, with adequate staffing, and the right mix of roles within the team.Trial registrationNot applicable—this study does not involve interventions on human participants.

BackgroundThe World Health Organization (WHO) revised its clinical guidelines for management of childhood pneumonia in 2013. Significant delays have occurred during previous introductions of new guidelines into routine clinical practice in low- and middle-income countries (LMIC). We therefore examined whether providing enhanced audit and feedback as opposed to routine standard feedback might accelerate adoption of the new pneumonia guidelines by clinical teams within hospitals in a low-income setting.MethodsIn this parallel group cluster randomized controlled trial, 12 hospitals were assigned to either enhanced feedback (n = 6 hospitals) or standard feedback (n = 6 hospitals) using restricted randomization. The standard (network) intervention delivered in both trial arms included support to improve collection and quality of patient data, provision of mentorship and team management training for pediatricians, peer-to-peer networking (meetings and social media), and multimodal (print, electronic) bimonthly hospital specific feedback reports on multiple indicators of evidence guideline adherence. In addition to this network intervention, the enhanced feedback group received a monthly hospital-specific feedback sheet targeting pneumonia indicators presented in multiple formats (graphical and text) linked to explicit performance goals and action plans and specific email follow up from a network coordinator. At the start of the trial, all hospitals received a standardized training on the new guidelines and printed booklets containing pneumonia treatment protocols. The primary outcome was the proportion of children admitted with indrawing and/or fast-breathing pneumonia who were correctly classified using new guidelines and received correct antibiotic treatment (oral amoxicillin) in the first 24 h. The secondary outcome was the proportion of correctly classified and treated children for whom clinicians changed treatment from oral amoxicillin to injectable antibiotics.ResultsThe trial included 2299 childhood pneumonia admissions, 1087 within the hospitals randomized to enhanced feedback intervention, and 1212 to standard feedback. The proportion of children who were correctly classified and treated in the first 24 h during the entire 9-month period was 38.2% (393 out of 1030) and 38.4% (410 out of 1068) in the enhanced feedback and standard feedback groups, respectively (odds ratio 1.11; 95% confidence interval [CI] 0.37–3.34; P = 0.855). However, in exploratory analyses, there was evidence of an interaction between type of feedback and duration (in months) since commencement of intervention, suggesting a difference in adoption of pneumonia policy over time in the enhanced compared to standard feedback arm (OR = 1.25, 95% CI 1.14 to 1.36, P < 0.001).ConclusionsEnhanced feedback comprising increased frequency, clear messaging aligned with goal setting, and outreach from a coordinator did not lead to a significant overall effect on correct pneumonia classification and treatment during the 9-month trial. There appeared to be a significant effect of time (representing cumulative effect of feedback cycles) on adoption of the new policy in the enhanced feedback compared to standard feedback group. Future studies should plan for longer follow-up periods to confirm these findings.Trial registrationUS National Institutes of Health—ClinicalTrials.gov identifier (NCT number) NCT02817971. Registered September 28, 2016—retrospectively registered.