Response

How Do We Address Health Care Inequalities for Transcatheter Aortic Valve Implantation in Canada?

Are adaptive randomised trials or non-randomised studies the best way to address the Ebola outbreak in west Africa?

Outcomes of Implementing Rapid Triage in the Pediatric Emergency Department

Individuals with serious mental illness often are unable to access consumer- and family-oriented community care, resulting in repeated hospitalizations, incarceration, and homelessness. The "medical home" concept was developed in primary care to provide accessible and accountable services for individuals with chronic medical conditions. Building on the work done in primary care, the authors propose a "mental health home." The model of care incorporates medical home characteristics, such as access to and coordination of services, integration of primary and preventive care, adoption of recovery orientation and evidence-based practices, and family and community outreach. Barriers to and strategies for implementation of mental health homes are discussed.

A New Kind of Homelessness for Individuals With Serious Mental Illness? The Need for a "Mental Health Home"

On Being Better Kidney Doctors: Understanding Trajectories, Probabilities, Predictability, and People

“Future randomized controlled studies are needed” is a conclusion all too familiar to readers of scientific articles. This statement, distributed liberally in the literature, has come to signify little more than an obligatory admission of a study’s deficiencies. It simultaneously seeks to absolve the authors of any fault from potentially exaggerating the importance of their statistical findings while leaving it to the reader to decide how to apply and disseminate the novel information. The statement also implies that higher quality studies could and should be devised and points to salvation by way of the randomized controlled trial (RCT), the pinnacle of evidence-based medicine.1 Despite this, historical and present-day data show that RCTs are undertaken less frequently in surgery than in other areas of medicine.2–4 This raises concerns about the quality of surgical research, but what can surgeons do about it? Surveys have demonstrated a preponderance of cohort studies and case series in surgical journals, with RCTs accounting for fewer than 10% of publications.5,6 Reasons for lack of quality surgical research have been attributed to unique methodologic, practical, and ethical considerations in evaluating surgical procedures, and surgeon reluctance and lack of experience with surgical trials.3,7,8 Although high-quality RCTs are something to strive for, it is also clear that not all problems in surgery can be evaluated in an RCT. Solomon and McLeod studied 250 research questions in gastrointestinal surgery and concluded that, under ideal conditions, only 38.8% could have been answered with an RCT.9 Rather than attempting to answer every question with an RCT, surgeons should focus on using the best research design for their clinical questions. A variety of study designs can shed important insights if they are applied to the correct problem and conducted with methodologic rigor. The idea, development, exploration, assessment, and long-term study (IDEAL) model is a five-stage framework developed to guide best research practices throughout progressive stages of surgical innovation (Table 1).10,11 Early stages focus on refining the technique and its safety profile, whereas later stages are generally amenable to higher levels of scientific inquiry. With consideration for the stage of development of the surgical technique being investigated, a simple stepwise approach can be applied using the highest feasible level of evidence when choosing a study design. In this article, we outline important concepts and discuss various study designs—RCTs, observational studies with control groups, and case series—with a focus on minimizing biases. We consider various practical and methodologic issues specific to surgical research throughout. Table 1. - IDEAL Stages of Surgical Innovationa Stage Objectives Study Methods 1. Idea Initial technical description of new surgical procedure or technology Case reports, case series 2a. Development Description of early experience with new procedure or technology, including technical modifications and short-term/safety-related outcomes Prospective case series, observational studies 2b. Exploration Larger exploratory study to better define the relevant comparator groups and outcomes, and evaluate feasibility of a future RCT Prospective observational studies, RCTsb 3. Assessment Definitive comparative evaluation against the standard of care RCTs, large prospective observational studiesc 4. Long-term study Collect data for surveillance in real-world practice, identify new complications, and revise indications as needed Registry-based prospective cohort studies or case-control studies, rare case reports aDeveloped from initial description of IDEAL framework by McCulloch P, Altman DG, Campbell WB, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009;374:1105–1112.bPerformed when feasible at this stage.cAppropriate when ethical or practical factors preclude performing an RCT. DESIGNING A SURGICAL RESEARCH STUDY A 1994 commentary entitled “The Scandal of Poor Medical Research” was a heated outcry for higher quality research to be performed for a purpose greater than expanding the researcher’s curriculum vitae.12 It pointed to a variety of factors plaguing medical research, including flawed designs, inadequate samples, inappropriate statistical analyses, and misleading interpretation of results. At the heart of the problem was the combination of the “publish or perish” research climate together with a “general failure to appreciate the basic principles underlying scientific research.” Although it is not the goal of this article to review these principles in detail, the concepts of validity, bias, and statistics warrant consideration for our subsequent discussion. Once a question and hypothesis have been established, choosing the study design is the next step. The process should consider ways to maximize validity and minimize bias. Internal validity refers to the accuracy with which the findings reflect true variation in the outcomes measured in the study sample, whereas external validity refers to the extent to which the findings are generalizable to clinical populations encountered in practice.13 The interplay between these concepts is complex. By way of selective inclusion and exclusion criteria, RCTs ensure that groups of patients compared are equivalent in almost all aspects, other than the treatment administered, in striving to achieve a high level of internal validity. This can come at the expense of external validity. For instance, surgeons in real-world practice encounter patients who would not meet the strict inclusion and exclusion criteria of such studies. Biases refer to factors outside of the surgical intervention that systematically influence the results of a study.14 They detract directly from the validity of the study and can occur during study design and preparation, data collection, analysis, and interpretation of results.13,14 Biases are categorized conceptually in terms of their overarching similarities.15 Important conceptual categories are selection biases, observation biases, and confounders (Table 2). Selection biases occur during design of a study and result in unintended differences between comparison groups. They are attributable to systematic flaws in the methodology used to select patients. For instance, in a nonrandomized study investigating a novel surgical procedure compared with nonoperative treatment, patients selected to undergo surgery would likely be healthier, more compliant, and have better access to care than patients treated nonoperatively. These baseline factors stack the odds in favor of surgical outcomes. Observation biases occur during data collection and can be attributed to flaws in the measurement tools used, effects of the study environment on patient behavior, or deferential attitudes of surgeons toward one of the groups. For example, surgeons may be more invested in obtaining favorable outcomes and minimizing complications of surgically treated patients, unintentionally providing better follow-up care. A confounder is any unaccounted for “third variable” responsible for the causal relationship observed in the study. This could be a patient factor, such as smoking, in a study comparing infection rates after different wound care regimens, or a treatment-related factor, such as asymmetry in the duration of postoperative immobilization, in a study comparing different interventions for fractures. Such factors could be more pertinent than the treatments themselves to the outcomes being investigated. Table 2. - Examples of Biases in Surgical Research Bias Description Means of Minimization Selection biases Nonrespondent bias Patients declining to participate in the study differ systematically from those who take part Optimizing communication of objectives, limiting burden of participating Referral bias Patients referred to clinician and recruited to study have more severe disease Predefined and evidence-based inclusion criteria Channeling bias Patients assigned to interventions based on prognostic factors Randomized studies with allocation concealment, stratification in nonrandomized studies Chronology bias Intervention group compared with historical cohort Using concurrent comparison groups Observation biases Recall bias Inaccurate reporting of treatment response by patients Using prospective design and objective outcome measurements Hawthorne effect Change in behavior as a result of participation in a study Randomization, blinding Performance bias Results influenced by variability in surgical technique or expertise Standardized protocols, different surgeons for different procedures, stratification by surgeon level of expertise Transfer bias Patients lost to follow-up differ from those who completed study By maintaining up-to-date contact information, limiting study burden, issuing reminders Detection bias Outcomes not measured uniformly in comparison groups Concurrent comparison groups, blinding, predefined objective outcomes Conflict of interest Competing interests or secondary gain influence outcomes Mandatory disclosure, randomized blinded designs Misuse of statistics and inappropriate interpretation of statistical findings are other common culprits of poor surgical research. The majority of these issues are attributable to unintentional oversights, but a small proportion are caused by deliberate fraud.16,17 Common statistical errors include inadequate power, selection of inappropriate statistical tests, and inappropriate subgroup analysis.18–20 Complex statistical techniques are often relied on to make up for deficiencies in the study design, such as lack of randomization. Studies also tend to focus on statistical differences without considering clinically important thresholds, leading to exaggeration of the findings. Like strategies to minimize bias, the use of statistics should be planned when designing the study, with primary and secondary outcome variables and the statistical plan specified a priori. A well-designed study can also simplify the statistical methods needed. A thorough understanding of validity, bias, and statistical methods is critical when selecting any of the specific study designs discussed below. RANDOMIZED CONTROLLED TRIALS Now suppose that you have formulated a novel and structurally sound research question and have the resources at your institution to conduct the ideal study. A logical approach would be to start atop the pyramid of evidence and determine whether an RCT could be devised before considering other study types at lower levels of evidence.21 RCTs are generally most appropriate in the exploration or assessment stage of surgical innovation, once the technical aspects and safety profile of the procedure are established. Conducting an RCT requires intensive devotion of resources and time. The Wrist and Radius Injury Surgical Trial, led by the senior author (K.C.C.) to evaluate outcomes of treatments for distal radius fractures, took 10 years to complete and required funding from two National Institutes of Health grants.22 Compared with other types of studies, the same factors that make the RCT a coveted entity can also make it unwieldy in surgical research. Ethical and Practical Considerations A variety of ethical and practical considerations pose challenges for surgical RCTs. Like other comparative studies, RCTs are designed to evaluate a null hypothesis, which requires a control group. Unlike pharmaceutical trials, a methodologically equivalent control in surgery, such as a placebo, is rarely possible. One might be tempted to consider nonoperative treatment to be analogous to the placebo. This common view has been the basis of many surgical trials but is often misguided. In reality, nonoperative treatments are highly variable and difficult to control. They can be interventions such as activity modification, immobilization, physical therapy, injections, or a combination thereof. Patients may also implement their own individualized lifestyle changes, such as specialized diets and fitness regimens. Even if the nonoperative treatment is tightly and reliably controlled, the natural history of the patient’s condition may confound outcomes. In surgical trials, the true equivalent of a placebo would be sham surgery, a notion that transgresses ethical boundaries except in the most selective circumstances.23 In most cases, the use of an RCT to evaluate a surgical procedure mandates clinical equipoise between the surgical treatment being investigated and some other treatment.8 Nevertheless, even the existence of true clinical equipoise does not guarantee the feasibility of an RCT. Randomization and allocation concealment can be difficult to execute.7,8 These attributes strip the patient and physician of the opportunity for shared decision-making. Patients may be uncomfortable with randomized treatment selection without input from the surgeon, or with the notion of being part of an experiment. Patients often arrive at the surgeon’s office with a preference for a particular treatment.24 Even when all of these factors are accounted for, patients who agree to participate in an RCT may differ from patients who do not consent, introducing selection bias.3 Ethical considerations are not limited to the patient. Surgeons conducting the study must strive to treat participants equally to avoid unduly influencing the results of the treatment groups. However, surgeons often have their own preferences for one procedure over another.25 If the same surgeon is performing multiple procedures in the study, he or she may also be more facile in one of the procedures, creating potential for performance bias. This brings forth the issue of blinding, which is generally easier to accomplish in medical trials than in surgical trials. Clearly, the surgeon performing the operation cannot be blinded, but it is also difficult to ensure that patients are blinded to the treatment that they received. Patients may become aware of the surgery that was performed based on the shape of a scar, length of time spent in the operating room, or some other discernible feature. The ethics of withholding information about the surgery from the patient and examiners at follow-up visits are controversial, particularly if difficulties with recovery or complications arise. Cost and challenges recruiting patients and ensuring they return to follow-up are other factors that can make the RCT unfeasible.26 Given some of the ethical and practical factors associated with RCTs, other study designs discussed below may be better options. Minimizing Bias in RCTs Good design is crucial for maximizing validity and minimizing bias. Alarmingly, analyses of all RCTs published in The Journal of Bone and Joint Surgery between 1998 and 2013 showed that trials evaluating surgical procedures (as opposed to nonoperative or medical interventions) and with surgeons as first authors scored significantly lower in quality.27,28 To illustrate features of a well-designed surgical RCT, consider a landmark trial investigating open reduction and internal fixation versus casting for minimally displaced scaphoid waist fractures (the Scaphoid Waist Internal Fixation for Fractures Trial).29 This study sought to answer a timely question that has been a longstanding area of clinical equipoise. Allocation concealment and randomization were used with a remote randomization service that stratified patients based on their pattern of fracture displacement, further minimizing confounders. Sample size was determined based on a priori power analysis, and statistical analyses were performed by a blinded statistician following a strict prespecified plan. Primary and secondary outcomes were stated from the outset, and potentially subjective radiologic tests were evaluated independently by two musculoskeletal radiologists and a surgeon. Instead of blinding patients and surgeons, the surgical technique and postoperative protocols were left to the discretion of the surgeons, allowing patient-centered care and accounting for differences in surgeons’ technical preference and skill. This practical approach sought to minimize bias while also maximizing external validity, which was important for a trial aiming to inform clinical practice on an international scale. Empiric data support the importance of sound study design on the results of an RCT. Colditz et al. found that the likelihood of an improved outcome from an experimental treatment increased significantly in the absence of randomization.30,31 In a study analyzing 250 controlled trials, treatment effects were increased by 41% when allocation concealment was inadequate and 30% when the extent of concealment was unclear.32 Trials that were not double-blinded yielded treatment effects 17% greater than double-blinded trials. Methodologic problems like these can lead to inaccurate interpretation of findings and distort knowledge found in the published literature. They can be avoided with use of validated quality checklists such as the Consolidated Standards of Reporting Clinical Trials guidelines when planning a study.33 Such frameworks serve as an important quality safeguard and should be familiar to surgeons undertaking RCTs. Grading systems specific to plastic surgery have also been developed.34 OBSERVATIONAL STUDIES WITH A CONTROL GROUP Many questions in surgery cannot be addressed with an RCT but are amenable to other study designs. A nonrandomized study with a control group should be considered when one of the ethical or practical factors discussed above precludes an RCT. Commonly used designs are cohort studies and case-control studies (Fig. 1).15 Although cohort studies are in a higher tier of evidence, both designs have specific indications, methodologic considerations, and biases.Fig. 1.: Observational study designs.Like RCTs, cohort studies can be highly influential in their scope. Because of fewer upfront demands on participants and less stringent inclusion criteria, these studies lend themselves to broader investigations when more knowledge is needed before performing an RCT. For instance, a prospective cohort study was devised to evaluate the effectiveness and safety of immediate implant-based breast reconstruction with or without mesh following mastectomy (Implant Breast Reconstruction Evaluation study).35 This is a current topic of interest for many plastic surgeons, with recent trends toward immediate reconstruction following mastectomy.36 Although previous RCTs evaluating immediate breast reconstruction had been performed, their validity was questioned because of performance biases and low sample size.37–39 The Implant Breast Reconstruction Evaluation study addressed this gap in knowledge with an exploratory design that included more than 2000 patients from 81 participating centers. Patients underwent different variations of breast reconstruction at the discretion of the treating surgeon, eliminating concerns for patient preference that are a major hindrance to RCTs in breast surgery.40 Multiple predefined primary outcomes were evaluated and multivariable logistic regression was used to control for confounders. In addition to eliminating practical hurdles of RCTs, advantages of cohort studies illustrated in this example include obtaining larger samples, evaluating multiple exposures and outcomes at once, and generating new hypotheses for future studies through an exploratory approach. Surgeons must also be aware of the pitfalls of nonrandomized studies. Without randomization, cohort studies have limited ability to account for confounders. A causal relationship is inferred from the chronologic sequence of exposures and outcomes, which is most convincing in prospective studies. Without blinding or allocation concealment, cohort studies are also susceptible to selection biases. Recall bias is a limitation for retrospective cohort studies because of factors such as inaccurate patient recall and reliance on information from patient charts.41 Variability in documentation practices makes the fidelity of large databases particularly concerning. In case-control studies, the inference of causation is especially problematic because comparison groups are selected based on their outcomes and analyzed retrospectively for exposures. Designing an observational study with a control group requires paying special attention to these inherent limitations and must be individualized to the research question. Exposures, outcomes, inclusion criteria, and statistical methods should all be planned in advance, even for studies meant to be “exploratory” or hypothesis-generating. Observational studies should be evaluated using standardized guidelines such as the Strengthening the Reporting of Observational Studies in Epidemiology framework.42,43 Retrospective cohort studies lend themselves to studying rare conditions and procedures, particularly with the availability of large national databases. Vast amounts of data can be extracted and analyzed rapidly. Meanwhile, case-control studies are useful for investigating factors underlying rare outcomes or complications. When using a registry or database, its purpose, completeness, and accuracy should be carefully evaluated.44 If the question would be better suited for an RCT and there are no major practical or ethical barriers, a higher level of evidence design should be pursued. CASE SERIES An appraisal of articles published in the general surgery literature in 1996 compared the preponderance of case series, 46% of all studies, to “comic opera,” pointing to a necessary shift toward higher quality research.6 A similar survey conducted 10 years later found the proportion of case series had decreased to 34%, with 51% of publications being cohort studies.5 Case series, which typically involve a retrospective account of outcomes from a single procedure, often from a single institution and surgeon, remain a common avenue for publishing novel information. The absence of a control group is their primary flaw, as a hypothesis cannot be tested.44 Retrospective case series are also culprits of publication bias, as new procedures with poor outcomes are unlikely to be published.45–47 Furthermore, failure to favorable results of new techniques are unlikely to be concerns about the validity of a case series from a single Case series are for providing for subsequent hypothesis They should be undertaken with specific such as the safety of a novel procedure or rare or complications. The IDEAL model the use of a case series to be appropriate in the or development first with a new procedure are a and are to the surgical To some of the pitfalls discussed prospective series should be performed with protocols for the new procedure and in of conducting the study. studies should also be to publication The use of retrospective series should be but at a involve patients without and a standardized reporting in the case of RCTs and observational studies, validated checklists can be used to minimize bias when reporting novel surgical interventions in a case A simple approach can be used to the best research design for a surgical question. with consideration for the IDEAL stage of the surgery being investigated, the highest level of appropriate for that stage should be evaluated for feasibility before considering designs at lower levels of the evidence RCTs the highest level of but are not suited for many questions. such as observation studies and case series, to be when practical and ethical considerations preclude the use of an RCT. of the study surgical research should quality design and minimize bias. funding from the National Institutes of from and and a research from to study outcomes. has no interest to in to the of this The authors would like to for and to the development of this They appreciate the review and from of at the of

Preoperative wait times in high-grade endometrial cancer: do surgical delays impact patient survival?

How should medical science change?

A Systems Approach to Morbidity and Mortality Conference

This article is the third in a 6-part series to chronicle the processes, work, and outcomes of the ACGME Pursuing Excellence Pathway Innovators Project. These articles provide an overview of the project, detail the 4 drivers developed to define the project, and present the evaluation process developed. This article highlights initiatives that integrate health system learners into quality, safety, equity, and value processes of health care organizations.While the National Academy of Medicine's To Err Is Human1 and Crossing the Quality Chasm2 ushered in a new focus on quality and safety, and Unequal Treatment3 focused on health disparities, the ACGME Clinical Learning Environment Review (CLER) Program centered the importance of these principles in the clinical learning environment (CLE).The CLER Program, developed in 2012, increased the focus on Sponsoring Institutions' engagement of residents and fellows in quality improvement and patient safety within the CLE.4 The new focus was in part motivated by literature describing wide variations in care for standard practices, value-based care, and patient outcomes for similar conditions across geographic regions and health care systems. Layered on this is a growing body of literature demonstrating that residents and fellows who practice in health systems with greater attention to high-quality care delivery carry these behaviors forward into practice, and therefore their patients may experience better health outcomes.5The first CLER National Report of Findings published in 2016 highlighted significant variation between institutions in the degree to which learners meaningfully engaged in activities related to quality, safety, equity, and value.6 For example, although residents were often aware of institutional quality priorities, they often defaulted to existing solutions and/or guidelines, rather than using knowledge and skills to develop and apply new approaches. Residents' improvement work was often siloed by profession and department, with limited engagement in interprofessional teams. Specific improvement efforts identified by residents related to disparities focused on provision of services to low-income or marginalized populations, rather than linking quality to equity. Robust programs to engage residents and fellows in identifying and reducing local disparities were lacking. Lastly, a lack of experiential engagement in patient safety at all levels was observed, from frontline incident reports to high-level engagement with service- and institution-specific safety priorities.The Pursuing Excellence in Clinical Learning Environments initiative (Pursuing Excellence) was created to address the variation across Sponsoring Institutions identified in the CLER National Report of Findings 2016 by testing innovations as part of a longitudinal social learning model. A core activity of Pursuing Excellence was the establishment of the Pathway Innovators Collaborative with the aim of enhancing the integration of graduate medical education and the health care delivery system to enable measurable improvement in patient outcomes and learner experience.7 Through a competitive application process 8 diverse Sponsoring Institutions,8 varying in size and governance structures, were selected to participate in the Pathway Innovators Collaborative. Their charge was to work within their organizations and across Sponsoring Institutions to create sustainable and adaptable projects and processes that could improve the CLE and be shared with the entire GME learning community. This article provides a detailed perspective on the second of 4 major areas of work within the Pathway Innovators Collaborative.Four primary drivers for improvement of the CLE for residents and fellows were identified to guide the Collaborative. These primary drivers included focusing on alignment of the CLE and GME, engaging residents and staff in quality, safety, value, and equity, faculty development, and interprofessional learning. All of these primary drivers included multiple secondary drivers (see Figure). Half of the teams began by focusing their initial efforts on Driver 2, “Establish the processes and practices that fully integrate clinical learning environment staff and learners into the pursuit of quality, safety, equity, and value in the organization.” This approach reflects the growing recognition of health systems science that complements and synergizes with traditional basic and clinical sciences. All 8 sites met through sequential learning sessions, including in-person site visits, guided by a 4-year curriculum to refine their innovations.The Collaborative teams noted several factors to consider for implementation of Driver 2. A key factor was that patient safety and quality improvement work often relied on established nursing unit–based structures. The transient nature of residents was not conducive to a shared learning experience. While residents commonly rotate through units, they are not a permanent part of the unit-based team, and may work on multiple rotations across units, in some cases across multiple hospitals. Also, many units held quality or safety huddles at set times to review institutional and local metrics, and residents were often not aware of those huddles or able to attend due to other clinical or educational obligations. The lack of integrated resident and unit-based quality and safety work could result in projects at risk of not being sustained after a resident rotated off-service or graduated.An additional factor was lack of improvement expertise available at the point of care where residents work. Many institutions have improvement experts charged with overseeing institutional improvement initiatives (ie, Operational Excellence, Lean Transformation); however, that expertise was often not known or available to residents. Likewise, residents and fellows often did not know what quality data were routinely collected or how to access these data to drive improvement. Recognizing these challenges, the Collaborative undertook the work of implementing and sustaining an infrastructure that aligned and integrated learners with staff through a specified set of action steps articulated below.Each of the Collaborative teams utilized the secondary drivers of the Driver Diagram to begin the journey toward integrating residents and fellows in the quality, safety, value, and equity mission of the Sponsoring Institution. Notably, the Collaborative found it more practical and logical to approach the secondary drivers in the reverse order on the driver diagram. Articulated below is how the teams implemented each of the secondary drivers, presented in the sequence we propose for future use. To make progress, some overlap will likely be required. Specific examples of team approaches are in the Table.Obtaining institutional resources and infrastructure to engage residents into quality, safety, equity, and value work is foundation to successful integration. This requires demonstrating a return on investment, allowing health system leaders to see this work as mission critical to health care delivery. Resources and infrastructure not only come in the form of leadership buy-in, but also in staff to support the work of engaging residents. The participants noted that this work can be both an educational and a clinical priority for the institution. One important way that alignment was achieved in the Collaborative was the explicit involvement of institutional leaders on each Pathway Innovators Steering Committee. This helped institutional leaders gain better understanding of the work of graduate medical education and promote and shape institutional strategies. Residents and fellows can identify challenges and become important allies to identify and solve systemic problems. In addition, each participating team created a business plan to gain buy-in from the C-suite by demonstrating the return on investment of meaningfully preparing faculty to effectively integrate residents, as well as other staff, into institutional improvement efforts. Interprofessional quality and safety initiatives were amplified and strengthened with broader participation. Bringing groups together helped standardized work and contributed to other efforts to improve patient outcomes, including length of stay, discharge planning, and interprofessional teamwork.A key element of success is ensuring that residents are not only aware of, but that they engage in health system–level approaches to improving quality, safety, value, and equity. The participants noted that it is especially important to ensure maximal impact and alignment of resident work with that of the health system. While many of the Collaborative organizations use Lean process improvement to drive transformation, residents were often an afterthought in the design of institutional improvement activities. The Collaborative helped institutional leaders reassess their Lean process improvement activities to include residents and fellows so they are able to enact changes that can improve patient care, interprofessional learning, and participation. These changes can be embedded into the system and thereby sustained after residents graduate. For example, several of the teams (Cleveland Clinic, Maine Medical Center, UCSF, University of Chicago, and University of Rochester) specifically integrated residents into existing systems-based improvement work using Lean transformation teams and Operational Excellence so that all residents learn and apply the institutional approach to quality improvement (Table).Local expertise, including coaches, can be an important and practical approach to successfully training residents to engage in institutional quality, safety, equity, and value work. Institutions developed models which fit with their existing institutional frameworks. Examples include bringing together groups of learners and frontline providers with institutional improvement experts (Rochester) and training a cadre of interprofessional faculty coaches to support ongoing trainee-led improvement work (Cleveland Clinic, UCSF). Some Collaborative teams used in-situ simulation to do this (Children's National), while others trained faculty to lead quality improvement discussions in the context of rounds (Our Lady of the Lake).Health care quality pioneer Ernest A. Codman, MD, stated, “To effect improvement, the first step is to admit and record the lack of perfection.”9 Residents are accustomed to receiving subjective feedback about their clinical performance. They may be less likely to receive or understand their patient outcomes for a variety of reasons, including lack of data and lack of appreciation of their own impact.10 Two key elements to consider when providing outcomes data are the extent to which the data are timely and actionable.11 In many large programs, residents rotate between services, units, and even health systems on a monthly basis. If residents are to change their own behavior based on data, they must receive data while they are still on the rotation. It's not necessary they receive their own individual data, as residents are members of teams affecting the care of a patient. However, at a minimum, they should have benchmarks to understand what they should aim to achieve.Residents should view the data as actionable (eg, residents' behavior changes may reasonably result in outcome changes). One can identify examples in which the link is linear (eg, timely removal of Foley catheters leads to fewer CAUTIs) or less linear (patients filling their post-discharge medications may depend on factors that are both within and beyond the trainees' control). Teams noted that faculty (physicians or non-physicians) need to be equipped to help residents understand how to interpret and contextualize data to drive behavior change which leads to better patient outcomes. In the collaborative, this often meant close alignment with the chief quality officer or quality staff to provide residents with access to existing quality dashboards or designing and deploying new ones.As checklists, backup processes, and other types of improvement work have become “standard work,” health care systems have come to depend on non-physician clinicians and staff to ensure that best practices are followed. Examples include having operating room nurses take responsibility for time outs or clinical pharmacists review medication orders for safety. Clinical educators can take advantage of teaching opportunities to focus on topics such as improvement science, rounding on visibility boards, or interprofessional collaboration. Faculty can role model how these systems-based topics are integrated into daily practice, rather than being viewed as an “add-ons.” For example, at Dell Medical School, faculty on one of their medicine teams, named the “Green Team,” test innovations in care while on service with their learners.The Pathway Innovators experience demonstrates that the secondary drivers can be used to guide integration of residents and fellows into the quality, safety, equity, and value mission of the Sponsoring Institution. Our experience was that the order of the steps mattered. In fact, for many participants, we found the ideal order was the reverse of how the secondary drivers were originally presented. It is important to note that to make progress, secondary drivers may need to be addressed concurrently, as appropriate to the institution or activity.While provision of resources and infrastructure are foundational, governance of most institutions was not designed to account for residents, making it harder to solicit support and create authentic roles for residents. Ironically, newer institutions may have an advantage because they can prospectively design systems to meaningfully integrate residents.Going forward, all institutions should be challenged to rethink CLEs to ensure that residents and fellows receive experiential learning not only in clinical care but also in the topics of quality, safety, equity, and value foundational to health systems science. For example, existing clinical rotations can be redesigned to incorporate a focus on systems-based practice, as a value add for the health system and for patient care. While documenting the impact of residents on quality, safety, equity, and value has been done and can be labor-intensive, measuring outcomes and return on investment is critical to sustainability and long-term commitment of resources.Perhaps most importantly, residents and fellows must also see the value in this integration. With increased workload and attention to well-being and burnout in training, faculty are obligated to demonstrate why quality, safety, equity, and value matter for patients, as well as for physician professional satisfaction and development. Leveraging external pressures can be a strategy for institutions to meaningfully engage residents and fellows.

150 Background: In 2013, a patient reported satisfaction survey indicated 19% of patients waited 20-40 minutes, 8% 40-60 minutes and 4% over 1 hour. We initiated a project to objectively quantify the components of wait times to investigate opportunities for improvement. Methods: Utilizing existing technology in the practice management system, clinic staff use the Day List feature to capture time stamps as patients move through the clinic. We focused on provider appointments but these visits could also include business office, labs, infusion and diagnostics. It was important to define where the wait(s) occurred. The Time Stamp durations measured are as follows: Arrival to Depart – duration of each appointment; Arrival to site to Exam Start – duration of activity until ready to be seen by the provider, includes rooming, labs and business office activity. Used to compare to the patient satisfaction survey responses; Exam Start to Depart – the provider portion of the office visit, includes patient wait plus exam time. Three reports are generated: Time Stamp Error Report indicating the completeness of data collection; Average Wait Times Report with appointment counts by physician by site and average durations; Provider Wait Times Report with office visit counts, Wait Time Category counts ( < 10 min, 10-20, 20-40, 40-60, and > 1 hour ) and average durations. Results: There was a correlation calculation to the patient satisfaction survey of .779, with long wait times more likely to be underreported by patients. Site and physician data were available for review at site Quality Committees. The data can be used by the site to improve processes, such as lab and infusion room scheduling. Time stamps are used to communicate patient readiness for next steps in the office visit. The time stamps provide objective data to discuss patient complaints with staff. Conclusions: Patient wait times are a valued measure of patient satisfaction and quality. Full utilization of the Day List and supporting technology allows us to objectively monitor and improve this aspect of patient care. Table 1: Sample Provider Report [Table: see text]

Clinical Effectiveness and Early Goal-Directed Therapy for Severe Sepsis and Septic Shock

AimThe primary aim of this systematic review of the literature was to determine whether interventions to reduce waiting time in outpatient and community health services can be sustained. The secondary...

The Hawthorne effect: can it be measured and utilized?