Developing an Audit and Feedback Dashboard for Family Physicians: User-Centered Design Process.

<h3>Context:</h3> Audit and Feedback (A&amp;F), the summary and provision of clinical performance, is a popular quality improvement strategy. We are developing a web-based dashboard that uses data from the electronic medical record to help physicians identify gaps in care and act. However, A&amp;F tools can only be effective if the targeted health professionals actively review their data and take action. In order to maximise the impact of A&amp;F, the design should consider the needs and goals of clinicians. <h3>Objective:</h3> To describe the development of a user-centered design process to optimize the effect of an A&amp;F dashboard for family physicians. <h3>Study Design and Analysis:</h3> Our design process includes (1) Prototype development based on A&amp;F theory and input from clinical improvement leaders; (2) a co-creation workshop with family physician quality improvement leaders to develop personas (i.e., fictional characters that represent an archetype character); (3) user-centered interviews with family physicians to learn about the physician’s who will be using the dashboard and their context, and their reactions to the dashboard. <h3>Setting or Dataset:</h3> A workshop for the creation of personas and user-centered qualitative interviews with family physicians. <h3>Population Studied:</h3> Family physicians who contribute data to the University of Toronto Practice-Based Research Network <h3>Intervention/Instrument:</h3> Audit and Feedback dashboard <h3>Outcome Measures:</h3> N/A <h3>Results:</h3> Our persona workshop produced four personas that enabled the team to identify physician’s needs and wishes and facilitated empathy during the design process: Dr. Skeptic, Frazzled Physician, The Eager Implementer, and Sidney Big Wig. Our interviews found that: (1) physicians are interested in how they compare with their peers; however, if their performance was above average, they were not motivated to improve even if gaps in care remained; (2) Burnout levels are high, physicians are trying to catch up on missed care during the pandemic, and were not highly motivated to act on the dashboard data; (3) Features that were important to physicians were integration within the EMR, and up-to-date and accurate data. <h3>Conclusions:</h3> A successful design of an A&amp;F performance dashboard should consider the serious lack of time and capacity among family physicians to engage in quality improvement work. If designed properly, the QI dashboard can be a great assistance in helping family physicians provide more proactive and targeted care.

—Thailand, as well as many other countries worldwide, is facing a shortage of medical staff. We propose a solution to improve medical services in rural areas: a robot teleoperation system that allow patients to consult with doctors from public hospitals, and for doctors to examine and make decisions about their required care. To develop such a system, we are following a user-centred design (UCD) process. Two research questions have been addressed: how should an effective robot teleoperation system be designed to support medical diagnosis and patient communication? And does the use of a social robot improve the social interaction and examination quality in a remote consultation? This paper discusses the overall UCD process, starting from requirement analysis with Thai end-users (finished), system design and development on Pepper robot (in-progress), and user-centred evaluation (future plan).

Scientific data volumes have been growing exponentially. This has resulted in the need for new tools that enable users to operate on and analyze data. Cyber infrastructure tools, including workflow tools, that have been developed in the last few years has often fallen short if user needs and suffered from lack of wider adoption. User-centered Design (UCD) process has been used as an effective approach to develop usable software with high adoption rates. However, UCD has largely been applied for user-interfaces and there has been limited work in applying UCD to application program interfaces and cyber infrastructure tools. We use an adapted version of UCD that we refer to as Scientist-Centered Design (SCD) to engage with users in the design and development of Tigres, a workflow application programming interface. Tigres provides a simple set of programming templates (e.g., sequence, parallel, split, merge) that can be can used to compose and execute computational and data transformation pipelines. In this paper, we describe Tigres and discuss our experiences with the use of UCD for the initial development of Tigres. Our experience-to-date is that the UCD process not only resulted in better requirements gathering but also heavily influenced the architecture design and implementation details. User engagement during the development of tools such as Tigres is critical to ensure usability and increase adoption.

Even though User-Centred Design (UCD) is widely accepted and employed in the design and development of interactive systems, there are limited guiding procedures on how such approaches can be translated to the development of products, systems, and services that are focusing primarily on accessibility. This paper reports on the application of UCD for such a system within an EU project CONTENT4ALL under the domain accessibility for the Deaf. Each step in the UCD process and the respective activities within each step are described in detail with a focus on the methods and techniques adopted for assisting in the development of the novel technology. The insights gained during the entirety of the user-centred design and evaluation process have led to a compilation of important factors for creating sign-translated media content. This possible guideline comprises a list of useful and necessary components pertaining to sign language delivery in media, particularly sign television. The objective of this paper is to highlight lessons learned presented in the form of recommendations for human factors researchers on key UCD procedures for the development of accessibility products, systems and services based on the performed user activities within the project. An attempt has been made to reduce the gap in literature and add to a possible UCD guiding process exclusively for accessibility.

IntroductionQuality improvement has failed to live up to its promise. In theory, the application of a method to solve complex quality issues holds face validity. In practice however, much of...

BackgroundStressful situations during intraoperative emergencies have negative impact on human cognitive functions. Consequently, task performance may decrease and patient safety may be compromised. Cognitive aids can counteract these effects and support anesthesiologists in their crisis management. The Professional Association of German Anesthesiologists set up a project to develop a comprehensive set of digital cognitive aids for intraoperative emergencies. A parallel development for several software platforms and stationary and mobile devices will accommodate the inhomogeneity of the information technology infrastructure within German anesthesia departments.ObjectiveThis paper aimed to provide a detailed overview of how the task of developing a digital cognitive aid for intraoperative crisis management in anesthesia was addressed that meets user requirements and is highly user-friendly.MethodsA user-centered design (UCD) process was conducted to identify, specify, and supplement the requirements for a digital cognitive aid. The study covered 4 aspects: analysis of the context of use, specification of user requirements, development of design solutions, and evaluation of design solutions. Three prototypes were developed and evaluated by end users of the application. Following each evaluation, the new requirements were prioritized and used for redesign. For the first and third prototype, the System Usability Scale (SUS) score was determined. The second prototype was evaluated with an extensive Web-based questionnaire. The evaluation of the third prototype included a think-aloud protocol.ResultsThe chosen methods enabled a comprehensive collection of requirements and helped to improve the design of the application. The first prototype achieved an average SUS score of 74 (SD 12), indicating good usability. The second prototype included the following main revisions: 2-column layout, initial selection of patient type (infant, adult, or parturient), 4 offered search options, and the option to check off completed action steps. Its evaluation identified the following major revision points: add quick selection for resuscitation checklists, design the top bar and tabs slightly larger, and add more pictograms to the text. The third prototype achieved an average SUS score of 77 (SD 15). The evaluation of the think-aloud protocol revealed a good intuitiveness of the application and identified a missing home button as the main issue.ConclusionsAnesthesiology—as an acute medical field—is particularly characterized by its high demands on decision making and action in dynamic, or time-critical situations. The integration of usability aspects is essential for everyday and emergency suitability. The UCD process allowed us to develop a prototypical digital cognitive aid, exhibiting high usability and user satisfaction in the demanding environment of anesthesiological emergencies. Both aspects are essential to increase the acceptance of the application in later stages. The study approach, combining different methods for determining user requirements, may be useful for other implementation projects in a highly demanding environment.

Adoption of Electronic Health Record (EHR) systems in all care settings is accelerating. EHRs can support and revolutionize the way information is used to provide high-quality and safe patient care. At the same time, however, issues with workflow integration have contributed to slow rates of EHR adoption in some specialty care settings. Workflow analysis is an integral part of the early stages of the User-Centered Design (UCD) process. UCD is an approach to designing systems and employs both formative and summative methods in order to achieve systematic discovery of useful functions grounded in an understanding of the work domain.

BackgroundDespite the unprecedented performance of deep learning algorithms in clinical domains, full reviews of algorithmic predictions by human experts remain mandatory. Under these circumstances, artificial intelligence (AI) models are primarily designed as clinical decision support systems (CDSSs). However, from the perspective of clinical practitioners, the lack of clinical interpretability and user-centered interfaces hinders the adoption of these AI systems in practice.ObjectiveThis study aims to develop an AI-based CDSS for assisting polysomnographic technicians in reviewing AI-predicted sleep staging results. This study proposed and evaluated a CDSS that provides clinically sound explanations for AI predictions in a user-centered manner.MethodsOur study is based on a user-centered design framework for developing explanations in a CDSS that identifies why explanations are needed, what information should be contained in explanations, and how explanations can be provided in the CDSS. We conducted user interviews, user observation sessions, and an iterative design process to identify three key aspects for designing explanations in the CDSS. After constructing the CDSS, the tool was evaluated to investigate how the CDSS explanations helped technicians. We measured the accuracy of sleep staging and interrater reliability with macro-F1 and Cohen κ scores to assess quantitative improvements after our tool was adopted. We assessed qualitative improvements through participant interviews that established how participants perceived and used the tool.ResultsThe user study revealed that technicians desire explanations that are relevant to key electroencephalogram (EEG) patterns for sleep staging when assessing the correctness of AI predictions. Here, technicians wanted explanations that could be used to evaluate whether the AI models properly locate and use these patterns during prediction. On the basis of this, information that is closely related to sleep EEG patterns was formulated for the AI models. In the iterative design phase, we developed a different visualization strategy for each pattern based on how technicians interpreted the EEG recordings with these patterns during their workflows. Our evaluation study on 9 polysomnographic technicians quantitatively and qualitatively investigated the helpfulness of the tool. For technicians with <5 years of work experience, their quantitative sleep staging performance improved significantly from 56.75 to 60.59 with a P value of .05. Qualitatively, participants reported that the information provided effectively supported them, and they could develop notable adoption strategies for the tool.ConclusionsOur findings indicate that formulating clinical explanations for automated predictions using the information in the AI with a user-centered design process is an effective strategy for developing a CDSS for sleep staging.

Learning about quality improvement (QI) in resident physician training is often relegated to elective or noncore clinical activities. The authors integrated teaching, learning, and doing QI into the routine clinical work of inpatient internal medicine teams at a Veterans Affairs (VA) hospital. This study describes the design factors that facilitated and inhibited the integration of a QI curriculum-including real QI work-into the routine work of inpatient internal medicine teams. A realist evaluation framework used three data sources: field notes from QI faculty; semistructured interviews with resident physicians; and a group interview with QI faculty and staff. From April 2011 to July 2012, resident physician teams at the White River Junction VA Medical Center used the Model for Improvement for their QI work and analyzed data using statistical process control charts. Three domains affected the delivery of the QI curriculum and engagement of residents in QI work: setting, learner, and teacher. The constant presence of the QI material on a public space in the team workroom was a facilitating mechanism in the setting. Explicit sign-out of QI work to the next resident team formalized the handoff in the learner domain. QI teachers who were respected clinical leaders with QI expertise provided role modeling and local system knowledge. Integrating QI teaching into the routine clinical and educational systems of an inpatient service is challenging. Identifiable, concrete strategies in the setting, learner, and teacher domains helped integrate QI into the clinical and educational systems.

Real-World Implementation of Sickle Cell Transition Programs As Part of a State Quality Improvement Network

The rapid advancement of AI technology has led to increasingly complex and opaque systems, creating a critical need for explainable AI (XAI) that enhances transparency and user trust. Despite extensive research on XAI methods and applications, the user-centered design (UCD) process for XAI remains fragmented and unclear. This systematic review analyzes 27 studies from 2020 to 2024 to develop a comprehensive framework for user-centered XAI system design (UCXAISD). We identify five key stages: contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement. Our framework aligns with traditional UCD processes while incorporating specialized elements for XAI, including user-centricity, transparency, and actionability. For each stage, we provide evidence-based guidelines derived from the literature. The framework serves as a blueprint for developing XAI systems that balance technical sophistication with user needs.

Value-based payments are rapidly replacing fee-for-service arrangements, necessitating advancements in physician practice capabilities and functions. The objective of this study was to examine potential differences among family physicians who are owners versus employed with respect to their readiness for value-based payment models. The authors surveyed more than 550 family physicians from the American Academy of Family Physician's membership; nearly 75% had made changes to participate in value-based payments. However, owners were significantly more likely to report that their practices had made no changes in value-based payment capabilities than employed physicians (owners 35.2% vs. employed 18.1%, P < 0.05). This study identified 3 key areas in which physician owners' value-based practice capabilities were not as advanced as the employed physician group: (1) quality improvement strategies, (2) human capital investment, and (3) identification of high-risk patients. Specifically, the employed physician group reported more quality improvement strategies, including quality measures, Plan-Do-Study-Act, root cause analysis, and Lean Six Sigma (P < 0.05 for all). More employed physicians reported that their practices had full-time care management staff (19.8% owners vs. 30.8% employed, P < 0.05), while owners were more likely to report that they had no resources/capacity to hire care managers or care coordinators (31.4% owners vs. 19.4% employed, P < 0.05). Owners were significantly more likely to respond that they do not have the resources/capacity to identify high-risk patients (23.1% owners vs. 19.3% employed, P < 0.05). As public and private payers transition to value-based payments, consideration of different population health management needs according to ownership status has the potential to support the adoption of value-based care delivery for family physicians.

In the 2015 Medicare Access and CHIP Reauthorization Act, the Centers for Medicare and Medicaid Services (CMS) began the transition to a value-based payment system through implementation of the Quality Payment Program (QPP) ([1][1]). Representing a paradigm shift away from traditional volume-based

Despite evidence supporting the effectiveness of diabetic retinopathy screening (DRS) in reducing the risk of sight loss, attendance for screening is consistently below recommended levels. The primary objective of the review was to assess the effectiveness of quality improvement (QI) interventions that seek to increase attendance for DRS in people with type 1 and type 2 diabetes.Secondary objectives were:To use validated taxonomies of QI intervention strategies and behaviour change techniques (BCTs) to code the description of interventions in the included studies and determine whether interventions that include particular QI strategies or component BCTs are more effective in increasing screening attendance;To explore heterogeneity in effect size within and between studies to identify potential explanatory factors for variability in effect size;To explore differential effects in subgroups to provide information on how equity of screening attendance could be improved;To critically appraise and summarise current evidence on the resource use, costs and cost effectiveness. We searched the Cochrane Library, MEDLINE, Embase, PsycINFO, Web of Science, ProQuest Family Health, OpenGrey, the ISRCTN, ClinicalTrials.gov, and the WHO ICTRP to identify randomised controlled trials (RCTs) that were designed to improve attendance for DRS or were evaluating general quality improvement (QI) strategies for diabetes care and reported the effect of the intervention on DRS attendance. We searched the resources on 13 February 2017. We did not use any date or language restrictions in the searches. We included RCTs that compared any QI intervention to usual care or a more intensive (stepped) intervention versus a less intensive intervention. We coded the QI strategy using a modification of the taxonomy developed by Cochrane Effective Practice and Organisation of Care (EPOC) and BCTs using the BCT Taxonomy version 1 (BCTTv1). We used Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, and Social capital (PROGRESS) elements to describe the characteristics of participants in the included studies that could have an impact on equity of access to health services.Two review authors independently extracted data. One review author entered the data into Review Manager 5 and a second review author checked them. Two review authors independently assessed risks of bias in the included studies and extracted data. We rated certainty of evidence using GRADE. We included 66 RCTs conducted predominantly (62%) in the USA. Overall we judged the trials to be at low or unclear risk of bias. QI strategies were multifaceted and targeted patients, healthcare professionals or healthcare systems. Fifty-six studies (329,164 participants) compared intervention versus usual care (median duration of follow-up 12 months). Overall, DRS attendance increased by 12% (risk difference (RD) 0.12, 95% confidence interval (CI) 0.10 to 0.14; low-certainty evidence) compared with usual care, with substantial heterogeneity in effect size. Both DRS-targeted (RD 0.17, 95% CI 0.11 to 0.22) and general QI interventions (RD 0.12, 95% CI 0.09 to 0.15) were effective, particularly where baseline DRS attendance was low. All BCT combinations were associated with significant improvements, particularly in those with poor attendance. We found higher effect estimates in subgroup analyses for the BCTs 'goal setting (outcome)' (RD 0.26, 95% CI 0.16 to 0.36) and 'feedback on outcomes of behaviour' (RD 0.22, 95% CI 0.15 to 0.29) in interventions targeting patients, and 'restructuring the social environment' (RD 0.19, 95% CI 0.12 to 0.26) and 'credible source' (RD 0.16, 95% CI 0.08 to 0.24) in interventions targeting healthcare professionals.Ten studies (23,715 participants) compared a more intensive (stepped) intervention versus a less intensive intervention. In these studies DRS attendance increased by 5% (RD 0.05, 95% CI 0.02 to 0.09; moderate-certainty evidence).Fourteen studies reporting any QI intervention compared to usual care included economic outcomes. However, only five of these were full economic evaluations. Overall, we found that there is insufficient evidence to draw robust conclusions about the relative cost effectiveness of the interventions compared to each other or against usual care.With the exception of gender and ethnicity, the characteristics of participants were poorly described in terms of PROGRESS elements. Seventeen studies (25.8%) were conducted in disadvantaged populations. No studies were carried out in low- or middle-income countries. The results of this review provide evidence that QI interventions targeting patients, healthcare professionals or the healthcare system are associated with meaningful improvements in DRS attendance compared to usual care. There was no statistically significant difference between interventions specifically aimed at DRS and those which were part of a general QI strategy for improving diabetes care. This is a significant finding, due to the additional benefits of general QI interventions in terms of improving glycaemic control, vascular risk management and screening for other microvascular complications. It is likely that further (but smaller) improvements in DRS attendance can also be achieved by increasing the intensity of a particular QI component or adding further components.

Vietnam is coping with an alarming increase of injury morbidity and mortality. Injury data is an indispensable component for developing and implementing prevention policies and programs. Injury information in the...