Adoption of a design-based research approach to improve understanding about complex educational problems

Editor’s note: The opinions expressed in this editorial do not necessarily reflect the opinions of the AAMC or its members. This issue of Academic Medicine is devoted to an exciting set of research reports, reviews, and commentaries selected for publication from more than 60 submissions as part of our ongoing collaboration with the Research In Medical Education (RIME) program.1–17 The topics are wide-ranging—experiences of moral distress among trainees, interprofessional team leadership, assessment and feedback during training, and the role of arts and humanities in professional identity formation, to name just a few. As in the past, each of these published articles will be presented and discussed as part of the RIME program at Learn Serve Lead 2024: The AAMC Annual Meeting, which takes place this month. RIME’s work highlights creativity and quality in education scholarship and communicates the importance of medical education inquiry to stakeholders throughout academic medicine, such as institutional leaders, promotions committee members, and learners. RIME articles elevate issues of concern to medical education in a timely manner. Such attention can lead to dialogue, consensus, and advances in the field. RIME also supports the professional development of medical education scholars. Manuscripts submitted for consideration by the RIME Program Planning Committee receive written feedback from reviewers and committee members, and service on the RIME committee itself is a valuable learning experience and opportunity for leadership in medical education and academic medicine. In celebrating RIME’s 60th anniversary in 2021, members of the RIME Program Planning Committee conducted a survey of past RIME chairs to better understand how RIME has helped advance medical education research.18 The past chairs recognized RIME as a catalyst for medical education research, strengthening the rigor and impact of medical education research, developing a community of scholars, and supporting academic career development. They also identified as priorities the need for greater innovation and translation of medical education research to practice. They welcomed new activities and collaborations, such as faculty mentorship initiatives and support for educators pursuing advanced degrees in education.18 Each year leaders on the RIME committee prepare a commentary, published as part of the RIME collection, that addresses a way to improve medical education research in academic medicine. In 2012, RIME leaders called for a more intentional, strategic, and inclusive effort. Medical education research requires an intellectual structure, a framework into which seemingly random studies can come together to improve learning and practice. The challenge before us is to look forward and develop a well-laid plan to further the field of medical education research nationally and internationally.19 For this year’s commentary, Sukhera and colleagues1 similarly challenge the field of academic medicine to do work that will improve learning and practice. They encourage medical education researchers to think beyond rigor, relevance, and utility in their work and to find ways to generate new knowledge that is readily—and will actually be—used. Drawing upon insights from implementation science, the authors call upon medical education researchers to mobilize toward actionable, scalable, impactful scholarship. Over the years, Academic Medicine has partnered with RIME to publish 30 journal supplements; RIME collections have also appeared in our regular issues in October 2012, October 2013, and November 2014. The editorial teams of Academic Medicine and our AAMC sister journal, MedEdPORTAL, have also been fortunate to benefit from the service of members of the RIME community, such as manuscript authors, creators of curricular and education resources, and expert reviewers. Authors of RIME articles have been featured on the Academic Medicine Podcast, which further increases the reach of the scholarly contributions. Academic Medicine also was pleased to collaborate with the leadership of RIME in 2022 when it launched its mentorship program for early and underrepresented faculty scholars, a program that has evolved to become MedEdSCHOLAR.20 This year, we are pleased to feature the RIME articles and commentaries in this issue of Academic Medicine. This decision reflects our continued strong support for the work of the RIME planning committee, RIME authors, and the broader RIME community in furthering medical education scholarship.

Medical education research in general is a young scientific discipline which is still finding its own position in the scientific range. It is rooted in both the biomedical sciences and the social sciences, each with their own scientific language. A more unique feature of medical education (and assessment) research is that it has to be both locally and internationally relevant. This is not always easy and sometimes leads to purely ideographic descriptions of an assessment procedure with insufficient general lessons or generalised scientific knowledge being generated or vice versa. For medical educational research, a plethora of methodologies is available to cater to many different research questions. This article contains consensus positions and suggestions on various elements of medical education (assessment) research. Overarching is the position that without a good theoretical underpinning and good knowledge of the existing literature, good research and sound conclusions are impossible to produce, and that there is no inherently superior methodology, but that the best methodology is the one most suited to answer the research question unambiguously. Although the positions should not be perceived as dogmas, they should be taken as very serious recommendations. Topics covered are: types of research, theoretical frameworks, designs and methodologies, instrument properties or psychometrics, costs/acceptability, ethics, infrastructure and support.

Likert-type scales are frequently used in medical education and medical education research. Common uses include end-of-rotation trainee feedback, faculty evaluations of trainees, and assessment of performance after an educational intervention. A sizable percentage of the educational research manuscripts submitted to the Journal of Graduate Medical Education employ a Likert scale for part or all of the outcome assessments. Thus, understanding the interpretation and analysis of data derived from Likert scales is imperative for those working in medical education and education research. The goal of this article is to provide readers who do not have extensive statistics background with the basics needed to understand these concepts. Developed in 1932 by Rensis Likert1 to measure attitudes, the typical Likert scale is a 5- or 7-point ordinal scale used by respondents to rate the degree to which they agree or disagree with a statement (table). In an ordinal scale, responses can be rated or ranked, but the distance between responses is not measurable. Thus, the differences between “always,” “often,” and “sometimes” on a frequency response Likert scale are not necessarily equal. In other words, one cannot assume that the difference between responses is equidistant even though the numbers assigned to those responses are. This is in contrast to interval data, in which the difference between responses can be calculated and the numbers do refer to a measureable “something.” An example of interval data would be numbers of procedures done per resident: a score of 3 means the resident has conducted 3 procedures. Interestingly, with computer technology, survey designers can create continuous measure scales that do provide interval responses as an alternative to a Likert scale. The various continuous measures for pain are well-known examples of this (figure 1). FIGURE 1 Continuous Measure Example TABLE Typical Likert Scales

The field of graduate medical education (GME) research is attracting increased attention and broader participation. The authors review the special ethical and methodological considerations pertaining to medical education research. Because residents are at once a convenient and captive study population, a risk of coercion exists, making the provision of consent important. The role of the institutional review board (IRB) is often difficult to discern because GME activities can have multiple simultaneous purposes, educational activities may go forward with or without a research component, and the subjects of educational research studies are not patients. The authors provide a road map for researchers with regard to research oversight by the IRB and also address issues related to research quality. The matters of whether educational research studies should have educational value for the study subject and whether to use individual information obtained when residents participate as research subjects are explored.

Higher education in Ecuador face new challenges related to improving the quality of teaching and innovation in students´learning. One of the challenges is to integrate research with teaching activities, academic management, and community engagement. The Design-Based Research (DBR) approach becomes key and strategic for effective education and it allows staff to join an interdisciplinary team and influence their immediate context to improve their teaching practice, ensure the quality of the curriculum design, develop contextualized theories about the educational process. Based on a review of the literature, academic manuscripts of design-based research generated by Ecuadorian researchers and studies conducted in Ecuadorian educational contexts in different sources of information -such as repositories of Ecuadorian universities- in the last six years will be selected. In this way, it is proposed that Design-Based Research provides the fundamental epistemological and methodological horizons for faculty to integrate research with curriculum design and teaching practice. Thus, educational research will be more concrete and applicable in the immediate contexts of the professor for their improvement and innovation, and in turn, the practice itself will reformulate the theory that underpins educational action.

This paper reports on a longitudinal, design-based research (DBR) study to promote clinical decision making using a virtual patient (VP) simulation for emergency renal care. The VP was piloted with pharmacy students, then offered as an interprofessional learning exercise for pharmacy and medical students, before being introduced as part of the curriculum. In this paper, the DBR framework used to design, implement and evaluate the VP is described. The iterative changes made and implications for the integration of virtual patient simulation in the pharmacy curriculum are discussed.

This study draws on the application of sociocultural theory to second-language learning and teaching to examine the impact of a design-based research approach on teacher development and literacy instruction to English-language learners (ELLs). Design-based research methodology was employed to derive theoretical suppositions relating to the process of learning as well as the means by which this process is supported. Our research questions were: (a) How will this professional development model result in shifts in teacher thinking about language and literacy learning for ELLs; (b) what innovations in teachers' repertoires of practice will be developed; and (c) in what ways will these shifts in teachers' thinking and innovations in their repertoire of practice bring about new forms of language and literacy learning? Our findings point to the need to place development in the forefront of teacher professional development models. Also foregrounded is the importance of promoting teachers' critical reflection on classroom practices and of creating hypotheses for pedagogical change vis-à-vis new understanding about students' linguistic, cognitive and academic needs.

Objective and Aim The growing emphasis on accountability, quality assurance, and outcomes-based education has intensified the demand for robust and critically appraised evidence in medical education. While the principles of evidence-based medicine have substantially shaped clinical decision-making, their direct transfer to educational contexts has proven conceptually and methodologically challenging. Objective: This systematic review aims to synthesize the literature on evidence types, critical appraisal frameworks, and methodological quality standards in medical education research, and to present an integrative appraisal matrix suitable for educational decision-making. Methods A systematic review was conducted in accordance with PRISMA 2020 guidelines (7). Searches were performed in MEDLINE (PubMed), ERIC, Scopus, and Web of Science for studies published between 2000 and 2024. Included studies addressed evidence hierarchies, critical appraisal, research quality instruments, or methodological frameworks in medical or health professions education. Data were synthesized narratively due to methodological heterogeneity. Results Eighty-four studies met inclusion criteria. Evidence in medical education was categorized into quantitative, qualitative, mixed-methods, and design-based research. Traditional biomedical hierarchies were insufficient to capture educational relevance and contextual complexity. Appraisal tools such as BEME, MERSQI, CASP, and realist review frameworks were commonly cited, yet inconsistently applied. Conclusions Critical appraisal in medical education requires a pluralistic, theory-informed, and context-sensitive approach. The proposed appraisal matrix integrates methodological rigor, educational relevance, outcomes meaningfulness, and transferability to support informed educational practice. Keywords: Medical education research; Systematic review; Critical appraisal; Evidence hierarchy; Research methodology

This paper discusses the potential of language centres in higher education as laboratories for innovation. Despite their hybrid character as central service providers that are also defined by their teaching, their positioning at the academic periphery, and the lack of a mandate to do academic research, language centres still have an obligation to conduct research as a means of promoting innovation. The design-based research (DBR) approach in learning and teaching, in which teaching stakeholders examine learning processes in real situation, offers a suitable opportunity for language centres to overcome the restrictions placed upon them. DBR can be done using existing resources and the innovation potential of a language centre’s teaching staff. The leadership and management of the language centre must be made responsible for providing and/or enabling the necessary structural, material, and cultural conditions for such research. This paper discusses the DBR approach at language centres in higher education and its impact on leadership. It then gives two examples of the successful implementation of DBR at the Language Center of UZH and ETH Zurich.

Due to the characteristics of educational virtual worlds, the present manuscript underlines the need for a research model that considers the social context as part of its unit of analysis instead of just the individual’s cognitive process and learning. It is proposed that such a research approach could be design-based research (DBR), because the methodology employed by the DBR perspective thoroughly meets the challenges related to understanding how learning occurs inside a complex context of activities and interactions like those that usually take place inside an educational virtual environment. To accomplish this, the DBR employs an iterative methodology, which consists of repeating cycles of design, implementation, analysis, and redesign. This systematic procedure allows theory to emerge during the process and, thus, using DBR a researcher not only understands how to improve the quality of a certain virtual world, but also addresses theoretical issues regarding the theoretical background on which her design was based to revise and extend it. The main objective of this article is to propose the use of design-based research as a viable methodology to do research in a virtual world like Second Life and to describe in detail how to do it. First, the design-based research approach is explained in terms of its origins, its methodological resources, and its theoretical underpinnings. Secondly, considering the characteristics and affordances of virtual worlds, an adaptation is proposed and explained to employ it for a virtual world research project. Finally, an example of a research project is built to show how the proposed design-based research methodology can be applied to plan it and revise its underlying theory.

BackgroundMedical education research plays a critical role in improving patient care and advancing medical practices, yet the engagement among healthcare professionals remains limited. This study aims to explore the factors that influence participation in medical education research and provide recommendations to foster a research-inclusive culture within healthcare institutions.MethodsA comprehensive literature review was conducted across four databases: Web of Science, MEDLINE, Scopus, and Airiti Library. The search focused on studies published between 2002 and 2024, utilizing keywords related to ‘medical education research,’ facilitate,’ ‘engage,’ and ‘involve’. The initial search yielded 814 records. After removing duplicates and conducting a thorough screening process based on predefined inclusion and exclusion criteria, eight studies were selected for in-depth analysis. These studies were evaluated to identify key facilitators and barriers that influence research participation among healthcare professionals.ResultsThe analysis, guided by the Ecological Model, revealed a complex interplay of facilitators and barriers affecting research engagement. At the individual level, opportunities for professional development and career advancement emerged as significant facilitators. The interpersonal level highlighted the importance of leadership, mentorship, and peer support in fostering engagement. The environmental level, including institutional policies, funding availability, and protected research time, were identified as critical influences on participation.ConclusionsThe findings suggest that a holistic approach is needed to promote participation in medical education research, addressing factors at the individual, interpersonal and environmental levels. Recommendations include developing targeted strategies to improve intrinsic motivation among healthcare professionals, strengthening mentorship and peer support programs, and implementing supportive institutional policies that allocate dedicated research time and secure financial resources. By fostering a culture that values and facilitates medical education research, healthcare institutions can drive innovation in medical education, ultimately enhancing the quality of patient care and safety.

Purpose: The purpose of this study is to investigate the effect of science activities prepared through the design-based research (DBR) method on middle school students' creative and critical thinking skills. Design/Methodology/Approach: A quasi-experimental design was employed. The study group consisted of 80 middle school students in Kastamonu, Turkey, randomly assigned to experimental and control groups. Design-based science activities were applied in the experimental group, while the control group followed traditional science instruction. Data were collected through pre-tests and post-tests using the Marmara Creative Thinking Dispositions Scale and Marmara Critical Thinking Dispositions Scale. Findings: Results indicated significant improvements in creative and critical thinking dispositions of students in the experimental group compared to the control group. The DBR activities provided students with active problem-solving opportunities, enhancing their ability to generate creative ideas and critically evaluate their work. Highlights: The study highlights that the design-based research approach in science education is more effective than traditional teaching methods in enhancing creative and critical thinking skills. It is recommended for educators and curriculum developers to integrate DBR-driven activities into educational programs.

Medical education researchers are inherently collaborators. This paper presents a discussion of theoretical frameworks, issues and challenges around collaborative research to prepare medical education researchers to enter into successful collaborations. It gives emphasis to the conceptual issues associated with collaborative research and applies these to medical education research. Although not a systematic literature review, the paper provides a rich discussion of issues which medical education researchers might consider when undertaking collaborative studies. Building on the work of others, we have classified collaborative research in three dimensions according to: the number of administrative units represented; the number of academic fields present, and the manner in which knowledge is created. Although some literature on collaboration focuses on the more traditional positivist perspective and emphasises outcomes, other literature comes from the constructivist framework, in which research is not driven by hypotheses and the approaches emphasised, but by the interaction between investigator and subject. Collaborations are more effective when participants overtly clarify their motivations, values, definitions of appropriate data and accepted methodologies. These should be agreed upon prior to commencing a study. The way we currently educate researchers should be restructured if we want them to be able to undertake interdisciplinary research. Despite calls for researchers to be educated differently, most training programmes for developing researchers have demonstrated a limited, if not contrary, response to these calls. Collaborative research in medical education should be driven by the problem being investigated, by the new knowledge gained and by the interpersonal interactions that may be achieved. Success rests on recognising that many of the research problems we, as medical educators, address are fundamentally interdisciplinary in nature. This represents a transition to bridge the dichotomy often presented in medical education between theory building and addressing practical needs.

Medical education is a rapidly growing field of research, incorporating diverse disciplinary perspectives to assist physician trainees in developing the complex skills needed for practice. Education science is happening in many medical specialties; however, Family Medicine or General Practice settings have not seen a proportional share of theory-driven education research. The limited nature of education research in Family Medicine is surprising, given that there are several aspects of general practice that make it a particularly unique and interesting context to study issues of general importance to medical education, and there is a particular need for education research to further the discipline of Family Medicine. It is important that the community of medical education researchers in Family Medicine have a strong understanding and perspective on the breadth and potential impact of their work, and what this means for the training that occurs within and for the discipline. This Reflection aims to inform strategic thinking, collaboration, and innovation in medical education research as it pertains to Family Medicine. It does so by discussing four hallmarks of Family Medicine practice and outlining their independent and interactive potential for medical education research.

Medical education research is underfunded. Little research focuses on basic issues of learning and teaching in medicine, and few educational innovations are tested across institutions to insure their generalizability. Despite repeated calls for the development of research organizations that would deal with substantial issues in medical education, funding remains a miniscule portion of the total budget for medical education. The author discusses how medical education research is supported in the present economic environment. The majority of published research is accomplished during the course of the researcher's employment. Many universities support pilot studies that lead to larger grants. Faculty development programs and career or academic awards support some research. Small to moderate amounts of funding are available through a variety of sources including medical specialty associations and private foundations. The author lists a variety of resources and strategies for finding foundations to fund research. Funding for larger research projects, particularly those that require more than a few years to accomplish or that involve multiple institutions, is more difficult to obtain. Federal training grants provide the laboratory for innovation and education research, but include inadequate funds to accomplish in-depth evaluation or research. The lack of emphasis on theory building and failure to prove the usefulness of innovations across schools cannot be overcome with current levels of funding. Medical educators must be advocates for the research support needed to move medical education research from a field with limited influence on practice to one that is able to connect sound insights and ideas with actual practices in education and improved patient health.