A Path to Greater Credibility: Large-Scale Collaborative Education Research

This essay is a call for world-systems analysis of education. Increasingly, the field of comparative education is moving toward more sophisticated examinations of education in relationship to economic, political, and social forces. Studies of the ecology of educational institutions and processes, however, often fail to take into account an international context of transactions. To date, most macro studies of education have taken the nation-state as the basic unit of analysis.' An examination of the international forces impinging upon education systems is no less essential than an examination of the international economic order would be to an

The adoption and incentivisation of open and transparent research practices is critical in addressing issues around research reproducibility and research integrity. These practices will require training and funding. Individuals need to be incentivised to adopt open and transparent research practices (e.g., added as desirable criteria in hiring, probation, and promotion decisions, recognition that funded research should be conducted openly and transparently, the importance of publishers mandating the publication of research workflows and appropriately curated data associated with each research output). Similarly, institutions need to be incentivised to encourage the adoption of open and transparent practices by researchers. Research quality should be prioritised over research quantity. As research transparency will look different for different disciplines, there can be no one-size-fits-all approach. An outward looking and joined up UK research strategy is needed that places openness and transparency at the heart of research activity. This should involve key stakeholders (institutions, research organisations, funders, publishers, and Government) and crucially should be focused on action. Failure to do this will have negative consequences not just for UK research, but also for our ability to innovate and subsequently commercialise UK-led discovery.

The scientific community is currently grappling with the “reproducibility crisis,” a phenomenon characterized by a significant loss of validity in research outcomes and a subsequent decline in public and scientific confidence. This crisis is evident in the high rate of studies, particularly within medicine and the humanities that fail replication tests and exhibit poor generalizability. This study aimed to specifically examine the issues of reproducibility in medical education research in Iran and propose effective solutions to address this challenge. This investigation employed a qualitative methodology, utilizing conventional content analysis. Data were collected through in-depth, semi-structured interviews with a carefully selected group of 24 medical science professors. Interviewing continued until data saturation was achieved, with each session lasting between 60 and 90 min. The analysis was systematically performed using the established Graneheim and Lundman’s framework. The analysis led to the identification of three themes: factors contributing to the reproducibility crisis, its consequences, and potential solutions. These themes were further divided into eight categories. Contributing factors encompassed research methodology, various forms of bias, and contextual elements. The consequences were primarily reflected in two domains—scientific progress and decision-making in educational contexts. In contrast, proposed solutions centered on enhancing methodological rigor and reinforcing a culture of accountability and oversight in research practice. According to the findings, improving the quality and credibility of research in medical education requires a comprehensive strategy built upon three key pillars: advancing researchers’ methodological competence, institutionalizing ethical and transparent research practices, and strengthening systems of evaluation and accountability. The effective implementation of initiatives such as the development of clear research guidelines, organization of training and capacity-building workshops, and promotion of robust inter-institutional collaboration can play a vital role in enhancing the quality, reliability, and trustworthiness of medical education research in Iran

Focuses on open science practices in suicide research. Specifically, the authors use examples from their own and others' work to demonstrate the opportunities for future-proofing research by implementing open science practices, and they discuss some of the challenges and their potential solutions. They cover implementing open science practices in new, ongoing, and concluded studies, and discuss practices in order of being 'low' to 'high' threshold to implement (based on Kathawalla et al., 2021). Space constraints preclude an extensive presentation of all open science practices although they briefly describe the strategies of providing preprints, sharing study materials and codes, preregistration of studies and registered reports, and sharing research data. To highlight the open science work of as many researchers as possible, examples are provided in Electronic Supplementary Material 1 (ESM 1) rather than in the text. The authors ask for readers' help in adding to these examples via their 'living' reading list (https://osf.io/v6y3t/).

Cronbach’s alpha remains one of the most widely used reliability coefficients in educational and psychological research. However, its application is often accompanied by misinterpretations and methodological limitations. This systematic review was conducted in accordance with PRISMA 2020 guidelines to examine the use, assumptions, cut-off criteria, and alternatives to Cronbach’s alpha in educational research. A comprehensive search of major academic databases identified 44 eligible studies published between 2000 and 2026, which were critically analyzed based on their relevance, methodological rigor, and contributions to reliability assessment. The review provides key conceptual clarifications, summarizes empirical applications, highlights methodological advancements, and identifies emerging alternatives to Cronbach’s alpha. The findings indicate that although Cronbach’s alpha remains a foundational index of internal consistency, its interpretation must be done with careful consideration of scale dimensionality, sample characteristics, and item properties. The review further emphasizes the importance of reporting Cronbach’s alpha alongside complementary reliability indices, adopting context-specific cut-off criteria, and ensuring transparent and robust estimation procedures. This synthesis offers evidence-based recommendations to enhance reliability assessment practices in educational research.

Previous articleNext article No AccessPresidential AddressCurrents Left and Right: Ideology in Comparative EducationErwin H. EpsteinErwin H. Epstein Search for more articles by this author PDFPDF PLUS Add to favoritesDownload CitationTrack CitationsPermissionsReprints Share onFacebookTwitterLinkedInRedditEmail SectionsMoreDetailsFiguresReferencesCited by Comparative Education Review Volume 27, Number 1Feb., 1983 Sponsored by the Comparative and International Education Society Article DOIhttps://doi.org/10.1086/446343 Views: 36Total views on this site Citations: 30Citations are reported from Crossref Copyright 1983 The Comparative and International Education SocietyPDF download Crossref reports the following articles citing this article:Joseph Zajda, Val Rust Theory in Comparative Education, (Mar 2021): 37–47.https://doi.org/10.1007/978-94-024-2054-8_3Danijela Milošević, Jelena Maksimović, Methodology of Comparative Research in Education: Role and Significance, International Journal of Cognitive Research in Science, Engineering and Education 8, no.33 (Dec 2020): 155–162.https://doi.org/10.23947/2334-8496-2020-8-3-155-162Florin D. Salajan, Tavis D. Jules Exploring Comparative and International Education as a Meta-Assemblage: The (Re)Configuration of an Interdisciplinary Field in the Age of Big Data, (Jun 2020): 133–151.https://doi.org/10.1108/S1479-367920200000039014Maren Elfert, Christine Monaghan Comparative and International Education: A Field Fraught with Contradictions, (Sep 2019): 65–71.https://doi.org/10.1108/S1479-367920190000037007David A. Turner Comparative and International Education: Development of a Field and Its Method and Theory, (Aug 2019): 11–28.https://doi.org/10.1108/S1479-367920190000036002Brian D. Denman Post-worldview? A dialogic meta-narrative analysis of North-South, South-South, and Southern theory, International Journal of Comparative Education and Development 19, no.2/32/3 (May 2017): 65–77.https://doi.org/10.1108/IJCED-05-2017-0007Jose Siles Gonzalez, Carmen Solano Ruiz, Angela Gaban Gutierrez International Appraisal of Nursing Culture and Curricula: A Qualitative Study of Erasmus Students, Scientifica 2016 (Jan 2016): 1–6.https://doi.org/10.1155/2016/6354045C. C. Wolhuter Quisnam Sum Ego? Crises of Identity in Comparative Education and the Call for a Comparison of Comparative Studies, (May 2015): 15–35.https://doi.org/10.1108/S1479-367920140000026002Robert Cowen Comparative education: stones, silences, and siren songs, Comparative Education 50, no.11 (Feb 2014): 3–14.https://doi.org/10.1080/03050068.2013.871834Jürgen Schriewer Vergleichende Erziehungswissenschaft als Forschungsfeld, (Feb 2013): 15–41.https://doi.org/10.1007/978-3-531-18937-6_2Moosung Lee, Tom Friedrich Continuously reaffirmed, subtly accommodated, obviously missing and fallaciously critiqued: ideologies in UNESCO's lifelong learning policy, International Journal of Lifelong Education 30, no.22 (Mar 2011): 151–169.https://doi.org/10.1080/02601370.2010.547619Supriya Baily Trajectories of Influence, (Jan 2011): 217–233.https://doi.org/10.1007/978-94-6091-722-6_12Val D. Rust, Amanda Kenderes Paulston and Paradigms, (Jan 2011): 19–29.https://doi.org/10.1007/978-94-6091-722-6_2Shoko Yamada, Jing Liu Between Epistemology and Research Practices, (Jan 2011): 371–393.https://doi.org/10.1007/978-94-6091-722-6_20Erwin H. Epstein, Katherine T. Carroll Erasing Ancestry, (Jan 2011): 31–48.https://doi.org/10.1007/978-94-6091-722-6_3Moosung Lee, Tom Friedrich Citation Network Analysis of Comparative Education Texts, (Jan 2011): 113–144.https://doi.org/10.1007/978-94-6091-722-6_7Sadullah Dede, Gülsün Atanur Baskan Theoretical basis of comparative education and suggestion of a model: comparative education council in Turkish education system, Procedia - Social and Behavioral Sciences 15 (Jan 2011): 3536–3542.https://doi.org/10.1016/j.sbspro.2011.04.331Erwin H. Epstein Setting the normative boundaries: crucial epistemological benchmarks in comparative education, Comparative Education 44, no.44 (Nov 2008): 373–386.https://doi.org/10.1080/03050060802481405 Erwin H. Epstein and Katherine T. Carroll Abusing Ancestors: Historical Functionalism and the Postmodern Deviation in Comparative Education Epstein and Carroll, Comparative Education Review 49, no.11 (Jul 2015): 62–88.https://doi.org/10.1086/426161 Bradley J. Cook , Steven J. Hite , and Erwin H. Epstein Discerning Trends, Contours, and Boundaries in Comparative Education: A Survey of Comparativists and Their Literature Cook, Hite, and Epstein, Comparative Education Review 48, no.22 (Jul 2015): 123–149.https://doi.org/10.1086/382619TIMOTHY J. MARTIN Divergent Ontologies with Converging Conclusions: A case study comparison of comparative methodologies, Comparative Education 39, no.11 (Feb 2003): 105–117.https://doi.org/10.1080/03050060302558Jürgen Schriewer Stichwort: Internationaler Vergleich in der Erziehungswissenschaft, Zeitschrift für Erziehungswissenschaft 3, no.44 (Dec 2000): 495–515.https://doi.org/10.1007/s11618-000-0050-3ANTHONY R. WELCH The Cult of Efficiency in Education: Comparative reflections on the reality and the rhetoric, Comparative Education 34, no.22 (Jun 2010): 157–175.https://doi.org/10.1080/03050069828252Anthony R. Welch Class, Culture and the State in Comparative Education: problems, perspectives and prospects, Comparative Education 29, no.11 (Jul 2006): 7–27.https://doi.org/10.1080/0305006930290102Michael Crossley, Patricia Broadfoot Comparative and International Research in Education: scope, problems and potential, British Educational Research Journal 18, no.22 (Jan 1992): 99–112.https://doi.org/10.1080/0141192920180201Michael Crossley Curriculum Policy and Practice in Papua New Guinea, Compare: A Journal of Comparative and International Education 20, no.22 (Jan 1990): 141–154.https://doi.org/10.1080/0305792900200204Keith Watson Dependence or independence in education? two cases from post-colonial South-East Asia, International Journal of Educational Development 5, no.22 (Jan 1985): 83–94.https://doi.org/10.1016/0738-0593(85)90016-1William T. Trent, Jomills Henry Braddock, Ronald D. Henderson Chapter 8: Sociology of Education: A Focus on Education as an Institution, Review of Research in Education 12, no.11 (Jun 2016): 295–336.https://doi.org/10.3102/0091732X012001295Traugott Sch�fthaler Multikulturelle Und Transkulturelle Erziehung: Zwei Wege Zu Kosmopolitischen Kulturellen Identit�ten, International Review of Education 30, no.11 (Jan 1984): 11–24.https://doi.org/10.1007/BF00598015Michael Crossley, Graham Vulliamy Case‐Study Research Methods and Comparative Education, Comparative Education 20, no.22 (Aug 2006): 193–207.https://doi.org/10.1080/0305006840200202

ObjectivesTo evaluate (1) the feasibility of an audit-feedback intervention to facilitate sports science journal policy change, (2) the reliability of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) policy evaluation form, and (3) the extent to which policies of sports science journals support transparent and open research practices.MethodsWe conducted a cross-sectional, audit-feedback, feasibility study of transparency and openness standards of the top 38 sports science journals by impact factor. The TRUST form was used to evaluate journal policies support for transparent and open research practices. Feedback was provided to journal editors in the format of a tailored letter. Inter-rater reliability and agreement of the TRUST form was assessed using intraclass correlation coefficients and the standard error of measurement, respectively. Time-based criteria, fidelity of intervention delivery and qualitative feedback were used to determine feasibility.ResultsThe audit-feedback intervention was feasible based on the time taken to rate journals and provide tailored feedback. The mean (SD) score on the TRUST form (range 0–27) was 2.05 (1.99), reflecting low engagement with transparent and open practices. Inter-rater reliability of the overall score of the TRUST form was moderate [ICC (2,1) = 0.68 (95% CI 0.55–0.79)], with standard error of measurement of 1.17. However, some individual items had poor reliability.ConclusionPolicies of the top 38 sports science journals have potential for improved support for transparent and open research practices. The feasible audit-feedback intervention developed here warrants large-scale evaluation as a means to facilitate change in journal policies.Registration: OSF (https://osf.io/d2t4s/).

Randomized controlled trials in educational research tend to be small. Small trials can have large, chance, imbalances in important covariates. For studies with sample sizes greater than 50, chance imbalances can be corrected using analysis of covariance; for small trials, however, statistical power is maximized if the trial is balanced and analysis of covariance is used in the analysis. The aim of the present study was to discuss methods of improving covariate balance in trial design and to demonstrate the method of minimization. Using an exemplar of a cluster or class‐randomized trial with 29 classes, we employed minimization to achieve covariate balance. Minimization achieved good balance on four prognostic variables. Many trialists in education use restricted forms of allocation, including pairing or stratified randomization. These approaches have disadvantages. Another approach rarely used in educational research is minimization. Minimization uses a simple arithmetic algorithm to produce balanced groups across a number of important covariates and should be more widely used in educational and psychological research.

Principles and applications of open science (also referred to as open research or open scholarship) in psychology have emerged in response to growing concerns about the replicability, transparency, reproducibility, and robustness of psychological research alongside global moves to open science in many fields. Our objective in this paper is to inform ways of collectively constructing open science practices and systems that are appropriate to, and get the best out of, the full range of qualitative and mixed‐method approaches used in psychology. We achieve this by describing three areas of open research practice (contributorship, pre‐registration, and open data) and explore how and why qualitative researchers might consider engaging with these in ways that are compatible with a qualitative research paradigm. We argue it is crucial that open research practices do not (even inadvertently) exclude qualitative research, and that qualitative researchers reflect on how we can meaningfully engage with open science in psychology.

Decision letter for "“Don’t Hate the Players, Hate the Game”: Qualitative Insights from Education Researchers on Questionable and Open Research Practices"

Review for "“Don’t Hate the Players, Hate the Game”: Qualitative Insights from Education Researchers on Questionable and Open Research Practices"

The authors evaluate the actual contribution to classroom practice of recent research in citizenship education conducted by two major international organizations: the International Association for the Evaluation of Educational Achievement (IEA) and the National Foundation for Educational Research (NFER). The paper adopts Egan's (2002) critical framework that contends much of the current research in education simply establishes empirical connections between ordinary language concepts that are already conceptually linked. Consistent with Egan's critique, the authors' analysis reveals a recurring pattern of analytic connections between the concept of democratic citizenship and the subsequent recommendations for practice offered in both of these reports.

The traditional research backgrounds of engineering academics do not adequately prepare them to conduct quality education research. Nevertheless, there is institutional pressure on these academics to take a scholarly approach to teaching and conduct education research. What is required is an effective strategy to systematically help engineering academics to innovate, evaluate and publish education research and scholarship. This paper explores one model for structuring activities to help engineering and science academics develop their practices of education research. It is argued that the proposed Model for Action and Learning in Engineering Education Research (MALEER) presented is an effective strategy to prepare engineering academics to conduct quality education research. An initial model was developed, situated in various bodies of literature including communities of practice, professional identity development and becoming a professional, adult learning and reflection in and on action. It was then trialled with an engineering and science education research group set-up to help academics develop their education research practice. Evidence of the success of MALEER was drawn from the reflections of the coordinators of the group and the groups’ developed practices and outcomes.

The traditional research backgrounds of engineering academics do not adequately prepare them to conduct quality education research. Nevertheless, there is institutional pressure on these academics to take a scholarly approach to teaching and conduct education research. What is required is an effective strategy to systematically help engineering academics to innovate, evaluate and publish education research and scholarship. This paper explores one model for structuring activities to help engineering and science academics develop their practices of education research. It is argued that the proposed Model for Action and Learning in Engineering Education Research (MALEER) presented is an effective strategy to prepare engineering academics to conduct quality education research. An initial model was developed, situated in various bodies of literature including communities of practice, professional identity development and becoming a professional, adult learning and reflection in and on action. It was then trialled with an engineering and science education research group set-up to help academics develop their education research practice. Evidence of the success of MALEER was drawn from the reflections of the coordinators of the group and the groups' developed practices and outcomes.

"The Gulf Comparative Education Society (GCES) held its fourth annual symposium under the sponsorship of the Sheikh Saud Bin Saqr Al Qasimi Foundation for Policy Research in collaboration with the Sultan Qaboos University from March 16 to 18, 2013. Entitled “Bridging the Policy/Research Divide in Education in the GCC,” the symposium was held at the Sultan Qaboos University in Muscat, Oman. It consisted of a pre-conference workshop, one keynote address, three featured panels, and 20 breakout sessions with a total of over 70 presentations by both invited speakers as well as those who had submitted abstracts. The speakers came from a wide variety of countries, including United Arab Emirates (UAE), Oman, Bahrain, Qatar, Switzerland, England, Australia, Germany, and the United States, and represented different voices in the education sector, such as policy makers, academics and researchers, school providers and leaders, consultants, and teachers. The purpose of this year’s GCES symposium was to examine the potential for policy makers in the Gulf region to devise evidence-based policies through utilizing local research in the education sector with a focus on the promise and possibilities of comparative education research to policy-making. Delivering the keynote address on the relationship between educational research and policy and practice was Professor Fernando Reimers, Ford Foundation Professor of International Education and Director of the International Education Policy Program at the Harvard Graduate School of Education. The featured panels and breakout sessions addressed the following topics: • Linking Research to Educational Policy • Implications of Technology in the Classroom • Lifelong Learning • Applying Transnational Models of Education • Evaluating and Transforming Educational Systems • Regional and International Youth Challenges • Improving Educational Quality • Research in Higher Education in Oman • Student Leadership and Development • Promoting an Education Research Culture • Perspectives on Teaching and Learning • Trends and Innovations in Education In addition, the symposium brought together over 400 participants working in a range of organizations across Oman, the Gulf States, and beyond, all of whom shared an interest in comparative education in the GCC. Following the symposium, presenters were asked if they would like to submit a 1,500 - 3,000 word paper on their presentation. This volume is the compilation of those papers that were submitted. While it does not cover all of the presentations that were made at the symposium, presentation slides for all the presentations are available on the GCES website (www.gulfcomped.ning.com). Dr. Hamood Al-Harthi - President Dr. Christina Gitsaki - Vice President Dr. Natasha Ridge and Soha Shami - Secretary Verena Woeppel - Proceedings Editor"