Randomised Controlled Trials (RCTs) in education research – methodological debates, questions, challenges

Randomized controlled trials in educational research tend to be small. Small trials can have large, chance, imbalances in important covariates. For studies with sample sizes greater than 50, chance imbalances can be corrected using analysis of covariance; for small trials, however, statistical power is maximized if the trial is balanced and analysis of covariance is used in the analysis. The aim of the present study was to discuss methods of improving covariate balance in trial design and to demonstrate the method of minimization. Using an exemplar of a cluster or class‐randomized trial with 29 classes, we employed minimization to achieve covariate balance. Minimization achieved good balance on four prognostic variables. Many trialists in education use restricted forms of allocation, including pairing or stratified randomization. These approaches have disadvantages. Another approach rarely used in educational research is minimization. Minimization uses a simple arithmetic algorithm to produce balanced groups across a number of important covariates and should be more widely used in educational and psychological research.

Randomised controlled trials represent the gold standard in intervention efficacy evaluation. However, suboptimal recruitment affects completion and the power of a therapeutic trial in detecting treatment differences. We conducted a systematic review to examine the barriers and enablers to patient recruitment for randomised controlled trials on chronic wound treatment. Review registration was under PROSPERO 2017:CRD42017062438. We conducted a systematic search of Ovid MEDLINE, EBSCOhost CINAHL, Ovid Cochrane Library, Ovid EMBASE, and Ovid PsycINFO databases in June 2017 for chronic wound treatment randomised controlled trials. Twenty-seven randomised controlled trials or qualitative studies met the inclusion criteria. Among the 24 randomised controlled trials, 21 were assessed as low quality in relation to recruitment, and 3 were assessed as high quality. All 27 studies reported barriers to recruitment in chronic wound randomised controlled trials. The reported barriers to recruitment were: study-related, patient-related, clinician-related, health system-related, and/or operational-related. No study reported recruitment enablers. To enhance randomised controlled trial recruitment, we propose the need for improved integration of research and clinical practice. To alleviate the problems arising from inadequate reporting of randomised controlled trials, the Consolidated Standards of Reporting Trials Statement could include an additional item on recruitment barriers. This approach will allow for increased awareness of the potential barriers to recruitment for Randomised controlled trials (RCTs) in both wound management and other health care research.

More nursing journals are recommending authors follow specific reporting guidelines when preparing manuscripts for submission to the journal. These guidelines indicate how to report different types of research studies, systematic reviews, and quality improvement (QI) projects, among others, to ensure they include critical information and are of high quality. Authors disseminate the findings of their research through the literature: unclear and poor reporting can be misleading and prohibit readers from accurately evaluating the study methodology and findings. Readers need clear, complete, and transparent information in published articles, and the guidelines serve an important role in specifying the content to include. The CONSORT guidelines were developed to improve the reporting of randomized controlled trials (RCTs). They include a checklist of essential items to include in the research report and a diagram for documenting the flow of participants through the trial (Schulz et al. 2010). Templates of the checklist and flow diagram are available to download, and there are other resources for authors at the CONSORT website at http://www.consort-statement.org/home/. Although the guidelines were intended for reporting RCTs, they are useful to authors when preparing manuscripts about other types of quantitative studies. Foster (2012) suggested that authors use the relevant items on the checklist as prompts for information to include in their research reports. The PRISMA statement includes items that are essential for reporting systematic reviews and metaanalyses. It is an update and expansion of the earlier QUOROM statement. Similar to CONSORT, PRISMA includes a checklist that identifies content to report and a diagram that shows the flow of information through the different phases of the review. With the diagram you can map out the number of records identified, included, and excluded in the review, with the reasons for the exclusions. Authors can download templates of the PRISMA checklist and flow diagram for use in conducting the systematic review and reporting the results. Although developed for systematic reviews and meta-analyses, PRISMA guidelines are valuable for reporting literature reviews. Prior to beginning the review, regardless of its type, authors should check the information that will be needed when reporting the results to ensure this information is retained during the review process. For example, you will need to report the databases searched, search strategy including any limits used, number of records screened, number excluded, and so forth. The checklist and flow diagram are useful tools for conducting the reviews and recording information that will be needed eventually for a manuscript. The PRISMA statement and resources are available at http://www.prisma-statement.org/. The SQUIRE guidelines are intended to help authors write usable papers about QI studies in health care. They ensure that the reports of a QI study are sufficiently comprehensive for readers to understand the problem, intervention, setting, and outcomes (Oermann 2009). Although developed for better reporting of QI, the SQUIRE guidelines can be used in planning a study that can eventually be published. There is a checklist of content to include in the paper that can be downloaded. Other resources to assist authors in reporting QI studies and projects are available at the SQUIRE website at http://squire-statement.org/. The STROBE guidelines are for conducting and reporting observational studies. The checklists of items in STROBE are recommended for disseminating the findings of cohort, case-control, and cross-sectional studies. They can be downloaded from http://www.strobe-statement.org/index.php?id=strobe-home. The EQUATOR Network (http://www.equator-network.org/home/) is an initiative to promote accurate and transparent reporting of health care research. The website includes an excellent library of resources for nurse authors, editors, and peer reviewers, including a comprehensive list of reporting guidelines organized by study type (http://www.equator-network.org/resource-centre/library-of-health-research-reporting/). While I reported four guidelines in this editorial, at the EQUATOR website there are many others listed to guide planning the study and then reporting the findings and outcomes. Links are provided to the guidelines for easy access by authors. There also is a resource center at the website with materials for authors, for editors such as links to developing a journal's policies on research reporting, and for peer reviewers. Courses and e-learning opportunities are also available. Reporting guidelines specify the elements needed when preparing manuscripts on different types of research studies, systematic reviews and meta-analyses, QI projects, and others to ensure a clear and transparent report. They provide a framework for nurse authors to use when writing these manuscripts and a checklist for including essential information in them. Studies of the medical literature suggest that journal endorsement of the CONSORT guidelines appears to improve the quality of the reporting of RCTs (Hopewell et al. 2012, Turner et al. 2012). In one of the few studies in nursing, Polit, Gillespie, and Griffin (2011) found that the use or mention of blinding (a strategy in RCTs to reduce bias from knowing who is receiving the intervention) correlated strongly to whether the journal endorsed CONSORT. The authors recommended that nurse authors reporting RCTs be more diligent in following those guidelines. Nurse authors should be familiar with the various guidelines described in this editorial. Authors can use them when planning a study to be sure they have considered important elements and recorded information they will need later when preparing their manuscripts. If a journal endorses specific guidelines such as CONSORT for reporting a RCT or SQUIRE for a QI paper, then follow those guidelines. For journals that do not explicitly endorse these guidelines, authors should be familiar with the standards and consider them when outlining the content for their papers to ensure they include clear and complete information for readers. The guidelines also may provide a framework for organizing the paper and headings to use in it. Editors should consider which guidelines, if any, might improve the quality of manuscripts submitted to their journals and also the peer reviews of those papers. As these guidelines were not developed for nursing, we need to explore whether they are the most appropriate ones for our use and if not to develop guidelines for improved reporting in the nursing literature. Marilyn H. Oermann, PhD, RN, ANEF, FAAN, is Director of Evaluation and Educational Research at Duke University School of Nursing, Durham, North Carolina, USA. She is the editor of Nurse Author & Editor and editor of the Journal of Nursing Care Quality. Her email is moermann@msn.com.

An introduction to Chuna manual medicine in Korea: History, insurance coverage, education, and clinical research in Korean literature

The purpose of this book is to provide a synthesis of thought and practice in research in literacy and science education intended to lead to evidence-based results and generalizations that will serve as a foundation for public policy and informed curriculum, teaching, and assessment practices in education. The Gold Standard of educational research funding in the United States can be viewed as a response to the general dissatisfaction with the utility of educational research; this federal mandate fosters a shift of educational research toward positivist empirical research approaches, such as random controlled trials (RCTs). There is an expectation of greater generalization and policy relevance as Gold Standard research is conducted and reported. It should be noted this dissatisfaction is not confined to the United States. An international response to this dissatisfaction with educational research—systematic reviews of educational research such as the Campbell Collaboration (Campbell Collaboration, n.d.), the What Works Clearinghouse of the US Office of Education (US Institute of Education Sciences, n.d.), and the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) in the United Kingdom (EPPI-Centre, n.d.)— has also identified the general dearth of rigorous empirical research that can support meaningful generalization of research findings. The deliberations about qualitative–quantitative approaches to educational research over the last 25 years have established parallels for quantitative data considerations (reliability, validity, significance, objectivity) that consider dependability, credibility, believability, and confirmability of information (Howe & Eisenhart, 1990; Husen, 1988; Lather, 1992; Phillips, 2005; Pring, 2000). The enactment of these considerations has produced a diverse collection of qualitative, quantitative, and mixed-methods research in literacy and science education. Inspection of these research studies during the 2nd Island Conference revealed concerns about the clarity

Evidence-based pharmacy practice requires a dependable evidence base. Randomised controlled trials (RCTs) are the gold standard of high-quality primary research, and tools exist to assist researchers in conducting and reporting high-quality RCTs. This review aimed to explore whether RCTs relevant to pharmacy are conducted and reported in line with Cochrane risk of bias and CONSORT standards, respectively. A MEDLINE search identified potential papers. After screening of titles, abstracts and full texts, the 50 most recent papers were reviewed and assessment of bias according to Cochrane domains and compliance with CONSORT checklist items was recorded. Each domain of the Cochrane tool and CONSORT checklist item and each article were given a percentage score, reported as median and interquartile range (IQR). Correlation between quality of conduct, quality of reporting, continent of origin, and journal impact factor was conducted using the R2 statistic. The median domain score for risk of bias by paper according to the Cochrane risk of bias tool was 53.0% (IQR 38.5-68.5), while the median compliance score by paper for the CONSORT checklist was 64.0% (IQR 36.0-94.0%). The median Cochrane domain and median CONSORT item completion scores, respectively, were 50.0% (IQR 33.3-66.7%) and 59.5% (IQR 52.0-70.3%). The highest risk of bias was associated with allocation concealment and blinding, and the least well-reported items were randomisation details, sequence generation and allocation concealment. A positive relationship between conduct and reporting of RCTs was found (R2 =0.75), while no correlation was found between quality of conduct or quality of reporting and journal impact factor, correlation coefficients (R2 =0.06 and R2 =0.05, respectively). This review identified that issues related to randomisation and blinding are often inadequately conducted or not comprehensively reported by researchers conducting pharmacy relevant RCTs, providing useful information for education and future research.

BackgroundRandomised controlled trials (RCTs) are the gold standard of evidence-based practice in medicine but they have had limited influence in the field of intellectual disabilities. Previous literature suggests that participants and professionals have limited tolerance for this type of research methodology. However, it is not known how well service users, carers and other health professionals understand and accept the need for RCTs, and why it is important for individuals with intellectual disabilities to be included in this kind of research.MethodsWe examined individual perceptions of RCTs in 51 participants (18 carers, 6 service users and 27 professionals) using semi-structured interviews. A framework approach was adopted in the analysis of data.ResultsWe found that participants had concerns about capacity and resource allocation but held positive views towards this type of research methodology. Understanding of the principles behind RCTs was poor amongst service users and a minority of carers, but mediated by previous exposure to research for professionals.ConclusionsThe social validity of RCTs in intellectual disabilities may be compromised by lack of understanding of the design and the on-going concerns about obtaining informed consent especially in incapacitated adults. However, the overall finding that the need for this form of research was seen in a positive light suggests that there is a turning point in the perceptions of stakeholders working in intellectual disabilities services. We recommend that researchers include on-going education on RCT design during trials, tailoring it to all stakeholders with emphasis on strong service user and care involvement. This could be a pivotal element in improving acceptability of, and recruitment to RCTs.

Because psychiatry residents have important roles as teachers and significant opportunities to contribute to medical student education, we set out to: identify all randomized control trials (RCT) for residents' teaching skills programs in psychiatry and to identify the efficacy of those interventions for improving teaching skills; identify the strengths and weaknesses of the available studies across medical disciplines; and identify currently available methods for enhancing residents' teaching skills for residents training in psychiatry. The published English-language literature was searched using PubMed, Social Sciences Index, and PsycINFO databases, with key search words including: residents, teaching skills, residents as teachers, psychiatry, and assessments. Both RCT and controlled, nonrandomized trials of residents' teaching programs directed to enhance residents' teaching skills were selected and critically appraised. Of 13 trials identified and reviewed, most included residents in internal medicine. Only one included psychiatry residents and assessed their ability to teach interviewing skills to medical students. Along with other studies, this study demonstrated improvement in residents' teaching skills. Overall, interventions and outcome measures were heterogeneous while the quality of methodologies varied. Five studies were of higher quality, representing examples of quality educational research. Several described group differences, blinding, good follow-up, and use of valid, reliable tools. Only one trial exists that incorporated psychiatry residents. Significant opportunity to advance educational research in this field exists. Psychiatry residency program directors should incorporate high-quality methodologies and can benefit from the findings of trials in other disciplines.

Randomized control trials (RCTs) are the gold standard in scientific research. Keeping participants actively involved throughout the RCT is critical to success, yet little is published with regard to retention strategies. Thus, this abstract describes the strategies used in the 18‐month HomeStyles childhood obesity prevention RCT to retain parents of preschoolers. Retention strategies included 1) high‐quality, attractive, quick and easy to use intervention materials (i.e., guides) responsive to the wants and needs of parents of preschoolers in written in Spanish and English; 2) flexible 24/7 participation opportunities (e.g., web delivery; online data collection) and implementing strategies to minimize participant burden; 3) elements of surprise, such as sending fun, unexpected guide‐related enhancements (e.g., beach ball, hula hoop) and holiday or reminder cards in eye‐catching mylar envelopes; 4) program identity branding of all RCT materials using the colorful HomeStyles logo to remind parents about the project each time they noticed project materials in their homes or received electronic or mailed items; 5) financial stipends in the form of eGift cards to participants’ favorite stores immediately after completion of RCT questionnaires and opportunities to receive earn extra stipends (e.g., bonus bucks for visiting the website frequently, submitting supplementary data including height, weight, and pedometer readings); 6) consistent contact methods in the form of frequent, friendly, positive multi‐mode communication including messages in the form of encouraging and motivating nudges sent every ~4 days via email, SMS, and/or voice mail, and protocols to reach out to participants by phone, email, and mail as soon as they showed signs of inactivity; 7) immediacy, ease, and consistency, such as making communication with the RCT team easy (toll‐free phone, email) and replying rapidly to participants queries by trained staff using uniform procedures and consistent messaging; 8) English‐ and Spanish‐speaking study personnel; 9) effective research management with trained RCT team members, established protocols for all RCT procedures, frequent monitoring of participant activity and rapid application of retention procedures; 10) providing an equivalent control group intervention (Home Safety) and not labeling this group as “control”; 11) nonfinancial incentives encouraging participants to make small changes towards their goal to maintain confidence and manage stress and reminding parents to keep the goal of an even happier, healthier family at the forefront as they worked to make improvements; 12) ability to easily update contact information each time participants logged on to the project website. The retention procedures used in this RCT have the potential to improve RCT retention in a broad range of nutrition education research studies.Support or Funding InformationUSDA NIFA #2011‐68001‐30170

Background: Since the introduction of randomized controlled trials (RCT) in clinical research, there has been discussion of whether enrolled patients have worse or better outcomes than comparable non-participants. Objective: To investigate whether very preterm infants randomized to a placebo group in an RCT have equivalent neurodevelopmental outcomes to infants who were eligible but not randomized (eligible NR). Methods: In the course of an RCT investigating the neuroprotective effect of early high-dose erythropoietin on the neurodevelopment of very preterm infants, the outcome data of 72 infants randomized to placebo were retrospectively compared with those of 108 eligible NR infants. Our primary outcome measures were the mental (MDI) and psychomotor (PDI) developmental indices of the Bayley Scales of Infant Development II at 24 months of corrected age. The outcomes of the two groups were considered equivalent if the confidence intervals (CIs) of their mean differences fitted within our ±5-point margin of equivalence. Results: Except for a higher socioeconomic status of the trial participants, both groups were balanced for most perinatal variables. The mean difference (90% CI) between the eligible NR and the placebo group was -2.1 (-6.1 and 1.9) points for the MDI and -0.8 (-4.2 and 2.5) points for the PDI. After adjusting for the socioeconomic status, maternal age and child age at follow-up, the mean difference for the MDI was -0.5 (-4.3 and 3.4) points. Conclusions: Our results indicate that the participation of very preterm infants in an RCT is associated with equivalent long-term outcomes compared to non-participating infants.

In an attempt to improve the quality of educational research, the US Department of Education's Institute of Education Sciences has provided funding for 65 randomized controlled trials of educational interventions. We argue that this research methodology is more effective in providing guidance to extremely troubled schools about how to make some progress than guidance to schools trying to move from making some progress to becoming high-performance organizations. We also argue that the conventional view of medical research—discoveries made in specialized laboratories that are then tested using randomized control trials—is an inaccurate description of the sources of advances in medical practice. Moreover, this conventional view of the sources of advances in medical practice leads to incorrect inferences about how to improve educational research. We illustrate this argument using evidence from the history of medical research on the treatment of cystic fibrosis.

In the increasingly diverse and global world where young people and families struggle to find themselves optimal educational opportunities in schools, homes, and communities, researchers and practitioners have been hard-pressed to offer “evidence-based policy and decision-making” for accountability and efficacy of educational outcomes (e.g., National Institute for Educational Policy Research, 2012; Pascarella, Wolniak, Seifert, Cruce, & Blaich, 2005; Wandersman, Alia, Cook, Hsu, & Ramaswamy, 2016). In addition, the cultural landscape of our society has been constantly changing because of globalization and sociopolitical climates in the past decade or so. Much of the impetus for evidence-based practice and policy in education reflects our societal or governmental pressures, as well as the long tradition of objectivism in educational research (cf. Campbell, & Stanley, 1966), the preeminence of randomized controlled trials (RCT) in medicine, the age-old intuition, or the Divine truth (Bridges, Meyers, & Smith, 2009).

Well-designed randomized controlled trials (RCTs) provide the best evidence of the primary research designs for evaluating the effectiveness of interventions. However, if RCTs are incompletely reported, the methodological rigor with which they were conducted cannot be reliably evaluated and it may not be possible to replicate the intervention. Missing information also may limit the reader's ability to evaluate the external validity of a trial. Reporting guidelines are available for clinical trials in human healthcare (CONSORT), livestock populations (REFLECT), and preclinical experimental research involving animals (ARRIVE 2.0). The PetSORT guidelines complement these existing guidelines, providing recommendations for reporting controlled trials in pet dogs and cats. The rationale and scientific background are explained for each of the 25 items in the PetSORT reporting recommendations checklist, with examples from well-reported trials.

BackgroundThe external validity of randomized controlled trials (RCTs) is critical for the relevance of trial results in a clinical setting. We aimed to assess the external validity of RCTs investigating...

This paper focuses on the way methodological debates in the social sciences are appropriated by teacher researchers to inform practice. It argues Hammersley's prescriptions for ethnography and the development of the sociology of education, if taken up by teacher researchers, leads to a truncated practice. A practice that folds into empiricism and has difficulty in grasping the complexities of social relations and therefore produces an inadequate practice. This aligns with pre‐occupations present in the training of teacher researchers which focuses on the solution of immediate practical problems. An alternative model of educational research would recognise our positionality as researchers and take seriously the notions of oppression and social antagonism.