Errors and biases in clinical research

Chapter 12 - Sex bias in surgical research

Previous studies demonstrate sex bias in surgical research. Female participants and investigators are underrepresented in surgical scientific research. To describe the distribution of male and female authors in 5 general-interest surgery journals, assess the association of author gender with sex bias, and explore whether investigators benefit from performing sex-inclusion research. For this bibliometric analysis, data were abstracted from 1921 original, peer-reviewed articles published from January 1, 2011, through December 31, 2012, in Annals of Surgery, American Journal of Surgery, JAMA Surgery, The Journal of Surgical Research, and Surgery. Excluded articles pertained to a sex-specific disease or did not report the number of study participants. An additional 119 articles contained gender-ambiguous author names and were omitted. Data were analyzed from April to June 2017. Male and female first and senior authors, number of female and male participants in each study, surgical specialty, and number of citations received per article. Of the 3604 authors of 1802 articles included in this study, 2791 first and senior authors (77.4%) were male and 813 (22.6%) were female. The prevalence of male and female authors was consistent across all 5 journals and among clinical and basic science research. Articles by female authors included a higher median number of female study participants compared with their male counterparts (27.5 vs 16.0; P = .01), but sex matched the inclusion of participants less frequently (36% vs 45%; P = .001). No sex-based differences occurred between male and female authors in reporting, statistical analysis, and discussion of the data or in the number of citations received. Compared with studies that did not report, analyze, or discuss data by sex, studies that performed sex-specific data reporting yielded a mean of 2.8 more citations (95% CI, 1.2-4.4; P = .001); those that performed statistical analysis, a mean of 3.5 more citations (95% CI, 1.8-5.1; P = .001); and those that discussed the data, a mean of 2.6 more citations (95% CI, 0.7-4.5; P = .001). Articles with a higher percentage of sex matching of participants also received more citations, with an increase of 1 citation per 4.8% (95% CI, 2.0%-7.7%; P = .001) increase in percentage of sex matching. Sex bias in surgical research is prevalent among male and female authors; however, female authors included proportionally more female participants in their studies compared with male authors. Notably, studies that addressed sex bias were rewarded by the scientific community with increased citations of their published work.

High-quality medical care is the result of clinical decisions based upon scientific principles garnered from basic, translational, and clinical research. Information regarding the natural history of diseases and their responses to various treatments is introduced into the medical literature through the approximately one million PubMed journal articles published each year. Pharmaceutical and device companies, universities, departments, and researchers all stand to gain from research publication. Basic and translational research is highly competitive. Success in obtaining research funding and career advancement requires scientific publication in the medical literature. Clinical research findings can lead to changes in the pattern of orthopaedic practice and have implications for the utilization of pharmaceuticals and orthopaedic devices. Research findings can be biased by ownership of patents and materials, funding sources, and consulting arrangements. The current high-stakes research environment has been characterized by an increase in plagiarism, falsification or manipulation of data, selected presentation of results, research bias, and inappropriate statistical analyses. It is the responsibility of the orthopaedic community to work collaboratively with industry, universities, departments, and medical researchers and educators to ensure the integrity of the content of the orthopaedic literature and to enable the incorporation of best practices in the care of orthopaedic patients.

Prescriptions for psychiatric drugs to children and adolescents skyrocketed in the past ten years. Meanwhile, concerns of suicidality and industry bias in research have prompted regulatory investigation to assess claims that selective serotonin reuptake inhibitors (SSRIs) are safe and effective for children. Family clinicians may be unaware of the controversy or do not have the time or expertise to evaluate drug research. A five flaws analysis of clinical trial research, notably SSRI and stimulant studies, offers an efficient strategy for examining scientific claims. The authors recommend that therapists critically evaluate the scientific basis for medicating youths. Guidelines are provided to assist clinicians helping young people and their families make informed decisions.

Background: Randomization, concealment of treatment allocation, and blinding are all known to limit bias in clinical research. Nonsurgical studies that fail to meet these standards have been reported to inflate the differences between treatment and control groups. While surgical trials can rarely blind surgeons or patients, they can often blind outcome assessors. The aim of this systematic review was threefold: (1) to examine the reporting of outcome measures in orthopaedic trials, (2) to determine the feasibility of blinding in published orthopaedic trials, and (3) to examine the association between the magnitude of treatment differences and the blinding of outcome assessors. Methods: We identified and reviewed thirty-two randomized, controlled trials published in The Journal of Bone and Joint Surgery (American Volume) in 2003 and 2004 for the appropriate use of outcome measures. These trials represented 3.4% of all 938 studies published during that time-period. All thirty-two trials were reviewed by two authors for (1) the outcome measures used and (2) the blinding of outcomes assessors. We calculated the magnitude of the treatment effect of the use of blinded compared with unblinded outcome assessors. Results: Ten (31%) of the thirty-two randomized controlled trials used a modified outcome instrument. Of the ten trials, four failed to describe how the outcome instrument was modified. Nine of the ten articles did not describe how the modified instrument was validated and retested. Sixteen of the thirty-two randomized controlled trials did not report blinding of outcome assessors when blinding would have been possible. Among the studies with continuous outcome measure, unblinded outcomes assessment was associated with significantly larger treatment effects than blinded outcomes assessment (standardized mean difference, 0.76 compared with 0.25; p = 0.01). Similarly, in the studies with dichotomous outcomes, unblinded outcomes assessments were associated with significantly greater treatment effects than blinded outcomes assessments (odds ratio, 0.13 compared with 0.42; p < 0.001). The ratio of odds ratios (unblinded to blinded outcomes assessment) was 0.31, suggesting that unblinded outcomes assessment was associated with a potential for exaggeration of the benefit of the effectiveness of a treatment in our cohort of studies. Conclusions: In future orthopaedic randomized controlled trials, emphasis should be placed on detailed reporting of outcome measures to facilitate generalization and the outcome assessors should be blinded, when possible, to limit bias.

BackgroundPublication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR.MethodsDatabases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively.ResultsAfter screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases.ConclusionsMethodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper.Systematic review registrationPROSPERO 2016 CRD42016052333.

As highlighted by research on typically developing children, various biases exist when evaluating bilingual children's abilities. These biases can lead to inequitable assessment of language and cognitive abilities-potentially over- or underestimating bilinguals' skills. Recent reviews on neurodivergent bilingual children alluded to the possibility that these biases are also present in clinical research. This review examines bilingual neurodiversity research in children through the lens of equity, diversity, and inclusion. Specifically, it evaluates potential biases in recent studies to determine whether linguistic and cognitive abilities are assessed equitably, identify the types of linguistic and neurodiverse experiences represented in research, and examine the roles bilingual individuals play in research. We conducted an abbreviated systematic review with a multi-pronged search of databases and a manual search for quantitative studies on linguistic and cognitive abilities with bilingual neurodivergent children. The Joanna Briggs Institute Checklist was adapted for risk of bias assessment. Data was extracted and analysed from 95 studies, including study methods, bilingualism-related information (e.g., age of acquisition, language history tools, socioeconomic status), outcomes of interest (language, cognition), tasks (e.g., domain, name), and the main results or conclusions of each article. We found that equitable bilingual assessment of language and cognition was highly affected by the lack of culturally and linguistically appropriate tools. Most studies used case-control designs, contrasting neurodivergent bilinguals with monolingual or typically developing peers, which promotes a deficit-based monolingual-centred view in bilingual neurodiversity research. We also identified persistent challenges in defining and measuring bilingualism that complicate cross-comparison across studies and conditions. Research focus remained largely on developmental language disorder (DLD; n = 34) and autism spectrum disorder (ASD; n = 29) given their language symptomology, while acquired disorders are understudied. Additionally, there is a lack of community-based research that could offer more inclusive methods by involving bilingual communities throughout the research process. This review emphasizes the need to adopt equitable and inclusive research practices to better understand and support neurodivergent bilingual children. Future research should embrace a nuanced understanding of bilingualism and neurodiversity, prioritizing inclusive methodologies as well as holistic assessments using culturally and linguistically appropriate tools to avoid misdiagnoses and ensure fair clinical evaluations of language and cognition. What is already known on this subject Prior research has demonstrated that neurotypical bilingual children are often compared to monolingual norms, which can introduce biases and result in mischaracterization of bilingual abilities. Monolingually normed assessments are inequitable for use with bilingual children. What this paper adds to existing knowledge This review examines biases in recent research on neurodivergent bilingual children, focusing on the assessment of cognitive and language abilities-skills also often evaluated by clinicians, including speech-language pathologists. What are the potential or actual clinical implications of this work? This review integrates a structured EDI framework to contextualise research on neurodiversity for clinicians and researchers. It highlights the need to implement holistic and culturally appropriate assessment methods for all bilingual children that can lead to more equitable evaluations and help to better support tailored interventions and inclusive clinical and research practices.

Purpose:Following surgical reconstructions considered successful, many patients with brachial plexus injuries (BPI) have limited limb motion. In addition to manual muscle testing, clinicians typically measure active range of motion (AROM). AROM measurement relies on in-person examination and manual goniometers or visual estimation, both of which are subject to interobserver variability. The purpose of this proof-of-concept study was to evaluate reliability of video-based AROM assessments. We hypothesized that video-based assessment of AROM would have high inter-rater reliability (IRR) among surgeons who perform BPI reconstruction.Methods:We video recorded a standardized examination in a convenience sample of 8 postreconstruction BPI patients performing 3 motions: elbow flexion (EF), shoulder flexion (FF), and shoulder abduction (ABD). Eight BPI surgeons were given access to the videos and instructed on how to measure AROM first visually and then digitally using ImageJ. We examined the correlation between video-based and in-person goniometry measurements and assessed IRR of visual estimates and digital goniometry using Shrout-Fleiss Intraclass 3 fixed set correlations.Results:For EF, digital goniometry had a higher correlation (r = 0.92; p < 0.01) than visual assessment (r = 0.73; p < 0.01) relative to in-person measurements. IRR for EF was 0.80 for visual assessments and 0.96 for digital assessments. For FF, digital goniometry (r = 0.80; p < 0.01) and visual assessment (r = 0.80; p < 0.01) had similar correlations relative to in-person measurements. IRR for FF was 0.95 for visual assessments and 0.99 for digital assessments. For ABD, digital goniometry had a higher correlation (r = 0.85; p < 0.01) than visual assessment (r = 0.80; p < 0.01) relative to in-person measurements. IRR for ABD was 0.91 for visual assessments and 0.96 for digital assessments.Conclusions:Using standardized footage, visual estimates and digital goniometry of patient with EF, FF, and ABD were highly reliable among BPI surgeons. Digital goniometry of AROM was slightly more reliable than visual estimates for all 3 motions.Clinical Relevance:In addition to facilitating remote assessments to minimize patient travel, video-based assessments may allow opportunity to minimize reporting bias in clinical research through evaluation of results by multiple raters.

Informed consent as a source of bias in clinical research

This thesis is part of the studies of gender bias in health which together with the paradigm of evidence-based medicine shares the empirical assumption that there are inaccuracies in medical practice, in addition to a lack of rigour and transparency. It worked with the distinction between the concepts of sex and gender and between the concepts of sex-related differences and gender inequalities, in terms of applying a gender perspective in the study design and the subsequent analysis. This PhD review presents the research process conducted in Spain, which can provide an example for future research. Study I described a review of 58 clinical trials (CTs) of etoricoxib to assess its compliance with the Recommendations of Evaluation of Gender Differences in the Clinical Evaluation of Drugs. In Study II, key informants from professions related to different areas in drug development and pharmacovigilance held a working meeting to reach a consensus document on recommendations for the study and evaluation of gender differences in CTs in Spain. In Study III, the websites of the eight best-selling hormone replacement therapy drugs in Spain on Google first page of results were analysed. In Study IV, a logistic regression analysis was performed to compare analgesic prescription by sex in regions with a higher or lower Gender Development Index (GDI) than the Spanish average. Gender biases identified in this thesis limited the legitimacy of medicine, which is not based on the best possible evidence. The results also demonstrate the existence of inequalities between men and women that are not due merely to biological differences, but are gender inequalities stemming from the social differences that exist between both sexes.

‘This study was funded by (Company A). Professor XYZ has received honoraria and travel support for lectures and advisory boards, as well as research grants, from (Company A) and (Company B).’

HomeCirculationVol. 113, No. 20Response to Letter Regarding Article by Khan et al, “Prognostic Value of Troponin T and I Among Asymptomatic Patients With End-Stage Renal Disease: A Meta-Analysis” Free AccessLetterPDF/EPUBAboutView PDFView EPUBSections ToolsAdd to favoritesDownload citationsTrack citationsPermissions ShareShare onFacebookTwitterLinked InMendeleyReddit Jump toFree AccessLetterPDF/EPUBResponse to Letter Regarding Article by Khan et al, “Prognostic Value of Troponin T and I Among Asymptomatic Patients With End-Stage Renal Disease: A Meta-Analysis” Nadia A. Khan, MD, MSc, FRCPC Brenda R. Hemmelgarn, MD, PhD, FRCPC Marcello Tonelli, MD, MSc, FRCPC Christopher R. Thompson, MD, CM, FRCPC Adeera Levin, MD, FRCPC Nadia A. KhanNadia A. Khan Division of Internal Medicine, University of British Columbia, British Columbia, Canada Search for more papers by this author Brenda R. HemmelgarnBrenda R. Hemmelgarn Division of Nephrology, University of Calgary, Calgary, Alberta, Canada Search for more papers by this author Marcello TonelliMarcello Tonelli Division of Nephrology, University of Alberta, Edmonton, Alberta, Canada Search for more papers by this author Christopher R. ThompsonChristopher R. Thompson Division of Cardiology, University of British Columbia, British Columbia, Canada Search for more papers by this author Adeera LevinAdeera Levin Division of Nephrology, University of British Columbia, British Columbia, Canada Search for more papers by this author Originally published23 May 2006https://doi.org/10.1161/CIRCULATIONAHA.106.619643Circulation. 2006;113:e776We thank Drs Apple, Jaffe, and Herzog for their interest in our article and for drawing attention to the National Kidney Foundation (NKF) Guidelines on Evaluation of Cardiovascular disease.1 As stated in our paper, because the troponin studies used differing assays and variable cut points and reported conflicting results, direct comparison of the results was problematic, and a rigorous meta-analysis was required to clarify this association.2 The review from the NKF did not include all relevant studies (eg, results from 8 trials were not included in the NKF review), reported on results using differing cut points or outcomes, and did not evaluate for study heterogeneity and publication bias, given the significant possibility that negative observational studies may not be published3 (as was indeed found in our analysis). In addition, we respectfully submit that recommendations from professional organizations and regulatory bodies, although valuable, can be usefully informed by additional research—the existence of the NKF and FDA documents does not contradict our statement that the available literature has produced conflicting results.Dr Apple and colleagues correctly point out that the cut point of 0.1 ng/mL is not a part of the ESC/ACC guidelines. We chose this cut point to improve between study comparisons and allow for mathematical synthesis, as the vast majority of troponin T studies evaluated prognosis at this level. Thus, from the evidence base, a cut point of 0.1 ng/mL was associated with reduced long-term survival among asymptomatic hemodialysis patients. The degree of imprecision at this cut point is also close to that recommended by the ESC/ACC. Most troponin assays do not currently meet this standard at the 99th percentile.4–5 However, as more studies accrue and precision of troponin assays at lower thresholds improves, a lower cut point may prove to have greater predictive power in this population.We agree that troponin T elevations are more often seen in patients with chronic kidney disease compared with troponin I. It remains difficult to assess the ability of troponin I to predict mortality, given the variability of assays and assay cut points studied.Thank you for identifying the typographical error in Table 2: The table column should read 99th percentile and not the 90th percentile.DisclosuresNone.1 K/DOQI Workgroup. K/DOQI clinical practice guidelines for cardiovascular disease in dialysis patients. Am J Kidney Dis. 2005; 45 (4 Suppl 3): S1–S153.Google Scholar2 Khan NA, Hemmelgarn BR, Tonelli M, Thompson CR, Levin A. Prognostic value of troponin T and I among asymptomatic patients with end-stage renal disease: a meta-analysis. Circulation. 2005; 112: 3088–3096.LinkGoogle Scholar3 Easterbrook PJ, Berlin J, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991; 337: 867–872.CrossrefMedlineGoogle Scholar4 Panteghini M, Pagani F, Yeo KT, Apple FS, Christenson RH, Dati F, Mair J, Ravkilde J, Wu AH; Committee on Standardization of Markers of Cardiac Damage of the IFCC. Evaluation of imprecision for cardiac troponin assays at low-range concentrations. Clin Chem. 2004; 50: 327–332.CrossrefMedlineGoogle Scholar5 Apple FS, Murakami MM. Serum 99th percentile reference cutoffs for seven cardiac troponin assays. Clin Chem. 2004; 50: 1477–1479.CrossrefMedlineGoogle Scholar Previous Back to top Next FiguresReferencesRelatedDetailsCited By Vircoulon M and Combe C (2012) Conséquences cardiaques de l’insuffisance rénale chronique, EMC - Néphrologie, 10.1016/S1762-0945(12)40514-9, 9:1, (1-14), Online publication date: 1-Jan-2012. May 23, 2006Vol 113, Issue 20 Advertisement Article InformationMetrics https://doi.org/10.1161/CIRCULATIONAHA.106.619643 Originally publishedMay 23, 2006 PDF download Advertisement SubjectsChronic Ischemic Heart DiseaseDiagnostic Testing

Non-publication and publication bias in animal research is a core topic in current debates on the “reproducibility crisis” and “failure rates in clinical research”. To date, however, we lack reliable evidence on the extent of non-publication in animal research. We collected a random and stratified sample (n = 210) from all archived animal study protocols of two major German UMCs (university medical centres) and tracked their results publication. The overall publication rate was 67%. Excluding doctoral theses as results publications, the publication rate decreased to 58%. We did not find substantial differences in publication rates with regard to i) the year of animal study approval, ii) the two UMCs, iii) the animal type (rodents vs. non-rodents), iv) the scope of research (basic vs. preclinical), or v) the discipline of the applicant. Via the most reliable assessment strategy currently available, our study confirms that the non-publication of results from animal studies conducted at UMCs is relatively common. The non-publication of 33% of all animal studies is problematic for the following reasons: A) the primary legitimation of animal research, which is the intended knowledge gain for the wider scientific community, B) the waste of public resources, C) the unnecessary repetition of animal studies, and D) incomplete and potentially biased preclinical evidence for decision making on launching early human trials. Results dissemination should become a professional standard for animal research. Academic institutions and research funders should develop effective policies in this regard.

Nonselection and nonresponse bias in clinical research

In this article, we consider the intersection of cognitive motor dissociation (CMD) and artificial intelligence (AI), hence when CMD meets AI. In covert consciousness, there is a discordance between the observed behavior, the traditional bedside mode of assessment, and the response to volitional commands as depicted by neuroimaging or EEG studies. This alphabet soup of acronyms represents both the promise and peril of nascent technology in covert consciousness. On the diagnostic side, there is the complexity and uncertainty of identifying the discordance between cognitive activity and overt behavior. On the therapeutic side, when AI is used to generate speech, there is the possibility of misrepresenting the thoughts and intentions of those who are otherwise voiceless. This concordance of factors makes the application of AI to CMD worthy of deeper consideration. We offer this analysis in the spirit of anticipatory governance, a prudential process by which one plans to prevent or mitigate unintended consequences of novel technology. We first consider the normative challenges posed by CMD for clinical practice, neuroethics, and the law. We then explore the history of covert consciousness and the relationship of severe brain injury to the right-to-die movement, before introducing three biographies of brain injury that highlight the potential impact of disability bias or ableism in clinical practice, assistive technology, and translational research. Subsequently, we explore how AI might give voice to conscious individuals who are unable to communicate and the ethical challenges that this technology must overcome to promote human flourishing drawing upon what Nussbaum and Sen have described as a "capabilities approach" to promote normative reasoning.