Expanding the scope of questionable research practices in applied linguistics

Focuses on open science practices in suicide research. Specifically, the authors use examples from their own and others' work to demonstrate the opportunities for future-proofing research by implementing open science practices, and they discuss some of the challenges and their potential solutions. They cover implementing open science practices in new, ongoing, and concluded studies, and discuss practices in order of being 'low' to 'high' threshold to implement (based on Kathawalla et al., 2021). Space constraints preclude an extensive presentation of all open science practices although they briefly describe the strategies of providing preprints, sharing study materials and codes, preregistration of studies and registered reports, and sharing research data. To highlight the open science work of as many researchers as possible, examples are provided in Electronic Supplementary Material 1 (ESM 1) rather than in the text. The authors ask for readers' help in adding to these examples via their 'living' reading list (https://osf.io/v6y3t/).

Recent research has documented both questionable and responsible research practices in published psychology research, but it is unclear which research practices psychology students engage in when graduating from their master's program. In this study, we documented the prevalence of responsible and questionable research practices in 300 psychology master's theses from Tilburg University, the Netherlands, and associated these practices with supervisor's grading of the theses. Compared to authors of published scientific manuscripts, master's students seemed to engage more in responsible research practices, conducted more power analyses, used larger sample sizes, reported fewer statistically significant results, and provided more detailed reporting of their results. However, statistical reporting errors were almost as common in master's theses as they are in the published literature. We found no relationship between thesis grades and any of the responsible or questionable research practices. We also found no relationships among practices, suggesting that there is no unidimensional construct of "responsible scientific behavior".

ObjectivesExplore the occurrence and nature of questionable research practices (QRPs) in the reporting of messages and conclusions in international scientific Health Services Research (HSR) publications authored by researchers from HSR...

Between 2-4% of researchers admit to have falsified or fabricated their data. The prevalence of such unethical behavior can be as high as 10% in some disciplines or countries. Data falsification is an extreme form of questionable research practices that are both less problematic and much more widespread: surveys on different disciplines have shown that more than half of researchers make some form of selective reporting or add new data until they obtain significant results. Unethical practices harm the global quality of research. Provided they have been validated by peer review, fabricated, distorted or selected data are featured in literature review or meta-analysis. They can in turn influence directions for future research or, even, policy decisions with wide range implications in health, economics or politics. Negative incentives have attracted significant attention: publishers, institutions and research evaluators tend to favor unprecedented research that does not simply confirm common hypotheses. The lack of proper tools, standards and workflows to deal efficiently with data is also a fundamental issue. In most disciplines, data collection is not well organized nor maintained: it has been estimated that as much as half of the datasets created from the 1990s in life science are already lost. Questionable research practices partly stem from common deficiencies in scientific data management. Open science and data sharing have recently emerged as a common framework to solve issues of research integrity. While initially focused on access to publications, the open science movement is more broadly concerned with transparency at all the stages of the research lifecycle. The diffusion of datasets in open repositories and infrastructures has already largely solved major issues of long term preservation. It also ensures that potential errors or adjustments to statistical indicators can be subsequently corrected, as posterior analysis and replications have access to the original data source. Open science is now increasingly integrated into ethical standards, following community-led initiatives like the TOP Guidelines (2014). The European Code of Conduct for Research Integrity from 2017 includes full requirements for open access, open data and reproducible workflow: “Researchers, research institutions and organizations ensure access to data is as open as possible, as closed as necessary”. The Hong Kong principles for assessing researchers (2020) acknowledge open science as one of the five pillars of scientific integrity. Open science is changing the nature of the debate over research integrity which had remained until now largely detached from the public space. 60%-90% of the audience of open scientific platforms comes from non-academic professionals and private citizens. This increased diffusion creates new responsibilities but also new opportunities to involve non-academic stakeholders in the spirit of citizen science.

Prevalence of research misconduct, questionable research practices (QRPs) and their associations with a range of explanatory factors has not been studied sufficiently among academic researchers. The National Survey on Research Integrity targeted all disciplinary fields and academic ranks in the Netherlands. It included questions about engagement in fabrication, falsification and 11 QRPs over the previous three years, and 12 explanatory factor scales. We ensured strict identity protection and used the randomized response method for questions on research misconduct. 6,813 respondents completed the survey. Prevalence of fabrication was 4.3% (95% CI: 2.9, 5.7) and of falsification 4.2% (95% CI: 2.8, 5.6). Prevalence of QRPs ranged from 0.6% (95% CI: 0.5, 0.9) to 17.5% (95% CI: 16.4, 18.7) with 51.3% (95% CI: 50.1, 52.5) of respondents engaging frequently in at least one QRP. Being a PhD candidate or junior researcher increased the odds of frequently engaging in at least one QRP, as did being male. Scientific norm subscription (odds ratio (OR) 0.79; 95% CI: 0.63, 1.00) and perceived likelihood of detection by reviewers (OR 0.62, 95% CI: 0.44, 0.88) were associated with engaging in less research misconduct. Publication pressure was associated with more often engaging in one or more QRPs frequently (OR 1.22, 95% CI: 1.14, 1.30). We found higher prevalence of misconduct than earlier surveys. Our results suggest that greater emphasis on scientific norm subscription, strengthening reviewers in their role as gatekeepers of research quality and curbing the “publish or perish” incentive system promotes research integrity.

The exposure of scientific scandals and the increase of dubious research practices have generated a stream of studies on Questionable Research Practices (QRPs), such as failure to acknowledge co-authors, selective presentation of findings, or removal of data not supporting desired outcomes. In contrast to high-profile fraud cases, QRPs can be investigated using quantitative, survey-based methods. However, several design issues remain to be solved. This paper starts with a review of four problems in the QRP research: the problem of precision and prevalence, the problem of social desirability bias, the problem of incomplete coverage, and the problem of controversiality, sensitivity and missing responses. Various ways to handle these problems are discussed based on a case study of the design of a large, cross-field QRP survey in the social and medical sciences in Sweden. The paper describes the key steps in the design process, including technical and cognitive testing and repeated test versions to arrive at reliable survey items on the prevalence of QRPs and hypothesized associated factors in the organizational and normative environments. Partial solutions to the four problems are assessed, unresolved issues are discussed, and tradeoffs that resist simple solutions are articulated. The paper ends with a call for systematic comparisons of survey designs and item quality to build a much-needed cumulative knowledge trajectory in the field of integrity studies.

This paper presents and discusses empirical results from a survey about the research practice of Danish chemistry students, with a main focus on the question of anomalous data. It seeks to investigate how such data is handled by students, with special attention to so-called 'questionable research practices' (QRPs) where anomalous data are simply deleted or discarded. This question of QRPs is of particular importance as the educational practices students experience may influence how they act in their future professional careers, for instance in research. The ethical evaluation of QRPs however is not univocal. In parts of the literature QRPs are seen as unquestionably bad, while in other parts of the literature certain QRPs are seen as a necessary aspect of scientific practice. Results from the survey of Danish chemistry students shows that many students engage in certain types of questionable practices, and that a large minority of the students have been actively encouraged by their teachers to engage in such practices. The paper discusses to what extent and under what circumstances such instructional practices can be defended and suggests how the instructional practice connected to the handling of anomalous data can be improved.

The current study sought to determine research scientists' sensitivity to various justifications for engaging in behaviors typically considered to be questionable research practices (QRPs) by asking them to evaluate the appropriateness and ethical defensibility of each. Utilizing a within-subjects design, 107 National Institutes of Health principal investigators responded to an invitation to complete an online survey in which they read a series of research behaviors determined, in prior research, to either be ambiguous or unambiguous in their ethical defensibility. Additionally, each behavior was paired with either an ostensibly sound or unsound reason for the behavior. Consistent with hypotheses, the results indicated that scientists perceived QRPs as more appropriate and defensible when paired with a justifiable motive relative to when paired with a clearly unethical motive, particularly for QRPs that are more ambiguous in their ethicality. In fact, ambiguous QRPs were perceived as categorically defensible when given a justifiable motive. This suggests scientists are sensitive to contextual factors related to QRPs' appropriateness, which could inform how institutions develop appropriate training modules for research integrity.

Fraud and Deceit in Medical Research

While Open Science has arguably initiated positive changes at some stages of the research process (e.g., increasing transparency through preregistration), problematic behaviors during data collection are still almost impossible to detect and pose a great risk to the validity and integrity of psychological research—especially, when researchers use data collected by others (e.g., students). Exploring students’ and supervisors’ perspectives, the present registered report enlightens this “black box” of student data collection, focusing on questionable research practices and research misconduct (QRP/M). The majority of students did not report having engaged in any problematic behaviors during data collection, but some QRP/M—ranging from somewhat questionable to highly fraudulent—seem quite common (e.g., telling participants the hypothesis beforehand, participating in one’s own survey). We provide an overview of students’ reported and supervisors’ suspected data collection QRP/M, explore potential drivers for these behaviors based on the fraud triangle model (including pressures, opportunities, and rationalizations), and report how students and supervisors perceive the eligibility of student data for further uses (e.g., scientific publications). Moreover, we explore the role of the student-supervisor relationship (e.g., communication and expectations) and Open Science practices in student projects. In summary, our findings suggest the potential scientific value of data from student projects. Fostering transparent communication regarding expectations, experiences, and intentions between supervisors and students might further contribute to strengthening this prospect.

As the field of psychology and law begins to embrace more transparent and accessible science, many questions arise about what open science actually is and how to do it. In this chapter, we contextualize this reform by examining fundamental concerns about psychological research—irreproducibility and replication failures, false-positive errors, and questionable research practices—that threaten its validity and credibility. Next, we turn to psychology’s response by reviewing the concept of open science and explaining how to implement specific practices—preregistration, registered reports, open materials/data/code, and open access publishing—designed to make research more transparent and accessible. We conclude by weighing the implications of open science for the field of psychology and law, specifically with respect to how we conduct and evaluate research, as well as how we train the next generation of psychological scientists and share scientific findings in applied settings.

In this evidence-based opinion piece on responsible conduct of research a short overview of the most prominent recent cases of sanctioned scientifi c misconduct, developments in the fi eld of responsible conduct of research, defi nitions of types of scientifi c misconduct, and qu estionable research practices is given. Intrinsic and extrinsic motivation of scientists to publish and perform research is discussed, as well as the perception of fraud, its acceptability and infl uence on science and the scientifi c (p ublic) record, with a special emphasis on the frequency of sanctioned and discovered fraudulent research. Data on (self)reported willingness to perform misconduct and questionable research practices are analyzed and discussed. An extensive overview on recent publications concerning topics of responsible conduct of research is presented. Finally, some suggestions to what the stakeholders could do are given, as well as easy (self)checks against scientifi c misconduct.

This cross-sectional study evaluates the knowledge, attitudes, experiences, and behavior of final year medical students, PhD students, and supervisors at the School of Medicine of the University of Zagreb in relation to research misconduct, questionable research practices, and the research environment. The overall response rate was 36.4% (68%-100% for the paper survey and 8%-15% for the online surveys). The analysis reveals statistically significant differences in attitude scores between PhD students and supervisors, the latter having attitudes more in concordance with accepted norms. The results overall show a nonnegligible incidence of self-reported misconduct and questionable research practices, as well as some problematic attitudes towards misconduct and questionable research practices. The incidence of problematic authorship practices was particularly high. The research environment was evaluated as being mostly supportive of research integrity.

Recently, many psychological effects have been surprisingly difficult to reproduce. This article asks why, and investigates whether conceptually replicating an effect in the original publication is related to the success of independent, direct replications. Two prominent accounts of low reproducibility make different predictions in this respect. One account suggests that psychological phenomena are dependent on unknown contexts that are not reproduced in independent replication attempts. By this account, internal replications indicate that a finding is more robust and, thus, that it is easier to independently replicate it. An alternative account suggests that researchers employ questionable research practices (QRPs), which increase false positive rates. By this account, the success of internal replications may just be the result of QRPs and, thus, internal replications are not predictive of independent replication success. The data of a large reproducibility project support the QRP account: replicating an effect in the original publication is not related to independent replication success. Additional analyses reveal that internally replicated and internally unreplicated effects are not very different in terms of variables associated with replication success. Moreover, social psychological effects in particular appear to lack any benefit from internal replications. Overall, these results indicate that, in this dataset at least, the influence of QRPs is at the heart of failures to replicate psychological findings, especially in social psychology. Variable, unknown contexts appear to play only a relatively minor role. I recommend practical solutions for how QRPs can be avoided.Electronic supplementary materialThe online version of this article (doi:10.3758/s13423-016-1030-9) contains supplementary material, which is available to authorized users.

Questionable research practices (QRPs) can occur whenever one result is favored over another, and tests of mediation are no exception. Given mediation's ubiquity and importance to both theory and practice, QRPs in tests of mediation pose a serious threat to the advancement of psychology. We investigate this issue through the introduction of a straightforward means of detecting the presence and magnitude of QRPs in tests of mediation and validate this methodology with a series of sensitivity tests and simulations. We then apply this method to 2,569 tests of mediation published in five leading psychology journals in 2018 and 2019. We find that despite most hypothesized tests of mediation being likely underpowered, most (76%) were nevertheless supported. Furthermore, confidence intervals (CIs) that just barely exclude zero are 3.6 to 4.4 times as prevalent as those CIs that just barely include zero. We also find a number of study- and test-level factors, such as whether the test of mediation was hypothesized, explain both whether the CI excluded zero (odds ratio [OR] = 17.87, p < .001) as well as the CI's proximity to zero (b = .27, p < .001). In addition, other factors, most notably sample size, do predict the CI's proximity to zero (γ = .00, p < .001), but surprisingly do not predict the CI's exclusion of zero (OR = .99, p = .803). We conclude with actionable QRP curtailment strategies so that both, academics and practitioners, can have greater and well-founded confidence in tests of mediation in psychological research. (PsycInfo Database Record (c) 2021 APA, all rights reserved).