The Power of Replicated Measures to Increase Statistical Power

To explore the relationship between sample size in the groups and statistical power of ANOVA and Kruskal-Wallis H test with an imbalanced design. The sample sizes of the two tests were estimated by SAS program with given parameter settings, and Monte Carlo simulation was used to examine the changes in power when the total sample size varied or remained fixed. In ANOVA, when the total sample size was fixed, increasing the sample size in the group with a larger mean square error improved the statistical power, but an excessively large difference in the sample sizes between groups led to reduced power. When the total sample size was not fixed, a larger mean square error in the group with increased sample size was associated with a greater increase of the statistical power. In Kruskal-wallis H test, when the total sample size was fixed, increasing the sample size in groups with large mean square errors increased the statistical power irrespective of the sample size difference between the groups; when total sample size was not fixed, a larger mean square error in the group with increased sample size resulted in an increased statistical power, and the increment was similar to that for a fixed total sample size. The relationship between statistical power and sample size in groups is affected by the mean square error, and increasing the sample size in a group with a large mean square error increases the statistical power. In Kruskal-Wallis H test, increasing the sample size in a group with a large mean square error is more cost- effective than increasing the total sample size to improve the statistical power.

Understanding statistical power in the context of applied research

A sample size with sufficient statistical power is critical to the success of genetic association studies to detect causal genes of human complex diseases. Genome-wide association studies require much larger sample sizes to achieve an adequate statistical power. We estimated the statistical power with increasing numbers of markers analyzed and compared the sample sizes that were required in case-control studies and case-parent studies. We computed the effective sample size and statistical power using Genetic Power Calculator. An analysis using a larger number of markers requires a larger sample size. Testing a single-nucleotide polymorphism (SNP) marker requires 248 cases, while testing 500,000 SNPs and 1 million markers requires 1,206 cases and 1,255 cases, respectively, under the assumption of an odds ratio of 2, 5% disease prevalence, 5% minor allele frequency, complete linkage disequilibrium (LD), 1:1 case/control ratio, and a 5% error rate in an allelic test. Under a dominant model, a smaller sample size is required to achieve 80% power than other genetic models. We found that a much lower sample size was required with a strong effect size, common SNP, and increased LD. In addition, studying a common disease in a case-control study of a 1:4 case-control ratio is one way to achieve higher statistical power. We also found that case-parent studies require more samples than case-control studies. Although we have not covered all plausible cases in study design, the estimates of sample size and statistical power computed under various assumptions in this study may be useful to determine the sample size in designing a population-based genetic association study.

Statistical significance, power and sample size – what does it all mean?

SU35 - EVALUATING THE IMPACT OF PHENOTYPIC DEFINITIONS ON STATISTICAL POWER IN MAJOR DEPRESSION GENETICS

Power and sample size calculations: A review and computer program

Sample size and power determination is the first step in the experimental design of a successful study. Sample size and power calculation is required for applications for National Institutes of Health (NIH) funding. Sample size and power calculation is well established for traditional biological studies such as mouse model, genome wide association study (GWAS), and microarray studies. Recent developments in high-throughput sequencing technology have allowed RNAseq to replace microarray as the technology of choice for high-throughput gene expression profiling. However, the sample size and power analysis of RNAseq technology is an underdeveloped area. Here, we present RNAseqPS, an advanced online RNAseq power and sample size calculation tool based on the Poisson and negative binomial distributions. RNAseqPS was built using the Shiny package in R. It provides an interactive graphical user interface that allows the users to easily conduct sample size and power analysis for RNAseq experimental design. RNAseqPS can be accessed directly at http://cqs.mc.vanderbilt.edu/shiny/RNAseqPS/.

Considerations of sample size in medical research

Statistical power and prediction accuracy in multisite resting-state fMRI connectivity

Epidemiologists have advocated for reporting confidence intervals and deemphasizing P values to address long-standing concerns about null-hypothesis statistical-significance testing, P hacking, and reproducibility. It is unknown if efforts to reduce reliance on P values have altered the distribution of P values. For 21 332 abstracts published 2000 to 2024 in 4 major epidemiology journals, two-sided P values (N=25 288) were calculated from estimates and confidence intervals scraped using ChatGPT's 4o-mini model. We evaluated trends over time to determine whether the empirical distribution of P values changed. We fitted to expected P-value distributions and simulated these distributions with and without assuming changes in statistical power over time. Average P values decreased from 2000 to 2024; the fraction of P values just below the .05 threshold also decreased. Fits to models indicate that statistical power increased. Increasing power would reduce average P value while also decreasing P values near the .05 threshold-precisely the trends observed in epidemiology journals. Although the frequency of P values near the .05 threshold has declined modestly, this likely reflects increases in statistical power rather than decreases in P hacking.

Cluster randomised crossover designs (CRXOs) are a powerful type of longitudinal cluster randomised trial in which all participating clusters switch between two treatment conditions. "Multiple-period" CRXOs divide the trial duration into a number of periods of equal length and can allow for multiple crossovers between treatment conditions. It can be assumed that increasing the number of crossovers leads to an increase in statistical power. We investigate whether this is true for standard correlation structures, comparing CRXO designs with equal numbers of clusters, participants, and periods but differing in the number of crossovers, considering continuous outcomes. We consider the formula for the variance of the treatment effect estimator for multiple-period CRXOs under exchangeable and block-exchangeable within-cluster correlation structures, assuming equal cluster-period sizes and different patterns of treatment conditions in the treatment sequences varying in the number of crossovers. We also conduct a simulation study to compare the statistical power between multiple-period CRXO designs with different numbers of crossovers that share the same number and duration of periods and the same number of participants in each cluster period. Under exchangeable and block-exchangeable correlation structures and equal cluster-period sizes, the number of crossovers in the treatment sequences does not impact study power, provided the design is balanced in terms of the number of periods and clusters implementing each condition. When an exchangeable or block-exchangeable within-cluster correlation structure and a time-invariant effect of treatment are assumed, power calculations for CRXO designs are invariant to the specific ordering of treatment conditions. In particular, a CRXO design with additional crossovers does not lead to increased statistical power compared to a CRXO design with just one crossover for these within-cluster correlation structures. Further work is required to investigate the utility of multiple crossovers in situations where the treatment effect varies over time.

P10. ‘Clustered sparse’ fMRI using simultaneous multislice excitation in language mapping

Several studies have compared the feeding of genetically modified (GM) grains and conventional grains to poultry. The general conclusion has been that there were no significant differences detected in the biological performance of the birds (i.e., the grains were bioequivalent). However, the question has been posed whether the experimental designs used in the studies had sufficient statistical power to detect treatment differences. The power of tests can be used to determine the ability of an experimental design to detect treatment differences. The definition of statistical power is the probability of rejecting the null hypothesis when it is false and should be rejected. The complement of statistical power is the Type II error (beta). That is, accepting the null hypothesis that there is no difference in treatments when there is one. A priori power analysis can indicate the probability at which the sampling regimen or experiment can actually detect an effect if a difference exists. Post hoc power analysis indicates the sufficiency or the sample size needed for an experiment that has already been conducted. In the current study, the power of tests for experiments published in the literature where significant and nonsignificant differences were reported between control birds and birds fed new feed grains was examined. With some exceptions, the power of tests is rarely formally considered or mentioned in poultry research. The results of the survey of the literature showed, in general, low power of statistical tests for feeding experiments involving non-GM grains or in those cases when GM and non-GM grains were compared in poultry feeding experiments. These results suggest that care needs to be taken when designing experiments for bioequivalence of grains fed to poultry.

IntroductionIn the last three decades, a critical movement occurring within quantitative psychological research began to develop. This development emerged in response to the misguided use of classical based on null hypothesis significance testing (NHST). promotes dichotomous thinking and provides limited information regarding the essence of investigated phenomena. Dichotomous thinking in science -- manifested as only accepting or rejecting research hypotheses -- prevents the advancement of science and may even skew the accumulation of knowledge by stimulating the exclusive publication of studies whose hypotheses have been accepted. The so-called new statistics movement critically challenges postulates and operations. This movement is an approach based on the analytical tool of estimation (Cummings, 2014), which promotes the use of effect size as descriptive statistic, confidence intervals as inferential statistics, and meta-analysis as a reliable form of knowledge accumulation. This paper is intended to analyze the disadvantages of and the advantages of using effect size, confidence intervals and statistical power in quantitative psychological research, especially in clinical studies. Also noted and stressed is the need for editors of scientific psychological journals to adhere to policies recommended by the A.P.A. in this regard.Quantitative research in psychologyTypically, quantitative psychological research is focused on detecting the occurrence of certain population phenomena by analyzing data from a sample. An example is the case in which a researcher wishes to know if a treatment to improve the quality of life of those who suffer from breast cancer performs better than a placebo treatment for another group or those on a waiting list (also known as the control group or contrast group) (Wilkinson, 1999). Similarly, to make the decision to confirm that an independent variable or treatment did or did not have an important effect, is used. In quantitative research methodology, there are two ways of quantifying this effect: (1) Null Hypothesis Significance Testing and (2) Effect Size (ES), as well as its respective confidence intervals (Cl). These two approaches are reviewed below.Null Hypothesis Significance Testing (NHST)NHST comes from the effect size on the population, the size of the sample used and the alpha level or that is selected (p being the abbreviation for probability). Most psychology research is focused on rejecting the null hypothesis and obtaining a small instead of observing the relevance of the results that are obtained (Kirk, 2001).Among the limitations of NHST, we found its sensitivity to sample size, its inability to accept the null hypothesis, and its lack of capacity to determine the practical significance of the statistical results.Kirk (2001) states that only establishes the probability of obtaining a more or less extreme effect if the null hypothesis is true. It does not, however, communicate the magnitude of the effect or its practical significance, meaning whether the effect is found to be useful or important. As a result, inferential statistical testing has been criticized; as expressed by Ivarsson, Andersen, Johnson, and Lindwall (2013): p levels may have little, if anything, to do with real-world meaning and practical value (p.97). Some authors, such as Schmidt (1996), even suggest that statistical contrast is unnecessary and recommend focusing only on ES estimation, and Cohen (1994) suggests that NHST has not only failed to support the advance of psychology as a science but also has seriously impeded it. (p. 997).Ronald Fisher was the father of modern and experimental design. Since his time, it has been established as a convention that the for statistical significance must be less than .05, which means that an observed difference between two groups has less than a 5% probability of occurring by chance or sampling error if the null hypothesis is assumed to be true initially. …

The challenge for nurse researchers is to design their intervention studies with sufficient sensitivity to detect the treatment effects they are investigating. In order to meet this challenge, researchers must understand the factors that influence statistical power. Underpowered studies can result in a majority of null results in a research area when, in fact, the interventions are effective. The sensitivity of a research design is not a function of just one element of the design but of the entire research design: its plan, implementation and statistical analysis. When discussing factors that can increase a research design's statistical power, attention is most often focused on increasing sample size. This paper addresses a variety of factors and techniques, other than increasing sample size, that nurse researchers can use to enhance the sensitivity of a research design so that it can attain adequate power.