Nmax and the Quest to Restore Caution, Integrity, and Practicality to the Sample Size Planning Process

In a time when the alarms of research replicability are sounding louder than ever, mapping out studies with statistical and inferential integrity is of paramount importance. Indeed, funding agencies almost always require grant applicants to present compelling a priori power analyses to justify proposed sample sizes, as a critical part of the information considered collectively to ensure a sound investment. Unfortunately, even researchers' most sincere attempts at sample size planning are fraught with the fundamental challenge of setting numerical values not just for the focal parameters for which statistical tests are planned, but for each of the model's other, more peripheral or contextual parameters as well. As we plainly demonstrate, regarding the latter parameters, even in very simple models, any slight deviation in well-intentioned numerical guesses can undermine power for the assessment of the more focal parameters that are of key theoretical interest. Toward remedying this all-too-common but seemingly underestimated problem in power analysis, we adopt a hope-for-the-best-but-plan-for-the-worst mindset and present new methods that attempt to (a) restore appropriate conservatism and robustness, and in turn credibility, to the sample size planning process, and (b) greatly simplify that process. Derivations and suggestions for practice are presented using the framework of measured variable path analysis models as they subsume many of the types of models (e.g., multiple linear regression, analysis of variance) for which sample size planning is of interest. (PsycInfo Database Record (c) 2026 APA, all rights reserved).

Considerations of sample size in medical research

At the most general level, sample size refers to the number of participants (typically human or animal subjects) who provide data for a scientific experiment or study. Selecting an appropriate sample size is a fundamental feature of a well-designed study, and, as the title suggests, sample size can be carefully planned ahead of time. Sample size planning is important because the sample size of a study has implications for statistical power as well as accuracy. In conventional terms, statistical power reflects the probability of rejecting the null hypothesis, assuming that the purported effect is non-null (non-zero) in reality. Although power can be treated as a function of unknown parameter values, power is often defined conditionally on a specific value of population effect size. (On a technical level, power is conditional upon the population noncentrality parameter, which is a combination of sample size and effect size, but relying on effect size as a proxy is generally acceptable.) Power is calculated from the statistical significance threshold (otherwise known as the nominal Type I error rate or alpha-level), the population effect size, and the sample size. Because of the connection between sample size and power, sample size planning is sometimes called a priori power analysis. For example, if a new therapy developed to treat depression is truly beneficial, the sample size of a study assessing this therapy should be large enough to demonstrate the beneficial outcome as statistically significant. Accuracy reflects how close an estimate is to the parameter it aims to estimate. Accurate estimates are expressed with narrow confidence intervals (small margins of error). Sample size also has a connection to accuracy. For example, if the investigators in the previous example want to estimate how much the therapy reduces depressive symptoms, the sample size should be large enough so that the effect size estimate derived from the study is an accurate estimate of the true magnitude of the treatment effect. An underlying challenge to sample size planning for both goals is that the population effect size is unknown. An important corollary is that whether a particular sample size provides appropriate power (and in some cases, accuracy) depends on the population effect size, and as such, sample size cannot completely be judged without context. Although there are sometimes limitations to sample size, such as when working with a limited budget or specialized populations, and although sample size is often selected based on convenience or historical precedent, when possible it is desirable to directly plan the study sample size at the study design phase. Explicitly planning sample size helps to ensure that the sample size will be effective in meeting the investigators’ goals and answering the scientific questions of interest.

This chapter will discuss the key statistical considerations in sample size planning in multi-regional clinical trials (MRCT). It will build on the training module IV, discussing the concept of pooled region and pooled subgroups to aid overall sample size planning and allocation of that sample size across regions. This chapter will describe five approaches to sample size allocation as well as the pros and cons of each approach. In addition to the E17 sample size concept, we will also provide some summaries of popular statistical methodologies for sample size calculation and some practical implementation considerations.

A fundamental part of experimental design is to determine the sample size of a study. However, sparse information about population parameters and effect sizes before data collection renders effective sample size planning challenging. Specifically, sparse information may lead research designs to be based on inaccurate a priori assumptions, causing studies to use resources inefficiently or to produce inconclusive results. Despite its deleterious impact on sample size planning, many prominent methods for experimental design fail to adequately address the challenge of sparse a-priori information. Here we propose a Bayesian Monte Carlo methodology for interim design analyses that allows researchers to analyze and adapt their sampling plans throughout the course of a study. At any point in time, the methodology uses the best available knowledge about parameters to make projections about expected evidence trajectories. Two simulated application examples demonstrate how interim design analyses can be integrated into common designs to inform sampling plans on the fly. The proposed methodology addresses the problem of sample size planning with sparse a-priori information and yields research designs that are efficient, informative, and flexible. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

BackgroundWhen evaluating the impact of environmental exposures on human health, study designs often include a series of repeated measurements. The goal is to determine whether populations have different trajectories of the environmental exposure over time. Power analyses for longitudinal mixed models require multiple inputs, including clinically significant differences, standard deviations, and correlations of measurements. Further, methods for power analyses of longitudinal mixed models are complex and often challenging for the non-statistician. We discuss methods for extracting clinically relevant inputs from literature, and explain how to conduct a power analysis that appropriately accounts for longitudinal repeated measures. Finally, we provide careful recommendations for describing complex power analyses in a concise and clear manner.MethodsFor longitudinal studies of health outcomes from environmental exposures, we show how to [1] conduct a power analysis that aligns with the planned mixed model data analysis, [2] gather the inputs required for the power analysis, and [3] conduct repeated measures power analysis with a highly-cited, validated, free, point-and-click, web-based, open source software platform which was developed specifically for scientists.ResultsAs an example, we describe the power analysis for a proposed study of repeated measures of per- and polyfluoroalkyl substances (PFAS) in human blood. We show how to align data analysis and power analysis plan to account for within-participant correlation across repeated measures. We illustrate how to perform a literature review to find inputs for the power analysis. We emphasize the need to examine the sensitivity of the power values by considering standard deviations and differences in means that are smaller and larger than the speculated, literature-based values. Finally, we provide an example power calculation and a summary checklist for describing power and sample size analysis.ConclusionsThis paper provides a detailed roadmap for conducting and describing power analyses for longitudinal studies of environmental exposures. It provides a template and checklist for those seeking to write power analyses for grant applications.

ABSTRACTWe consider clinical trials with a binary composite endpoint where the trial is successful when a significant result is achieved for the composite or one prespecified main component. Appropriate sample size planning is challenging in this situation, as in addition to the Type I error rate, power, and target difference the overall event rates and the correlation between the test statistics have to be defined. Reliable estimates of these quantities, however, are usually hard to obtain and therefore there is a high risk to not achieve the intended power in a fixed sample size design. In this article, we propose an internal pilot study design where the nuisance parameters are estimated in a blinded way at an interim stage and where the sample size is then revised accordingly. We investigate the characteristics of the proposed design with respect to the actual Type I error rate, power, and sample size. The application of this design is illustrated by a clinical trial example.

ObjectiveTo investigate publication rates in small trials and to explore which factors are associated with publication rates in small trials, including sample size, the type and number of primary and secondary outcomes. Study design and settingWe studied a subgroup of ‘small’ trials from a pre-existing dataset (IntoValue), containing German trials completed between 2009 and 2017. Small trials were defined as phase II-III, III and IV trials with 150 or fewer participants. We performed an updated publication search and collected additional data from online trial records. ResultsOut of 499 trials, 325 (65%) trials published their results in a journal article or dissertation. Median time-to-publication was 3.41 years (95% CI: 3.04–4.10). Planned sample size was not associated with publication rates, but the difference between planned and achieved sample size was (per 10% unsuccessfully recruited participants, HR = 0.95, 95% CI: 0.91–1.00). Phase III vs. II-III trials, studied intervention (device vs. other) and clearly vs. unclearly defined primary outcomes predicted a higher likelihood of earlier publication. ConclusionAbout 35% of small trials in Germany remain unpublished, even after an extensive follow-up period of over 9 years. Publication rates are low and were associated with sample size, trial phase and type of intervention.

Design and analysis features used in small population and rare disease trials: A targeted review

In clinical trials, researchers usually determine a study sample size prior to the start of the study to provide a sufficient power at a targeted treatment difference. When the targeted treatment difference deviates from the true one, the study may either have insufficient power or use more subjects than necessary. To address the difficulty in sample size planning, researchers have developed various flexible sample size designs and compared their performances. Some previous work suggests that re-estimation designs are inefficient and that one can improve uniformly by using standard group sequential likelihood ratio tests, although more interim analyses are involved. However, researchers need to further study the statement and the minimal number of tests needed before a standard group sequential test might outperform a re-estimation design. In this paper, we conducted simulation studies to answer these questions using various optimality criteria.

Some Thoughts About Data Type, Distribution, and Statistical Significance

To evaluate the reporting of critical design issues and methods of statistical analysis in pharmacogenetic studies published in the medical literature. Systematic review of 65 original pharmacogenetic studies published in the literature over the last 15 years. The sample size determination and the planned sample size were lacking in 63 papers. The study design characterization was lacking in 43 papers. The number of patients analyzed ranged from 36 to 1400 (median=161 and interquartile range of 119-250). The Pearson's χ2 test and the Fisher's test were the most common forms of analysis. Multiple statistical testing was relevant to 59 papers, but only 11 addressed the issue of multiplicity (Bonferroni correction). Sources of multiplicity were multiple association assessment (45 papers), analysis of both genotype and allelic frequencies (44), and multiple analysis methods (unadjusted and adjusted). Hardy-Weinberg equilibrium was tested in 12 of 45 papers performing allelic analysis and was fully reported in four of them. The results of association analyses were commonly reported as P values but rarely as estimates of an association measure (odds ratio or relative risk) and its accuracy. These results show that there is considerable room for improvement in the current standards of design, analysis, and reporting of pharmacogenetic research.

Several studies have compared the feeding of genetically modified (GM) grains and conventional grains to poultry. The general conclusion has been that there were no significant differences detected in the biological performance of the birds (i.e., the grains were bioequivalent). However, the question has been posed whether the experimental designs used in the studies had sufficient statistical power to detect treatment differences. The power of tests can be used to determine the ability of an experimental design to detect treatment differences. The definition of statistical power is the probability of rejecting the null hypothesis when it is false and should be rejected. The complement of statistical power is the Type II error (beta). That is, accepting the null hypothesis that there is no difference in treatments when there is one. A priori power analysis can indicate the probability at which the sampling regimen or experiment can actually detect an effect if a difference exists. Post hoc power analysis indicates the sufficiency or the sample size needed for an experiment that has already been conducted. In the current study, the power of tests for experiments published in the literature where significant and nonsignificant differences were reported between control birds and birds fed new feed grains was examined. With some exceptions, the power of tests is rarely formally considered or mentioned in poultry research. The results of the survey of the literature showed, in general, low power of statistical tests for feeding experiments involving non-GM grains or in those cases when GM and non-GM grains were compared in poultry feeding experiments. These results suggest that care needs to be taken when designing experiments for bioequivalence of grains fed to poultry.

The planning of sample size for research studies often focuses on obtaining a significant result given a specified level of power, significance, and an anticipated effect size. This planning requires prior knowledge of the study design and a statistical analysis to calculate the proposed sample size. However, there may not be one specific testable analysis from which to derive power (Silberzahn et al., Advances in Methods and Practices in Psychological Science, 1(3), 337356, 2018) or a hypothesis to test for the project (e.g., creation of a stimuli database). Modern power and sample size planning suggestions include accuracy in parameter estimation (AIPE, Kelley, Behavior Research Methods, 39(4), 755–766, 2007; Maxell et al., Annual Review of Psychology, 59, 537–563, 2008) and simulation of proposed analyses (Chalmers & Adkins, The Quantitative Methods for Psychology, 16(4), 248–280, 2020). These toolkits offer flexibility in traditional power analyses that focus on the if-this, then-that approach. However, both AIPE and simulation require either a specific parameter (e.g., mean, effect size, etc.) or a statistical test for planning sample size. In this tutorial, we explore how AIPE and simulation approaches can be combined to accommodate studies that may not have a specific hypothesis test or wish to account for the potential of a multiverse of analyses. Specifically, we focus on studies that use multiple items and suggest that sample sizes can be planned to measure those items adequately and precisely, regardless of the statistical test. This tutorial also provides multiple code vignettes and package functionality that researchers can adapt and apply to their own measures.

TPS417 Background: On the basis of JCOG1109, neoadjuvant chemotherapy (NAC) with Docetaxel, Cisplatin, and 5-FU (DCF) has become one of the standard treatments for locally advanced esophageal squamous cell carcinoma (LAESCC). Although the efficacy of adjuvant therapy with nivolumab after preoperative chemoradiotherapy for LAESCC has been proven by CheckMate 577, the survival benefits of adjuvant therapy after NAC remain controversial. Additionally, a single-arm phase II trial of adjuvant S-1 for LAESCC who received NAC demonstrated favorable survival outcomes and safety. Methods: Eligibility criteria include the followings: thoracic esophageal squamous cell carcinoma, adenosquamous carcinoma, or basaloid squamous cell carcinoma, reception of NAC (at most three courses of DCF or two courses of Cisplatin plus 5-FU) and a diagnosis of clinical stage I (except for T1N0), II, III, IVA (except for T4), or IVB (only supraclavicular lymph node metastasis except for T4), right thoracoabdominal esophagectomy with D2 or higher lymph node dissection, R0 resection was performed, residual pathological disease in the resected specimens [ypStage T0-4aN1-3M0, T1-4aN0M0, T0-4aNanyM1 (only supraclavicular lymph node metastasis)], aged 18-80 years, ECOG performance status 0 or 1, and adequate organ function. The primary endpoint is relapse-free survival (RFS), and the secondary endpoints are disease-free survival, overall survival, adverse events, and incremental cost-effectiveness ratio. After the confirmation of patient eligibility including pathological diagnosis, patients will be randomized (1:1:1) to either the observation alone arm, adjuvant nivolumab arm (240 mg/body every two weeks or 480 mg/bodies every four weeks for one year), or adjuvant S-1 arm (four courses of twice-daily oral S-1 for four weeks followed by two weeks of rest for 6 month, three dose levels of S-1 are set according to the patient’s body surface area (BSA), as follows: BSA < 1.25 m2, 40 mg; BSA 1.25–1.50 m2, 50 mg; BSA > 1.50 m2, 60 mg, twice a day). We assumed a three-year RFS of 60% in the observation alone arm and expected an 11% increase for the nivolumab arm and the S-1 arm (HR = 0.67). The total required sample size was calculated as 435 (145 per arm) to achieve the desired power of 70% for each pair comparison (observation alone arm vs. the nivolumab arm and observation alone arm vs. S-1 arm) with an overall one-sided alpha of 5%, an accrual period of 5.5 years, and a follow-up period of three years. The planned sample size was set at 450 patients (150 patients per arm). This trial has been registered in the Japan Registry of Clinical Trials as jRCTs031230219. This trial was initiated in July 2023, and the first patient was enrolled in September 2023. Clinical trial information: jRCTs031230219 .