Sample size determination and re‐estimation for matched pair designs with multiple binary endpoints

Estimation of sample size in clinical trials requires knowledge of parameters that involve the treatment effect and variability, which are usually uncertain to medical researchers. The recent release within the European Union of a Note for Guidance from the Commission for Proprietary Medical Products (CPMP) highlights the importance of this issue. Most previous papers considered the case of continuous response variables that assume a normal distribution; some regarded the portion up to the interim stage as an 'internal pilot study' and required unblinding. In this paper, our concern is with the case of binary response variables, which is more difficult than the normal case since the mean and variance are not distinct parameters. We offer a design with a simple stratification strategy that enables us to verify and update the assumption of the response rates given initially in the protocol. The design provides a method to re-estimate the sample size based on interim data while preserving the trial's blinding. An illustrative numerical example and simulation results show slight effect on the type I error rate and the decision making characteristics on sample size adjustment.

ABSTRACTSample size estimation (SSE) is an important issue in the planning of clinical studies. While larger studies are likely to have sufficient power, it may be unethical to expose more patients than necessary to answer a scientific question. Budget considerations may also cause one to limit the study to an adequate size to answer the question at hand. Typically at the planning stage, a statistically based justification for sample size is provided. An effective sample size is usually planned under a pre-specified type I error rate, a desired power under a particular alternative and variability associated with the observations recorded. The nuisance parameter such as the variance is unknown in practice. Thus, information from a preliminary pilot study is often used to estimate the variance. However, calculating the sample size based on the estimated nuisance parameter may not be stable. Sample size re-estimation (SSR) at the interim analysis may provide an opportunity to re-evaluate the uncertainties using accrued data and continue the trial with an updated sample size. This article evaluates a proposed SSR method based on controlling the variability of nuisance parameter. A numerical study is used to assess the performance of proposed method with respect to the control of type I error. The proposed method and concepts could be extended to SSR approaches with respect to other criteria, such as maintaining effect size, achieving conditional power, and reaching a desired reproducibility probability.

When designing a clinical trial, a number of design features including the sample size have to be decided upon. The sample size calculation usually requires some discussion of relevant effect sizes and information about nuisance parameters such as standard deviations or overall event rates, with nuisance parameters being estimated from previous studies. Using a novel endpoint or moving into a new indication, no or only very limited information might be available and the sample size calculation is therefore subject to considerable uncertainty. Internal pilot study designs that allow sample size reestimation midcourse of the ongoing study have been proposed to make trials more robust to misspecifications of nuisance parameters in the planning phase. In this article we present the design of a recently completed randomized active controlled trial in dermatology as a case study. Furthermore, we demonstrate how type I error rate control can be achieved when testing for noninferiority and explore operating characteristics such as power and sample size distributions through simulations motivated by the case study. Finally, relevant regulatory guidelines on sample size reestimation are referred to.

When a generic drug is developed, it is important to assess the equivalence of therapeutic efficacy between the new and the standard drugs. Although the number of publications on testing equivalence and its relevant sample size determination is numerous, the discussion on sample size determination for a desired power of detecting equivalence under a randomized clinical trial (RCT) with non-compliance and missing outcomes is limited. In this paper, we derive under the compound exclusion restriction model the maximum likelihood estimator (MLE) for the ratio of probabilities of response among compliers between two treatments in a RCT with both non-compliance and missing outcomes. Using the MLE with the logarithmic transformation, we develop an asymptotic test procedure for assessing equivalence and find that this test procedure can perform well with respect to type I error based on Monte Carlo simulation. We further develop a sample size calculation formula for a desired power of detecting equivalence at a nominal alpha-level. To evaluate the accuracy of the sample size calculation formula, we apply Monte Carlo simulation again to calculate the simulated power of the proposed test procedure corresponding to the resulting sample size for a desired power of 80 per cent at 0.05 level in a variety of situations. We also include a discussion on determining the optimal ratio of sample size allocation subject to a desired power to minimize a linear cost function and provide a sensitivity analysis of the sample size formula developed here under an alterative model with missing at random.

This manuscript consists of two topics. Firstly, we explore the utility of internal pilot study (IPS) approach for reestimating sample size at an interim stage when a reliable estimate of the nuisance shape parameter of the Weibull distribution for modeling survival data is unavailable during the planning phase of a study. Although IPS approach can help rescue the study power, it is noted that the adjusted sample size can be as much as twice the initially planned sample size, which may put substantial practical constraints to continue the study. Secondly, we discuss Bayesian predictive probability for conducting interim analyses to obtain preliminary evidence of efficacy or futility of an experimental treatment warranting early termination of a clinical trial. In the context of single-arm clinical trials with time-to-event endpoints following Weibull distribution, we present the calculation of the Bayesian predictive probability when the shape parameter of the Weibull distribution is unknown. Based on the data accumulated at the interim, we propose two approaches which rely on the posterior mode or the entire posterior distribution of the shape parameter. To account for uncertainty in the shape parameter, it is recommended to incorporate its entire posterior distribution in our calculation.

This paper revisits the problem of power analysis and sample size calculations in randomised experiments, with a focus on settings where inference on average treatment effects is conducted using randomisation tests. While standard formulas based on the two‐sample ‐test are widely used in practice, we show that these calculations may yield misleading results when directly applied to randomisation‐based inference – unless certain assumptions are met. We demonstrate that differences in potential outcome variances or unequal group sizes can distort the behaviour of the randomisation test, leading to incorrect power and flawed sample size calculations. However, a simple adjustment – studentising the test statistic – restores the validity of the randomisation test in large samples. This adjustment allows researchers to safely apply standard power and sample size formulas, even when using randomisation inference. We extend these results to a range of experimental designs commonly used in applied economics, including stratified randomisation, matched pairs and cluster‐randomised trials. Throughout, we provide practical guidance to help researchers ensure that their design‐stage calculations remain valid under the inferential methods they plan to use.

A two-stage Bayesian design with sample size reestimation and subgroup analysis for phase II binary response trials

Objectives To assess the completeness of reporting of sample size determinations in unpublished research protocols and to develop guidance for research ethics committees and for statisticians advising these committees.Design Review...

The number of samples needed to identify significant effects is a key question in biomedical studies, with consequences on experimental designs, costs and potential discoveries. In metabolic phenotyping studies, sample size determination remains a complex step. This is due particularly to the multiple hypothesis-testing framework and the top-down hypothesis-free approach, with no a priori known metabolic target. Until now, there was no standard procedure available to address this purpose. In this review, we discuss sample size estimation procedures for metabolic phenotyping studies. We release an automated implementation of the Data-driven Sample size Determination (DSD) algorithm for MATLAB and GNU Octave. Original research concerning DSD was published elsewhere. DSD allows the determination of an optimized sample size in metabolic phenotyping studies. The procedure uses analytical data only from a small pilot cohort to generate an expanded data set. The statistical recoupling of variables procedure is used to identify metabolic variables, and their intensity distributions are estimated by Kernel smoothing or log-normal density fitting. Statistically significant metabolic variations are evaluated using the Benjamini-Yekutieli correction and processed for data sets of various sizes. Optimal sample size determination is achieved in a context of biomarker discovery (at least one statistically significant variation) or metabolic exploration (a maximum of statistically significant variations). DSD toolbox is encoded in MATLAB R2008A (Mathworks, Natick, MA) for Kernel and log-normal estimates, and in GNU Octave for log-normal estimates (Kernel density estimates are not robust enough in GNU octave). It is available at http://www.prabi.fr/redmine/projects/dsd/repository, with a tutorial at http://www.prabi.fr/redmine/projects/dsd/wiki.

Sample size determination is a critical aspect of biomedical research, as it dictates the number of samples needed for a successful experiment that can yield generalizable results to the broader population. This paper outlines the methodology for calculating sample sizes in various categories of biomedical and clinical research, encompassing cross-sectional surveys, comparative studies, case–control studies, cohort studies, and animal studies. Detailed examples are provided for each category. The determination of an appropriate sample size holds significant importance from scientific, ethical, and resource allocation perspectives. It is noteworthy that research outcomes are often directly influenced by the accuracy of sample size calculations. A robust sample size calculation serves as a cornerstone for researchers, enabling them to draw precise inferences with a high level of precision across a spectrum of biomedical and clinical investigations.

Sample size determination under an exponential model in the presence of a confounder and type I censoring

To the Editor:—We write to make the case that the practice of providing a priori sample size calculations, recently endorsed in an Anesthesiology editorial, 1is in fact undesirable. Presentation of confidence intervals serves the same purpose, but is superior because it more accurately reflects the actual data, is simpler to present, addresses uncertainty more directly, and encourages more careful interpretation of results. The clinical trial report 2lauded in the editorial in fact serves to illustrate the drawbacks of sample size calculation as a data analysis tool. The a priori calculation presented is based on assumptions about length of stay (normally distributed with a SD of 4.5 days) that did not hold in the actual data, an analysis (comparison of mean length of stay between two groups by t test) that was not presented, and a sample size that was not attained. It therefore does not help the reader interpret the results, which is the proper goal when reporting on a study that has been completed. The post hoc power calculation presented retains most of these deficiencies, and therefore does not help the reader to assess the strength of evidence against a 1.0-day mean advantage for one treatment versus another. In contrast, a confidence interval for the difference in means would directly address this issue. Although the presence of outliers would require a bootstrapping method 3to obtain a valid confidence interval for a difference in means, this bit of extra effort is certainly worthwhile for the central issue of a study, and in any case, much better than relying on convoluted reasoning with invalid power approximations.Perhaps the worst aspect of reporting sample size or power calculations is that it encourages interpretation of studies’ results based only on P values, in particular the widespread fallacy of interpreting P > 0.05 as proving the null hypothesis. The other article 4cited by the editorial provides a glaring example of this type of reasoning, concluding that reporting of sample size calculations did not change over time in any journal but did increase overall (see their fig. 2). Returning to the clinical trial report, consider the statement that death rates “were similar” in the four subgroups. While this is an accurate characterization of what was actually observed, unsophisticated readers are liable to interpret this (contrary to the authors’ intentions) to mean that the study found strong evidence against any substantial difference in death rates. In fact, the exact 595% confidence interval around the odds ratio for death comparing intravenous versus epidural postoperative analgesia goes from 0.36 to 5.4, which is wide enough to make clear to most readers that this study by itself provides only very weak evidence against a clinically important difference in death rates.We urge reviewers, editors, and quality studies to give authors full credit for providing confidence intervals instead of sample size calculations in reports of completed studies. Indeed, for the reasons illustrated here, it would be best to discourage the practice of using sample size and power calculations as substitutes for more direct assessment of uncertainty using confidence intervals.

Background: Sample size calculations are an important tool for planning epidemiological studies. Large sample sizes are often required in Mendelian randomization investigations.Methods and results: Resources are provided for investigators to perform sample size and power calculations for Mendelian randomization with a binary outcome. We initially provide formulae for the continuous outcome case, and then analogous formulae for the binary outcome case. The formulae are valid for a single instrumental variable, which may be a single genetic variant or an allele score comprising multiple variants. Graphs are provided to give the required sample size for 80% power for given values of the causal effect of the risk factor on the outcome and of the squared correlation between the risk factor and instrumental variable. R code and an online calculator tool are made available for calculating the sample size needed for a chosen power level given these parameters, as well as the power given the chosen sample size and these parameters.Conclusions: The sample size required for a given power of Mendelian randomization investigation depends greatly on the proportion of variance in the risk factor explained by the instrumental variable. The inclusion of multiple variants into an allele score to explain more of the variance in the risk factor will improve power, however care must be taken not to introduce bias by the inclusion of invalid variants.

Ventilator-associated pneumonia (VAP) is an important healthcare acquired infection, which is associated with high morbidity and mortality. Conducting conventional randomised controlled trials (RCTs) on VAP prevention is often challenging, due to low numbers of eligible patients and events per site, especially for pathogen-specific interventions. We explored how group sequential designs (GSD) and sample size re-estimation (SSR) trial designs could improve RCT efficiency in simulated superiority trials to prevent VAP. Simulations were informed using data from the prospective observational Hospital Network Study - Preparation for a Randomised Evaluation of anti-Pneumonia Strategies (HONEST-PREPS). We tested the impact of different GSD and SSR designs on expected sample size (considering early stopping) and maximum sample size (no early stopping). We varied the type of stopping boundary, number and timepoint of interim analyses, and assumed and true prevention effect. We applied time-to-event analyses, with effect estimates expressed as hazard ratios, for the primary endpoint. The estimated 28-day cumulative incidence of VAP in HONEST-PREPS was 15.5%. For a 30% reduction in VAP (hazard ratio of 0.68), a standard RCT (power 80%) would require a sample size of 1291 patients. For GSD, Pocock boundariesresult in a smaller expected sample size (E[N] = 1128), but a larger maximum sample size (max(N) = 1578) than O'Brien Fleming boundaries (E[N] = 1170 and max(N) = 1389), when utilising the optimal placement of a single interim analysis, 48% and 64% of the maximum number of events for Pocock and O'Brien Fleming boundaries, respectively. SSR is more efficient compared to GSD when the incorrect prevention effect is initially used to plan the trial, as it maintains a power closer to the pre-specified desired power without substantial impact on the expected sample size. GSD and SSR are effective adaptive designs, preferable to fixed RCTs in a superiority trial comparing the effectiveness of an investigational intervention with a standard of care in preventing VAP among critically ill, ventilated patients. They can reduce the expected sample size between 9% and 12% and should be considered at the trial design stage.

This paper discusses the statistical principles, methods, and software programs used to calculate sample size. In addition, it reviews the practical challenges faced in calculating sample size. We show that because of such challenges, statistical calculations often do not provide us with a clear-cut number for the study sample size; rather they suggest a range of reasonable numbers. The paper also discusses several important nonstatistical considerations in determination of sample size, such as novelty of the study and availability of resources.