Blinded sample size recalculation in clinical trials with binary composite endpoints

Baseline adjusted analyses are commonly encountered in practice, and regulatory guidelines endorse this practice. Sample size calculations for this kind of analyses require knowledge of the magnitude of nuisance parameters that are usually not given when the results of clinical trials are reported in the literature. It is therefore quite natural to start with a preliminary calculated sample size based on the sparse information available in the planning phase and to re-estimate the value of the nuisance parameters (and with it the sample size) when a portion of the planned number of patients have completed the study. We investigate the characteristics of this internal pilot study design when an analysis of covariance with normally distributed outcome and one random covariate is applied. For this purpose we first assess the accuracy of four approximate sample size formulae within the fixed sample size design. Then the performance of the recalculation procedure with respect to its actual Type I error rate and power characteristics is examined. The results of simulation studies show that this approach has favorable properties with respect to the Type I error rate and power. Together with its simplicity, these features should make it attractive for practical application.

When designing a clinical trial, a number of design features including the sample size have to be decided upon. The sample size calculation usually requires some discussion of relevant effect sizes and information about nuisance parameters such as standard deviations or overall event rates, with nuisance parameters being estimated from previous studies. Using a novel endpoint or moving into a new indication, no or only very limited information might be available and the sample size calculation is therefore subject to considerable uncertainty. Internal pilot study designs that allow sample size reestimation midcourse of the ongoing study have been proposed to make trials more robust to misspecifications of nuisance parameters in the planning phase. In this article we present the design of a recently completed randomized active controlled trial in dermatology as a case study. Furthermore, we demonstrate how type I error rate control can be achieved when testing for noninferiority and explore operating characteristics such as power and sample size distributions through simulations motivated by the case study. Finally, relevant regulatory guidelines on sample size reestimation are referred to.

Prevention of decompensation is a primary therapeutic target in patients with compensated cirrhosis (CC). However, a major problem is the large sample size and long follow-up required to demonstrate a significant treatment effect because of the relatively low baseline risk. For this reason, it has been recently suggested that ordinal outcomes may be used in this area to gain power and reduce sample size. The aim of this study was to assess the applicability of ordinal outcomes in cirrhosis. An inception cohort of 202 patients with CC (no ascites, gastrointestinal bleeding, encephalopathy, or jaundice) without esophageal varices was included, and 5-year outcome is reported. Etiology was mostly viral and alcoholic, and there were no dropouts. Ordinal outcome was set according to six grades with a previously established prognostic ordinality: grade 1=no disease progression; grade 2=development of varices; grade 3=bleeding alone; grade 4=nonbleeding single decompensation; grade 5=more than one decompensating event; and grade 6=death. At the 60-month time point, patients were distributed in grades 1 through 6 as follows: 129, 43, 2, 7, 5, and 16, respectively. Emulation of a clinical trial performed by dividing patients based on baseline platelet count into two groups (cutoff, 150×109 /L) demonstrated a statistically significant outcome difference between groups when using ordinal outcomes not detectable by binary logistic or chi-square or time-to-event analyses. Additionally, using ordinal outcomes in a hypothetical study to prevent decompensation resulted in sample-size estimates 3-to 4-fold lower than using a binary composite endpoint. Compared to traditional binary outcomes, the use of ordinal outcomes in trials of cirrhosis decompensation may provide more power and thus may require a smaller sample size.

Considerations of sample size in medical research

6576 Background: The aim of phase II trials is to determine if a new treatment is promising for further testing in confirmatory clinical trials. Most phase II clinical trials are designed as single-arm trials using a binary outcome with or without interim monitoring for early stopping. In this context, we propose a Bayesian adaptive design denoted as PSSD, predictive sample size selection design (Statistics in Medicine 2012;31:4243-4254). Methods: The design allows for sample size selection followed by any planned interim analyses for early stopping of a trial, together with sample size determination before starting the trial. In the PSSD, we determined the sample size using the predictive probability criterion with two kinds of prior distributions, that is, an ‘analysis prior’ used to compute posterior probabilities and a ‘design prior’ used to obtain prior predictive distributions. In the sample size determination, we provide two sample sizes, that is, N and Nmax, using two types of design priors. At each interim analysis, we calculate the predictive probability of achieving a successful result at the end of the trial using analysis prior in order to stop the trial in case of low or high efficacy, and we select an optimal sample size, that is, either N or Nmax as needed, on the basis of the predictive probabilities. Results: We investigated the operating characteristics through simulation studies, and the PSSD retrospectively applies to a lung cancer clinical trial. As the number of interim looks increases, the probability of type I errors slightly decreases, and that of type II errors increases. The type I error probabilities of the probabilities of the proposed PSSD are almost similar to those of the non-adaptive design. The type II error probabilities in the PSSD are between those of the two fixed sample size (N or Nmax) designs. Conclusions: From a practical standpoint, the proposed design could be useful in phase II single-arm clinical trials with a binary endpoint. In the near future, this approach will be implemented in actual clinical trials to assess its usefulness and to extend it to more complicated clinical trials.

Better design leads to better results - Importance of virological outcome design in clinical trials for antiviral treatment of coronavirus disease 2019.

At the most general level, sample size refers to the number of participants (typically human or animal subjects) who provide data for a scientific experiment or study. Selecting an appropriate sample size is a fundamental feature of a well-designed study, and, as the title suggests, sample size can be carefully planned ahead of time. Sample size planning is important because the sample size of a study has implications for statistical power as well as accuracy. In conventional terms, statistical power reflects the probability of rejecting the null hypothesis, assuming that the purported effect is non-null (non-zero) in reality. Although power can be treated as a function of unknown parameter values, power is often defined conditionally on a specific value of population effect size. (On a technical level, power is conditional upon the population noncentrality parameter, which is a combination of sample size and effect size, but relying on effect size as a proxy is generally acceptable.) Power is calculated from the statistical significance threshold (otherwise known as the nominal Type I error rate or alpha-level), the population effect size, and the sample size. Because of the connection between sample size and power, sample size planning is sometimes called a priori power analysis. For example, if a new therapy developed to treat depression is truly beneficial, the sample size of a study assessing this therapy should be large enough to demonstrate the beneficial outcome as statistically significant. Accuracy reflects how close an estimate is to the parameter it aims to estimate. Accurate estimates are expressed with narrow confidence intervals (small margins of error). Sample size also has a connection to accuracy. For example, if the investigators in the previous example want to estimate how much the therapy reduces depressive symptoms, the sample size should be large enough so that the effect size estimate derived from the study is an accurate estimate of the true magnitude of the treatment effect. An underlying challenge to sample size planning for both goals is that the population effect size is unknown. An important corollary is that whether a particular sample size provides appropriate power (and in some cases, accuracy) depends on the population effect size, and as such, sample size cannot completely be judged without context. Although there are sometimes limitations to sample size, such as when working with a limited budget or specialized populations, and although sample size is often selected based on convenience or historical precedent, when possible it is desirable to directly plan the study sample size at the study design phase. Explicitly planning sample size helps to ensure that the sample size will be effective in meeting the investigators’ goals and answering the scientific questions of interest.

Phase 2 clinical trials typically rely on a single primary endpoint, yet in many settings, treatment efficacy must be demonstrated across multiple co-primary endpoints. Such settings require intersection-union hypothesis testing, in which the global null hypothesis is rejected only when all individual component hypotheses are rejected. Existing phase 2 designs for binary endpoints are largely restricted to single-arm or single-endpoint settings, or rely on large-sample normal approximations for two-arm trials, which may be inadequate when sample sizes are small. We developed exact and simulation-based methods for designing two-arm, two-stage phase 2 trials with two co-primary binary endpoints using the bivariate binomial distribution, allowing different correlations between endpoints across treatment arms. The proposed framework identifies optimal designs that minimize the expected sample size and minimax designs that minimize the maximum sample size, while allowing early termination for futility at the first stage. For small sample sizes (approximately per arm), the exact method is computationally feasible and yields analytically exact operating characteristics under the assumed bivariate binomial distribution. For larger sample sizes, however, exact computations become intensive. To address this, we evaluated a simulation-based approach and a normal approximation by comparing their performance against the exact method. The simulation approach selected designs that closely matched those from the exact method, whereas the normal approximation often deviated. Therefore, we recommend the simulation-based method as a computationally efficient and accurate alternative for moderate-to-large sample size trials.

Meta-analysis plays an important role in the analysis and interpretation of clinical trials in medicine and of trials in the social sciences but is of importance in other fields (e.g., particle physics 1) as well. In 2001, Hartung and Knapp 2, 3 introduced a new approach to test for a nonzero treatment effect in a meta-analysis of k studies. Hartung and Knapp 2, 3 suggest to use the random effects estimate according to DerSimonian and Laird 4 and propose a variance estimator q so that the test statistics for the treatment effect is t distributed with k − 1 degrees of freedom. In their paper on dichotomous endpoints, results of a simulation study with 6 and 12 studies illustrate for risk differences, log relative risks and log odds ratios, the excellent properties regarding control of the type I error, and the achieved power 2. They investigate different sample sizes in each study, and different amounts of heterogeneity between studies and compare their new approach (Hartung and Knapp approach (HK)) with the fixed effects approach (FE) and the classical random effects approach by DerSimonian and Laird (DL). It can be clearly seen that, with increasing heterogeneity, the FE as well as the DL does not control the type I error rate, while the HK keeps the type I error rate in nearly every situation and in every scale. Advantages and disadvantages of the two standard approaches and respective test statistics have been extensively discussed (e.g., 5-7). While it is well known that the FE is too liberal in the presence of heterogeneity, the DL is often thought to be rather conservative because heterogeneity is incorporated into the standard error of the estimate for the treatment effect and this should lead to larger confidence intervals and smaller test statistics for the treatment effect (8 chapter 9.4.4.3). This was disproved among others by Ziegler and Victor 7, who observed in situations with increasing heterogeneity severe inflation of the type I error for the DerSimonian and Laird test statistic. Notably, the asymptotic properties of this approach will be valid, if both the number of studies and the number of patients per study are large enough (8 chapter 9.54, 9, 10). Although power issues of meta-analysis tests have received some interest, comparisons between the approaches and the situation with two studies were not the main interest 11, 12. Borenstein et al. (10, pp. 363/364) recommend the random effects approach in general for meta-analysis and do not recommend meta-analyses of small numbers of studies. However, meta-analyses of few and of even only two trials are of importance. In drug licensing in many instances, two successful phase III clinical trials have to be submitted as pivotal evidence for drug licensing 13, and summarizing the findings of these studies is required according to the International Conference on Harmonisation guidelines E9 and M4E (14, 15). It is stated that ‘An overall summary and synthesis of the evidence on safety and efficacy from all the reported clinical trials is required for a marketing application [...]. This may be accompanied, when appropriate, by a statistical combination of results’ (14, p. 31). For the summary, ‘The use of meta-analytic techniques to combine these estimates is often a useful addition, because it allows a more precise overall estimate of the size of the treatment effects to be generated, and provides a complete and concise summary of the results of the trials’ (14, p. 32). While in standard drug development, this summary will include usually more than two studies; in rare diseases for the same intervention, barely ever more than two studies are available because of the limited number of patients. Likewise, decision making in the context of health technology assessment is based on systematic reviews and meta-analyses. Often in practice, only two studies are considered homogeneous enough from clinical grounds to be included into a meta-analysis and then form the basis for decision making about reimbursement 16. Despite the fact that meta-analysis is non-experimental observational (secondary) research 17 and p-values should be interpreted with caution, meta-analyses of randomized clinical trials are termed highest-level information in evidence-based medicine and are the recommended basis for decision making 18. As statistical significance plays an important role in the assessment of the meta-analysis, it is mandatory to understand the statistical properties of the relevant methodology also in a situation, where only two clinical trials are included into a meta-analysis. We found Cochrane reviews including meta-analyses with two studies only, which are considered for evidence-based decision making even in the presence of a large amount of heterogeneity (I2≈75%) 19-21 We repeated the simulation study for dichotomous endpoints of Hartung and Knapp 2 with programs written in R 3.1.0 22 to compare the statistical properties of the FE, the DL, and the HK for testing the overall treatment effect θ (H0: θ = 0) in a situation with two to six clinical trials. We considered scenarios under the null and alternative hypothesis for the treatment effect with and without underlying heterogeneity. We present the findings for the odds ratio with pC=0.2 and did vary probability of success in the treatment group pT to investigate the type I error and the power characteristics. The total sample size per meta-analysis was kept constant in the different scenarios (n = 480) and n/k number of patients per study to clearly demonstrate the effect of the number of included studies on power and type I error of the various approaches. Likewise, we attempted to avoid problems with zero cell counts or extremely low event rates that may impact on type I error and power as well. I2 was used to describe heterogeneity because thresholds have been published (low: I2=25%, moderate: I2=50%, and high: I2=75%) 23 for the quantification of the degree of heterogeneity with this measure. We termed I2≤15% negligible, and this refers to simulations assuming no heterogeneity (i.e., the fixed effects model). Table 1 summarizes the results of our simulation study. The well-known anticonservative behavior of the FE and the DL in the presence of even low heterogeneity is visible for small numbers of studies in the meta-analysis. Particularly for the FE, the increase in the type I error is pronounced. With more than four studies even in situations with substantial heterogeneity, the HK perfectly controls the type I error. There is almost no impact on the power of the test in situations with no or low heterogeneity, and overall, it seems as if the only price to be paid for an increased heterogeneity is a reduced power of the test. This is in strong contrast to the situation with only two studies. Again, the HK perfectly controls the prespecified type I error. However, even in a homogeneous situation, the power of the meta-analysis test was lower than 15% in situations where the power of the FE and the DL approximates 70% and 60%, respectively. In the presence of even low heterogeneity with the HK, there is not much chance to arrive at a positive conclusion even with substantial treatment effects. Figure 1 summarizes the main finding of our simulation study with k = 2 and 6 studies impressively. In the homogeneous situation with two studies, the DL and even better the FE can be used to efficiently base conclusions on a meta-analysis. In contrast, already with mild to moderate heterogeneity, both standard tests severely violate the prespecified type I error, and there is a high risk of false positive conclusion with the classical approaches. This has major implications for decision making in drug licensing as well. We have noted previously that a meta-analysis can be confirmatory if a drug development program was designed to include a preplanned meta-analysis of the two pivotal trials 24. As an example, thrombosis prophylaxis was discussed in the paper by Koch and Röhmel 24, where venous thromboembolism is accepted as primary endpoint in the pivotal trials. In case when both pivotal trials are successful, they can be combined to demonstrate a positive impact on, for example, mortality. This can be preplanned as a hierarchical testing procedure: first, both pivotal trials will be assessed individually before confirmatory conclusions will be based on the meta-analysis. As explained, neither the FE, nor the DL, nor the HK can be the methodology to be recommended for a priori planning in this sensitive area unless any indication for heterogeneity is taken as a trigger not to combine studies in a meta-analysis at all. It is our belief that not enough emphasis has been given to this finding in the original paper and the important role of heterogeneity is not acknowledged enough in the discussion of findings from meta-analyses, in general. This work has been funded partly by the FP7-HEALTH-2013-INNOVATION-1 project Advances in Small Trials Design for Regulatory Innovation and Excellence (ASTERIX) Grant Agreement No. 603160. We thank the editors and reviewers for useful comments to tailor the main messages.

This manuscript consists of two topics. Firstly, we explore the utility of internal pilot study (IPS) approach for reestimating sample size at an interim stage when a reliable estimate of the nuisance shape parameter of the Weibull distribution for modeling survival data is unavailable during the planning phase of a study. Although IPS approach can help rescue the study power, it is noted that the adjusted sample size can be as much as twice the initially planned sample size, which may put substantial practical constraints to continue the study. Secondly, we discuss Bayesian predictive probability for conducting interim analyses to obtain preliminary evidence of efficacy or futility of an experimental treatment warranting early termination of a clinical trial. In the context of single-arm clinical trials with time-to-event endpoints following Weibull distribution, we present the calculation of the Bayesian predictive probability when the shape parameter of the Weibull distribution is unknown. Based on the data accumulated at the interim, we propose two approaches which rely on the posterior mode or the entire posterior distribution of the shape parameter. To account for uncertainty in the shape parameter, it is recommended to incorporate its entire posterior distribution in our calculation.

Equal allocation is accepted almost universally in the design of randomised clinical trials and it is often assumed that this approach provides the most efficient use of available resources. The design of a phase II study often depend on a binary endpoint with assessments of efficacy made using an odds ratio. In a trial setting however, precision about this odds ratio will only be optimal under equal allocation when there is no difference in the response rates between two treatment arms. A result which typically is of limited clinical interest. For a clinical trial with response rates are px and py in the experimental and control arm respectively, allocation proportions are derived that seek to maximise the precision about an odds ratio in settings where difference between response rates are expected. Sample size calculations are performed and compared to study designs using equal allocation. Sample size calculations based on the exact methods of Jung and Sargent [8] show that under the same type 1 error rate and power, the required sample sizes using unequal allocation ratios are smaller than those where equal allocation is used. This discrepancy is greater as the control mean response rate tends towards 0 or 1 while the relative treatment effect remain fixed. Trialist involved in the design of phase II studies should take account of possible savings in sample size that may be gained by unequal allocation of patients.

When designing a randomized clinical trial to compare two treatments, the sample size required to have desired power with a specified type 1 error depends on the hypothesis testing procedure. With a binary endpoint (e.g., response), the trial results can be displayed in a 2 × 2 table. If one does the analysis conditional on the number of positive responses, then using Fisher's exact test has an actual type 1 error less than or equal to the specified nominal type 1 error. Alternatively, one can use one of many unconditional "exact" tests that also preserve the type 1 error and are less conservative than Fisher's exact test. In particular, the unconditional test of Boschloo is always at least as powerful as Fisher's exact test, leading to smaller required sample sizes for clinical trials. However, many statisticians have argued over the years that the conditional analysis with Fisher's exact test is the only appropriate procedure. Since having smaller clinical trials is an extremely important consideration, we review the general arguments given for the conditional analysis of a 2 × 2 table in the context of a randomized clinical trial. We find the arguments not relevant in this context, or, if relevant, not completely convincing, suggesting the sample-size advantage of the unconditional tests should lead to their recommended use. We also briefly suggest that since designers of clinical trials practically always have target null and alternative response rates, there is the possibility of using this information to improve the power of the unconditional tests.

High Rate of Exclusion of HIV Infected Patients from Modern Lymphoma Studies: An Analysis of Current United States Therapeutic Trials

Innovation in the Design of Clinical Trials: Necessary for Innovation in Medicine.

Including long- and short-term data in blinded sample size recalculation for binary endpoints