Using Bayesian adaptive designs to improve phase III trials: a respiratory care example

BackgroundBayesian adaptive methods are increasingly being used to design clinical trials and offer several advantages over traditional approaches. Decisions at analysis points are usually based on the posterior distribution of the treatment effect. However, there is some confusion as to whether control of type I error is required for Bayesian designs as this is a frequentist concept.MethodsWe discuss the arguments for and against adjusting for multiplicities in Bayesian trials with interim analyses. With two case studies we illustrate the effect of including interim analyses on type I/II error rates in Bayesian clinical trials where no adjustments for multiplicities are made. We propose several approaches to control type I error, and also alternative methods for decision-making in Bayesian clinical trials.ResultsIn both case studies we demonstrated that the type I error was inflated in the Bayesian adaptive designs through incorporation of interim analyses that allowed early stopping for efficacy and without adjustments to account for multiplicity. Incorporation of early stopping for efficacy also increased the power in some instances. An increase in the number of interim analyses that only allowed early stopping for futility decreased the type I error, but also decreased power. An increase in the number of interim analyses that allowed for either early stopping for efficacy or futility generally increased type I error and decreased power.ConclusionsCurrently, regulators require demonstration of control of type I error for both frequentist and Bayesian adaptive designs, particularly for late-phase trials. To demonstrate control of type I error in Bayesian adaptive designs, adjustments to the stopping boundaries are usually required for designs that allow for early stopping for efficacy as the number of analyses increase. If the designs only allow for early stopping for futility then adjustments to the stopping boundaries are not needed to control type I error. If one instead uses a strict Bayesian approach, which is currently more accepted in the design and analysis of exploratory trials, then type I errors could be ignored and the designs could instead focus on the posterior probabilities of treatment effects of clinically-relevant values.

This article provides a regulatory view of the design of clinical trials using Bayesian statistics and adaptive methods. The foci are the similarities and differences between Bayesian and adaptive designs for clinical studies for medical devices and the contrast with trials of pharmaceutical drugs. The critical role of the U.S. Food and Drug Administration in such designs is highlighted. The previous device experience with Bayesian trials has provided some important insight into the planning of adaptive trials. This article discusses both frequentist and Bayesian adaptive designs and presents several real examples of designs for medical device studies. The crucial role that simulations play in assessing the operating characteristics of the designs is emphasized. The unique advantage of predictive modeling in the Bayesian adaptive design is explored. For adaptive designs there are challenging problems including operational bias and logistical issues. The reporting of both Bayesian and adaptive trials is discussed.

The design of phase I studies is often challenging, because of limited evidence to inform study protocols. Adaptive designs are now well established in cancer but much less so in other clinical areas. A phase I study to assess the safety, pharmacokinetic profile and antiretroviral efficacy of C34‐PEG4‐Chol, a novel peptide fusion inhibitor for the treatment of HIV infection, has been set up with Medical Research Council funding. During the study workup, Bayesian adaptive designs based on the continual reassessment method were compared with a more standard rule‐based design, with the aim of choosing a design that would maximise the scientific information gained from the study. The process of specifying and evaluating the design options was time consuming and required the active involvement of all members of the trial's protocol development team. However, the effort was worthwhile as the originally proposed rule‐based design has been replaced by a more efficient Bayesian adaptive design. While the outcome to be modelled, design details and evaluation criteria are trial specific, the principles behind their selection are general. This case study illustrates the steps required to establish a design in a novel context. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd

BackgroundPhase III trials often require large sample sizes, leading to high costs and delays in clinical decision-making. Group sequential designs can improve trial efficiency by allowing for early stopping for efficacy and/or futility and thus may decrease the sample size, trial duration and associated costs. Bayesian approaches may offer additional benefits by incorporating previous information into the analyses and using decision criteria that are more practically relevant than those used in frequentist approaches. Frequentist group sequential designs have often been used for phase III studies, but the use of Bayesian group sequential designs is less common. The aim of this work was to explore how Bayesian group sequential designs could be constructed for phase III trials conducted in emergency medicine.MethodsThe PARAMEDIC2 trial was a phase III randomised controlled trial that compared the use of adrenaline to placebo in out-of-hospital cardiac arrest patients on 30-day survival rates. It used a frequentist group sequential design to allow early stopping for efficacy or harm. We constructed several alternative Bayesian group sequential designs and studied their operating characteristics via simulation. We then virtually re-executed the trial by applying the Bayesian designs to the PARAMEDIC2 data to demonstrate what might have happened if these designs had been used in practice.ResultsWe produced three alternative Bayesian group sequential designs, each of which had greater than 90% power to detect the target treatment effect. A Bayesian design which performed interim analyses every 500 patients recruited produced the lowest average sample size. Using the alternative designs, the PARAMEDIC2 trial could have declared adrenaline superior for 30-day survival with approximately 1500 fewer patients.ConclusionsUsing the PARAMEDIC2 trial as a case study, we demonstrated how Bayesian group sequential designs can be constructed for phase III emergency medicine trials. The Bayesian framework enabled us to obtain efficient designs using decision criteria based on the probability of benefit or harm. It also enabled us to incorporate information from previous studies on the treatment effect via the prior distributions. We recommend the wider use of Bayesian approaches in phase III clinical trials.Trial registrationPARAMEDIC2 Trial registration ISRCTN, ISRCTN73485024. Registered 13 March 2014, http://www.isrctn.com/ISRCTN73485024

We are honored to be guest editors for this special issue of Applied Stochastic Models in Business and Industry, dedicated to Kathryn Chaloner's life and achievements. Contributions from her colleagues and friends in this volume are devoted to Kathryn's early research interest in Bayesian optimal design. She was a pioneer in this field and had a broad impact on the subsequent literature on experimental design. Kathryn was internationally known for her research in Bayesian Statistics, which included design of experiments, outlier detection, prior elicitation, and clinical trials, and for her advancements in the study of HIV/AIDS. She was an accomplished researcher, teacher, and mentor. She actively encouraged women and under-represented minority students to join the field of biostatistics and provided mentorship, support, and encouragement. The special issue contains seven papers. The first three papers present some of the recent work in Bayesian optimal designs for nonlinear models, binary responses, and experiments with adversarial components. This is followed by two articles on Bayesian designs in clinical trials. The last two articles of the special issue involve engineering design problems. The paper by Giovagnoli considers the binary response models of Chaloner and Larntz (1989, Journal of Statistical Planning and Inference, 21:191–208) and develops an adaptive version of A-optimal Bayesian designs that are revised as additional data become available during the experiment. Adaptive Bayesian compound designs is also discussed as another extension. Konstantinou and Dette develop approximate Bayesian D-optimal designs for nonlinear regression models that involve covariates that cannot be observed directly. Applications to specific nonlinear models including the exponential regression model are considered and characterizations of the D-optimal saturated designs are presented. Optimal sample size selection for experiments is considered by De Santis and Gubbiotti in an adversarial setting where the decision makers have different prior opinions while having common utility functions and data. Using a Bayesian decision approach with a quadratic loss function, the authors present results for the one-parameter exponential family and investigate effect of the priors on the optimal sample size. Müller, Xu, and Thall discuss advantages and limitations of Bayesian decision theoretic approach in design of clinical trials. In doing so, the authors present a specific case study and point out the limitations such as the computational difficulties involved in solving sequential problems as well as selection of utility functions capturing different stakeholder interests. The article by Ventz, Parmigiani, and Trippa also consider limitations of the Bayesian framework in design of clinical trials and propose a strategy to alleviate these. The proposed strategy involves combining Bayesian designs with frequentist metrics such as error rates, confidence intervals, etc. that are commonly used by medical community and regulatory agencies. The authors show that this can be achieved by an inclusion of frequentist constraints into the Bayesian formulation. The constrained decision theoretic framework is illustrated via several applications and computational algorithms are presented for implementing the proposed approach. Nakamura, Seepaul, Kadane, and Reeja-Jayan discuss design of experiments in materials science. The authors consider use of a batch-sequential experiment and a regression model to determine optimal levels of input variables for synthesis of ceramic materials. It is illustrated that the optimal settings produce more reliable results than other experiments. The final article by Polson and Soyer considers Bayesian design of accelerated life tests for reliability assessment. An augmented probability simulation (APS) approach using Lindley's (1976, Annals of Statistics, 4:1–10) conjugate utility functions is proposed to compute optimal designs in an efficient manner. The proposed approach is illustrated using single and multiple point designs. Use of particle based methods for APS is also discussed as an alternative to Markov chain Monte Carlo methods.

Quality improvement in health service programmes often involves introducing small, low-risk programmatic changes, such as modifying workflows, to incrementally improve outcomes, which accumulate over time to have significant overall gains in quality and efficiency. Although these changes are common in health services, they are rarely evaluated using statistically rigorous designs, partly because conventional randomised trials are perceived as inefficient for detecting modest effects. This study was motivated by a vision screening and referral programme and focuses on evaluating the modest but important impacts of low-risk QI interventions. To advance data-driven approaches for achieving quality improvement, we used a simulation study to explore the use of Bayesian adaptive trial designs to compare two variants of programmatic changes that yield small improvements in outcomes. The study examined key adaptive design features, including interim analysis frequency, prior specification, and early stopping rules for efficacy and equivalence. Changes in error rates, sample size, and bias were assessed across scenarios with small effect sizes ranging from 0% to 5%. The findings were used to configure an ideal trial design that prioritises rapid identification of the more effective programmatic change while minimizing the risk of adopting an inferior one. The recommended trial design incorporates a sceptical prior, a stringent stopping rule for efficacy, and a relaxed criteria to stop for equivalence. Under this design, a marginal improvement as small as 1% could be detected with high probability using considerably fewer participants than would be required under conventional, fixed-size randomised controlled trials. Bayesian adaptive trial designs offer a feasible approach for evaluating low-risk, incremental QI interventions in high-throughput service settings. Their use may support more efficient, data-driven decision-making when modest improvements are expected and the consequences of incorrect adoption are limited.

Current tuberculosis (TB) regimen development pathways are slow and in urgent need of innovation. We investigated novel phase IIc and seamless phase II/III trials utilizing multi-arm multi-stage and Bayesian response adaptive randomization trial designs to select promising combination regimens in a platform adaptive trial. Clinical trial simulation tools were built using predictive and validated parametric survival models of time to culture conversion (intermediate endpoint) and time to TB-related unfavorable outcome (final endpoint). This integrative clinical trial simulation tool was used to explore and optimize design parameters for aforementioned trial designs. Both multi-arm multi-stage and Bayesian response adaptive randomization designs were able to reliably graduate desirable regimens in ≥ 95% of trial simulations and reliably stop suboptimal regimens in ≥ 90% of trial simulations. Overall, adaptive phase IIc designs reduced patient enrollment by 17% and 25% with multi-arm multi-stage and Bayesian response adaptive randomization designs respectively compared to the conventional sequential approach, while seamless designs reduced study duration by 2.6 and 3.5years respectively (typically ≥ 8.5years for standard sequential approach). In this study, we demonstrate that adaptive trial designs are suitable for TB regimen development, and we provide plausible design parameters for a platform adaptive trial. Ultimately trial design and specification of design parameters will depend on clinical trial objectives. To support decision-making for clinical trial designs in contemporary TB regimen development, we provide a flexible clinical trial simulation tool that can be used to explore and optimize design features and parameters.

A Bayesian adaptive design is proposed for a comparative two-armed clinical trial using decision-theoretic approaches. A loss function is specified, based on the cost for each patient and the costs of making incorrect decisions at the end of a trial. At each interim analysis, the decision to terminate or to continue the trial is based on the expected loss function while concurrently incorporating efficacy, futility and cost. The maximum number of interim analyses is determined adaptively by the observed data. We derive explicit connections between the loss function and the frequentist error rates, so that the desired frequentist properties can be maintained for regulatory settings. The operating characteristics of the design can be evaluated on frequentist grounds. Extensive simulations are carried out to compare the proposed design with existing ones. The design is general enough to accommodate both continuous and discrete types of data. We illustrate the methods with an animal study evaluating a medical treatment for cardiac arrest. Copyright 2005, Oxford University Press.

In oncology, the treatment paradigm is shifting toward personalized medicine, where the goal is to match patients to the treatments most likely to deliver benefit. Treatment effects in various subpopulations may provide some information about treatment effects in other subpopulations. We compare different approaches to Phase II trial design where a new treatment is being investigated in several groups of patients. We compare considering each group in an independent trial to a single trial with hierarchical modeling of the patient groups. We assume four patient groups with different background response rates and simulate operating characteristics of three trial designs, Simon's Optimal Two-Stage design, a Bayesian adaptive design with frequent interim analyses, and a Bayesian adaptive design with frequent interim analyses and hierarchical modeling across patient groups. Simon's designs are based on 10% Type I and Type II error rates. The independent Bayesian designs are tuned to have similar error rates, but may have a slightly smaller mean sample size due to more frequent interim analyses. Under the null, the mean sample size is 2-4 patients smaller. A hierarchical model across patient groups can provide additional power and a further reduction in mean sample size. Under the null, the addition of the hierarchical model decreases the mean sample size an additional 4-7 patients in each group. Under the alternative hypothesis, power is increased to at least 98% in all groups. Hierarchical borrowing can make finding a single group in which the treatment is promising, if there is only one, more difficult. In a scenario where the treatment is uninteresting in all but one group, power for that one group is reduced to 65%. When the drug appears promising in some groups and not in others, there is potential for borrowing to inflate the Type I error rate. The Bayesian hierarchical design is more likely to correctly conclude efficacy or futility than the other two designs in many scenarios. The Bayesian hierarchical design is a strong design for addressing possibly differential effects in different groups.

BackgroundThis study presents a Bayesian Adaptive Semiparametric approach designed to address the challenges of pediatric randomized controlled trials (RCTs). The study focuses on efficiently handling primary and secondary endpoints, a critical aspect often overlooked in pediatric trials. This methodology is particularly pertinent in scenarios where sparse or conflicting prior data are present, a common occurrence in pediatric research, particularly for rare diseases or conditions.MethodOur approach considers Bayesian adaptive design, enhanced with B-Spline Semiparametric priors, allowing for the dynamic updating of priors with ongoing data. This improves the efficiency and accuracy of the treatment effect estimation. The Semiparametric prior inherent flexibility makes it suitable for pediatric populations, where responses to treatment can be highly variable. The design operative characteristics were assessed through a simulation study, motivated by the real-world case of the REnal SCarring Urinary infEction Trial (RESCUE).ResultWe demonstrate that Semiparametric prior parametrization exhibits an improved tendency to correctly declare the treatment effect at the study conclusion, even if recruitment challenges, uncertainty, and prior-data conflict arise. Moreover, the Semiparametric prior design demonstrates an improved ability in truly stopping for futility, with this tendency varying with the sample size and discontinuation rates. Approaches based on Parametric priors are more effective in detecting treatment efficacy during interim assessments, particularly with larger sample sizes.ConclusionOur findings indicate that these methods are especially effective in managing the complexities of pediatric trials, where prior data may be limited or contradictory. The flexibility of Semiparametric prior design in incorporating new evidence proves advantageous in addressing recruitment challenges and making informed decisions with restricted data.

The time is right for the use of Bayesian Adaptive Designs (BAD) in comparative effectiveness trials. For example, Patient Centered Outcomes Research Institute has joined the Food and Drug Administration and National Intitutes of Health in adopting policies/guidelines encouraging their use. There are multiple aspects to BAD that need to be considered when designing a comparative effectiveness design. First, the adaptation rules can determine the expected size of the trial. Second, a utility function can be used to combine extremely important co-endpoints (e.g., efficacy and tolerability) and is a valuable tool for incorporating clinical expertise and potentially patient preference. Third, accrual rate is also very, very important. Specifically, there is a juxtaposition related to accrual and BAD. If accrual rate is too fast we never gain efficient information for adapting. If accrual rate is too slow we never finish the clinical trial. We propose methodology for finding the 'sweet spot' for BAD that addresses these as design parameters. We demonstrate the methodology on a comparative effectiveness BAD of pharmaceutical agents in cryptogenic sensory polyneuropathy. The study has five arms with two endpoints that are combined with a utility function. The accrual rate is assumed to stem from multiple sites. We perform simulations from which the composite accrual rates across sites result in various piecewise Poisson distributions as parameter inputs. We balance both average number of patients needed and average length of time to finish the study.

BackgroundWindow-of-opportunity trials, evaluating the engagement of drugs with their biological target in the time period between diagnosis and standard-of-care treatment, can help prioritise promising new systemic treatments for later-phase clinical trials. Renal cell carcinoma (RCC), the 7th commonest solid cancer in the UK, exhibits targets for multiple new systemic anti-cancer agents including DNA damage response inhibitors, agents targeting vascular pathways and immune checkpoint inhibitors. Here we present the trial protocol for the WIndow-of-opportunity clinical trial platform for evaluation of novel treatment strategies in REnal cell cancer (WIRE).MethodsWIRE is a Phase II, multi-arm, multi-centre, non-randomised, proof-of-mechanism (single and combination investigational medicinal product [IMP]), platform trial using a Bayesian adaptive design. The Bayesian adaptive design leverages outcome information from initial participants during pre-specified interim analyses to determine and minimise the number of participants required to demonstrate efficacy or futility. Patients with biopsy-proven, surgically resectable, cT1b+, cN0–1, cM0–1 clear cell RCC and no contraindications to the IMPs are eligible to participate. Participants undergo diagnostic staging CT and renal mass biopsy followed by treatment in one of the treatment arms for at least 14 days. Initially, the trial includes five treatment arms with cediranib, cediranib + olaparib, olaparib, durvalumab and durvalumab + olaparib. Participants undergo a multiparametric MRI before and after treatment. Vascularised and de-vascularised tissue is collected at surgery. A ≥ 30% increase in CD8+ T-cells on immunohistochemistry between the screening and nephrectomy is the primary endpoint for durvalumab-containing arms. Meanwhile, a reduction in tumour vascular permeability measured by Ktrans on dynamic contrast-enhanced MRI by ≥30% is the primary endpoint for other arms. Secondary outcomes include adverse events and tumour size change. Exploratory outcomes include biomarkers of drug mechanism and treatment effects in blood, urine, tissue and imaging.DiscussionWIRE is the first trial using a window-of-opportunity design to demonstrate pharmacological activity of novel single and combination treatments in RCC in the pre-surgical space. It will provide rationale for prioritising promising treatments for later phase trials and support the development of new biomarkers of treatment effect with its extensive translational agenda.Trial registrationClinicalTrials.gov: NCT03741426 / EudraCT: 2018–003056-21.

With the advancement in biomedicine, many biologically targeted therapies have been developed. These targeted agents, however, may not work for everyone. Biomarker profiles can be used to identify effective targeted therapies. Our goals are to characterize the molecular signature of individual tumors, offer the best-fit targeted therapies to patients in a study, and identify promising agents for future development. We propose an outcome-based adaptive randomization trial design for patients with advanced stage non-small cell lung cancer. All patients have baseline biopsy samples taken for biomarker assessment prior to randomization to treatments. The primary endpoint of this study is the disease control rate at 8 weeks after randomization. The Bayesian probit model is used to characterize the disease control rate. Patients are adaptively randomized to one of four treatments with the randomization rate based on the updated disease control rate from the accumulated data in the trial. For each biomarker profile, high-performing treatments have higher randomization rates, and vice versa. An early stopping rule is implemented to suspend low-performing treatments from randomization. Based on extensive simulation studies, with a total of 200 evaluable patients, our trial has desirable operating characteristics to: (1) identify effective agents with a high probability; (2) suspend ineffective agents; and (3) treat more patients with effective agents that correspond to their biomarker profiles. Our trial design continues to update and refine the estimates as the trial progresses. This biomarker-based trial requires biopsible tumors and a two-week turn around time for biomarker profiling before randomization. Additionally, in order to learn from the interim data and adjust the randomization rate accordingly, the outcome-based adaptive randomization design is applicable only for trials when the endpoint can be assessed in a relative short period of time. Bayesian adaptive randomization trial design is a smart, novel, and ethical design. In conjunction with an early stopping rule, it can be used to efficiently identify effective agents, eliminate ineffective ones, and match effective treatments with patients' biomarker profiles. The proposed design is suitable for the development of targeted therapies and provides a rational design for personalized medicine.

In this paper, we develop the fixed-borrowing adaptive design, a Bayesian adaptive design which facilitates information borrowing from a historical trial using subject-level control data while assuring a reasonable upper bound on the maximum type I error rate and lower bound on the minimum power. First, one constructs an informative power prior from the historical data to be used for design and analysis of the new trial. At an interim analysis opportunity, one evaluates the degree of prior-data conflict. If there is too much conflict between the new trial data and the historical control data, the prior information is discarded and the study proceeds to the final analysis opportunity at which time a noninformative prior is used for analysis. Otherwise, the trial is stopped early and the informative power prior is used for analysis. Simulation studies are used to calibrate the early stopping rule. The proposed design methodology seamlessly accommodates covariates in the statistical model, which the authors argue is necessary to justify borrowing information from historical controls. Implementation of the proposed methodology is straightforward for many common data models, including linear regression models, generalized linear regression models, and proportional hazards models. We demonstrate the methodology to design a cardiovascular outcomes trial for a hypothetical new therapy for treatment of type 2 diabetes mellitus and borrow information from the SAVOR trial, one of the earliest cardiovascular outcomes trials designed to assess cardiovascular risk in antidiabetic therapies.

At the time of developing a biosimilar, the reference product has been on market for years and thus ample data are available on its efficacy and characteristics. We develop a Bayesian adaptive design for randomized biosimilar clinical trials to leverage the rich historical data on the reference product. This design takes a group sequential approach. At each interim, we employ the elastic meta-analytic-predictive (EMAP) prior methodology to adaptively borrow information from the historical data of the reference product to make go/no-go decision based on Bayesian posterior probabilities. In addition, the randomization ratio between the test and reference arms is adaptively adjusted at the interim with the goal to balance the sample size of the two arms at the end of trials. Simulation study shows that the proposed Bayesian adaptive design can substantially reduce the sample size of the reference arm, while achieving comparable power as the traditional randomized clinical trials that ignore the historical data. We apply our design to a biosimilar trial for treating breast cancer patients.