Bayesian adaptive randomization in the I-SPY2 sequential multiple assignment randomized trial

Sequential Multiple Assignment Randomized Trials: An Opportunity for Improved Design of Stroke Reperfusion Trials

In clinical practice, as well as in other areas where interventions are provided, a sequential individualized approach to treatment is often necessary, whereby each treatment is adapted based on the object's response. An adaptive intervention is a sequence of decision rules which formalizes the provision of treatment at critical decision points in the care of an individual. In order to inform the development of an adaptive intervention, scientists are increasingly interested in the use of sequential multiple assignment randomized trials (SMART), which is a type of multi-stage randomized trial where individuals are randomized repeatedly at critical decision points to a set treatment options. While there is great interest in the use of SMART and in the development of adaptive interventions, both are relatively new to the medical and behavioral sciences. As a result, many clinical researchers will first implement a SMART pilot study (i.e., a small-scale version of a SMART) to examine feasibility and acceptability considerations prior to conducting a full-scale SMART study. A primary aim of this paper is to introduce a new methodology to calculate minimal sample size necessary for conducting a SMART pilot.

Due to the cost and complexity of conducting a sequential multiple assignment randomized trial (SMART), it is desirable to pre-define a small number of personalized regimes to study. We proposed a simulation-based approach to studying personalized dosing strategies in contexts for which a therapeutic agent's pharmacokinetic and pharmacodynamics properties are well understood. We take dosing of warfarin as a case study, as its properties are well understood. We consider a SMART in which there are five intervention points in which dosing may be modified, following a loading phase of treatment. Realistic SMARTs are simulated, and two methods of analysis, G-estimation and Q-learning, are used to assess potential personalized dosing strategies. In settings where outcome modelling may be complex due to the highly non-linear nature of the pharmacokinetic and pharmacodynamics mechanisms of the therapeutic agent, G-estimation provides for which the more promising method of estimating an optimal dosing strategy. Used in combination with the simulated SMARTs, we were able to improve simulated patient outcomes and suggest which patient characteristics were needed to best individually tailor dosing. In particular, our simulations suggest that current dosing should be determined by an individual's current coagulation time as measured by the international normalized ratio (INR), their last measured INR, and their last dose. Tailoring treatment only based on current INR and last warfarin dose provided inferior control of INR over the course of the trial. The ability of the simulated SMARTs to suggest optimal personalized dosing strategies relies on the pharmacokinetic and pharmacodynamic models used to generate the hypothetical patient profiles. This approach is best suited to therapeutic agents whose effects are well studied. Prior to investing in a complex randomized trial that involves sequential treatment allocations, simulations should be used where possible in order to guide which dosing strategies to evaluate.

The logic of the SMART (Sequential Multiple Assignment Randomization Trial) design was applied to assess the replicability of original-replicate study pairs for Open Science Collaboration (OSC) intervention studies. Within SMART, we utilized both subtests of the correspondence test (CT) to assess study pair comparability. First, we implemented a CT difference test to determine if an effect size difference between the original study and its replicate pair was close to zero; second, we implemented a CT equivalence test to determine if the effect size difference of that study pair was within a designated threshold. In Stage 1 of SMART, each study pair was randomly assigned to one of two alphas (.01 and .05), thereby creating two, probabilistically similar subsets of study pairs. Within each alpha subset, successful difference tests (test of significance was not significantly different than zero) and unsuccessful difference tests were then determined. In Stage 2 of SMART, study pairs in each combination of alpha level and successful or unsuccessful difference tests were randomly assigned to one of two thresholds (±.25 SD, ±.50 SD). Equivalence tests were then conducted for all study pairs in each of these four subsets. Successful equivalence occurred when the distance between an original and its replicate pair was statistically significantly less than a given threshold. Thus, initial randomization followed by a second randomization was used to gauge comparability of each OSC original study and its replicate, for two alpha levels and two thresholds. In the first set of results, to mirror the common replicability assessment case in which only difference tests are conducted, 16 of 96 difference tests (16.7%) conducted in Stage 1 were successful. In the second set of results, for initially successful difference tests, two thresholds were used to determine the percent of study pairs that also passed the equivalence test. Depending on α and threshold, 8.0%-13.8% of studies successfully passed both difference and equivalence CT subtests. In the third set of results, using SMART, after randomization to two α-values and contingent on success or lack of success of a difference test, study pairs were randomized to two thresholds and a statistical test of equivalence conducted. Using meta-analysis methods within SMART-based subsets of study pairs, original-replicate average effect size differences were compared to differences in the second set of results. We found a similarly-sized 10.3% of study pairs passed both CT subtests (nine of 87 study pairs successfully passed the difference test at either alpha and successfully passed the equivalence test at either threshold). Reflecting the importance of incorporating both CT subtests, of 16 study pairs that initially passed the difference test, nearly half (43.7%) failed the equivalence test. Thus, for CT success, we found that α choice had little impact, while threshold choice was an important determinant. In all three sets of results, the percent of successful replications was substantially smaller than the 36% of OSC replicates that were statistically significant. To confirm this study’s replicability, we found very similar patterns of CT success and lack of success for two, SMART-based tables, one for alpha = .01 and one for alpha = .05. The current research extends the utility of CT established by Steiner and Wong (2018) in which results were based on simulation data.

Minority adolescents are at highest risk for obesity and extreme obesity; yet, there are few clinical trials targeting African American adolescents with obesity. The purpose of the study was to develop an adaptive family-based behavioral obesity treatment for African American adolescents using a sequential multiple assignment randomized trial (SMART) design. Fit Families was a SMART where 181 African American adolescents (67% female) aged 12-17 were first randomized to office-based versus home-based behavioral skills treatment delivered from a Motivational Interviewing foundation. After 3 months, nonresponders to first phase treatment were rerandomized to continued home-based behavioral skills treatment or contingency management with voucher-based reinforcement for adolescent weight loss and for caregiver adherence to the program. All interventions were delivered by community health workers. The primary outcome was treatment retention and percent overweight. All adolescents reduced percent overweight by -3.20%; there were no significant differences in percent overweight based on treatment sequence. Adolescents receiving home-based delivery in Phase 1 and contingency management in Phase 2 completed significantly more sessions than those receiving office-based treatment and continued skills without CM (M = 8.03, SD = 3.24 and M = 6.62, SD = 2.95, respectively). The effect of contingency management was strongest among older and those with lower baseline confidence. Younger adolescents experienced greater weight reductions when receiving continued skills (-4.90% compared with -.02%). Behavioral skills training can be successfully delivered to African American adolescents with obesity and their caregivers by community health workers when using a home-based service model with incentives. More potent interventions are needed to increase reductions in percent overweight and may need to be developmentally tailored for younger and older adolescents.

This pilot study was conducted to assess the feasibility and acceptability of 4 adaptive treatment strategies (ATSs) for adolescent depression to plan for a subsequent full-scale clinical trial. The ATSs aim to address 2 questions that arise when personalizing treatment: (a) For adolescents treated with Interpersonal Psychotherapy for depressed adolescents (IPT-A; Mufson et al., 2004), at what time point should therapists make the determination that the adolescent is not likely to respond if the initial treatment plan is continued (week 4 or week 8)? (b) For adolescents who are judged to need their treatment augmented, should the therapist increase the number of IPT-A sessions or add pharmacotherapy (fluoxetine)? A 16-week pilot sequential multiple assignment randomized trial (SMART) was conducted with 32 adolescents (M age = 14.9) who had a diagnosis of major depressive disorder, dysthymic disorder, or depressive disorder not otherwise specified. Adolescents were primarily female (75%) and Caucasian (84.4%). Data regarding the feasibility and acceptability of the study and treatment procedures and treatment response rates were collected. Week 4 was the more feasible and acceptable decision point for assessing need for a change to treatment. Adolescents, parents, and therapists reported a range of attitudes about medication and more intensive therapy as treatment options. Results from the pilot study have yielded additional research questions for the full-scale SMART and will improve our ability to successfully conduct the trial.

Background In adaptive interventions, dynamic information about individuals is used to determine whether and how to intervene. Depending on the nature of the intervention (eg, human-delivered vs digitally delivered), adaptations may need to occur slowly or rapidly (eg, after a month vs daily). The optimization of adaptive interventions has been powered by 2 experimental designs, both employing sequential randomizations but on different timescales: the sequential multiple assignment randomized trial (SMART) on a slow timescale for standard adaptive interventions and the microrandomized trial (MRT) on a fast timescale for just-in-time adaptive interventions. Purpose This paper intends to help investigators who have research questions about the delivery and adaptation of intervention options at more than 1 time point, select and plan a trial design with the right form of sequential randomizations, on the appropriate timescale. Methods This is a conceptual paper discussing considerations related to designing trials with sequential randomizations. Results We clarify the differences and similarities between the SMART and the MRT, outline the types of questions that can be answered by the 2 experimental designs, and provide guidelines for selecting and planning the appropriate form of trial design based on the questions that need answers. Conclusions When designing a trial with sequential randomizations, it is essential to consider the need for sequential randomizations, the appropriate timescale for randomizations, whether to impose restrictions on randomizations, whether to randomize upfront or sequentially, the timing for measuring the primary outcome, and sample size planning.

Advances in digital technologies have created unprecedented opportunities to deliver effective and scalable behavior change interventions. Many digital interventions include multiple components, namely several aspects of the intervention that can be differentiated for systematic investigation. Various types of experimental approaches have been developed in recent years to enable researchers to obtain the empirical evidence necessary for the development of effective multiple-component interventions. These include factorial designs, Sequential Multiple Assignment Randomized Trials (SMARTs), and Micro-Randomized Trials (MRTs). An important challenge facing researchers concerns selecting the right type of design to match their scientific questions. Here, we propose MCMTC – a pragmatic framework that can be used to guide investigators interested in developing digital interventions in deciding which experimental approach to select. This framework includes five questions that investigators are encouraged to answer in the process of selecting the most suitable design: (1) Multiple-component intervention: Is the goal to develop an intervention that includes multiple components; (2) Component selection: Are there open scientific questions about the selection of specific components for inclusion in the intervention; (3) More than a single component: Are there open scientific questions about the inclusion of more than a single component in the intervention; (4) Timing: Are there open scientific questions about the timing of component delivery, that is when to deliver specific components; and (5) Change: Are the components in question designed to address conditions that change relatively slowly (e.g., over months or weeks) or rapidly (e.g., every day, hours, minutes). Throughout we use examples of tobacco cessation digital interventions to illustrate the process of selecting a design by answering these questions. For simplicity we focus exclusively on four experimental approaches—standard two- or multi-arm randomized trials, classic factorial designs, SMARTs, and MRTs—acknowledging that the array of possible experimental approaches for developing digital interventions is not limited to these designs.

LBA501 Background: I-SPY2.2 is a multicenter phase 2 platform sequential multiple assignment randomized trial (SMART) in the neoadjuvant breast cancer setting that evaluates novel experimental regimens as first in a sequence (Block A) followed by standard chemo/targeted therapies (Blocks B/C) if indicated. The goal is to achieve a pCR after novel targeted agents alone or in sequence with standard therapies, with the optimal therapy assigned based on the tumor response predictive subtype (RPS). RPS incorporates expression-based immune, DNA repair deficiency (DRD), and luminal signatures with hormone receptor (HR) and HER2 status to subset patients into 6 subtypes: S1: HR+HER2-Immune-DRD-; S2: HR-HER2-Immune-DRD-; S3: HER2-Immune+; S4: HER2-Immune-DRD+; S5: HER2+/non-Luminal; S6: HER2+/Luminal. Methods: RPS S1, S2, S3, and S4 were eligible for assignment to Dato+Durva in Block A. Patients were followed by MRI during treatment (at 3, 6, and 12 weeks after start of Blocks A and B). Predicted responders by MRI and biopsy at the end of Block A or B have the option of going to surgery early; otherwise, they proceed to next treatment Block (B +/- C). Randomization to Block B includes a taxane-based regimen specific to the RPS, and options include S1: paclitaxel; S2 and S3: paclitaxel + carboplatin + pembrolizumab; S4: paclitaxel + carboplatin vs. paclitaxel + carboplatin + pembrolizumab. Patients who did not go to surgery after Block B proceeded to Block C (AC or AC + Pembrolizumab if HR-HER2-). The primary endpoint is pCR. Efficacy is evaluated within each RPS and HR+HER2- and HR-HER2- signatures. To estimate the arm's efficacy as a stand-alone therapy, we use a Bayesian covariate-adjusted model to estimate the pCR rate and compare the posterior distribution to a subtype-specific fixed threshold. This model uses pCR data when available and MRI data when pCR is not. To estimate pCR rate in the context of a multi-decision treatment regimen, we use a Bayesian model based on if and when a pCR occurred in the trial. The posterior is compared to a subtype-specific dynamic control generated from historical I-SPY data. Results: 106 patients were randomly assigned to the Dato+Durva arm between September 2022 and August 2023. The results for Dato+Durva as a stand-alone therapy are summarized in Table. After completion of Block A, 36 patients proceeded to surgery without completing Blocks B/C. Conclusions: Dato+Durva meets threshold for graduation within the RPS S3 subtype based on estimated pCR rate of 72% and warrants further investigation in a larger randomized controlled trial. Clinical trial information: NCT01042379 . [Table: see text]

The primary goal of clinical trials is to collect enough scientific evidence for a new intervention. Despite the widespread use of equal randomization in clinical trials, response-adaptive randomization has attracted considerable interest in terms of ethical concerns. In this thesis, delayed response problems and innovative designs for cytostatic agents in oncology clinical trials are studied.
\n
\nThere is typically a prerun of equal randomization before the implementation of response-adaptive randomization, while it is often not clear how many subjects are needed in this prephase, and in practice an arbitrary number of patients are allocated in this equal randomization stage. In addition, real-time response-adaptive randomization often requires patient response to be immediately available after the treatment, while clinical response, such as tumor shrinkage, may take a relatively long period of time to exhibit. In the first part of the thesis, a nonparametric fractional model and a parametric optimal allocation scheme are developed to tackle the common problem caused by delayed response. In addition, a two-stage procedure to achieve a balance between power and the number of responders is investigated, which is equipped with a likelihood ratio test before skewing the allocation probability toward a better treatment. The operating characteristics of the two-stage designs are evaluated through extensive simulation studies and an HIV clinical trial is used for illustration. Numerical results show that the proposed method satisfactorily resolves the issues involved in response-adaptive randomization and delayed response.
\n
\nIn phase I clinical trials with cytostatic agents, toxicity endpoints, as well as efficacy effects, should be taken into consideration for identifying the optimal biological dose (OBD). In the second part of the thesis, a two-stage Bayesian mixture modeling approach is developed, which first locates the maximum tolerated dose (MTD) through a mixture of parametric and nonparametric models, and then determines the most efficacious dose using Bayesian adaptive randomization among multiple candidate models. In the first stage searching for the MTD, a beta-binomial model in conjunction with a probit model as a mixture modeling approach is studied, and decisions are made based on the model that better fits the toxicity data. The model fitting adequacy is measured by the deviance information criterion and the posterior model probability. In the second stage searching for the OBD, the assumption that efficacy monotonically increases with the dose is abandoned and, instead, all the possibilities that each dose could have the highest efficacy effect are enumerated so that the dose-efficacy curve can be increasing, decreasing, or umbrella-shape. Simulation studies show the advantages of the proposed mixture modeling approach for pinpointing the MTD and OBD, and demonstrate its satisfactory performance with cytostatic agents.

For each patient, treatment for a disease often is a multistage process involving an alternating sequence of observations and therapeutic decisions, with the physician’s decision at each stage based on the patient’s entire history up to that stage. This chapter begins with discussion of a simple two-stage version of this process, in which a Frontline treatment is given initially and, if and when the patient’s disease worsens, i.e., progresses, a second, Salvage treatment is given, so the two-stage regime is (Frontline, Salvage). The discussion of this case will include examples where, if one only accounts for the effects of Frontline and Salvage separately in each stage, the effect of the entire regime on survival time may not be obvious. Discussions and illustrations will be given of the general paradigms of dynamic treatment regimes (DTRs) and sequential multiple assignment randomized trials (SMARTs). Several statistical analyses of data from a prostate cancer trial designed by Thall et al. (2000) to evaluate multiple DTRs then will be discussed in detail. As a final example, several statistical analyses of observational data from a semi-SMART design of DTRs for acute leukemia, given by Estey et al. (1999), Wahed and Thall (2013), and Xu et al. (2016), will be discussed.

Given that over 20 million adults each year do not receive care for their mental health difficulties, it is imperative to improve system-level capacity issues by increasing treatment efficiency. The present study aimed to collect feasibility/acceptability data on two strategies for increasing the efficiency of cognitive behavioral therapy: (1) personalized skill sequences and (2) personalized skill selections. Participants (N = 70) with anxiety and depressive disorders were enrolled in a pilot sequential multiple assignment randomized trial (SMART). Patients were randomly assigned to receive skill modules from the Unified Protocol in one of three sequencing conditions: standard, sequences that prioritized patients’ relative strengths, and sequences that prioritized relative deficits. Participants also underwent a second-stage randomization to either receive 6 sessions or 12 sessions of treatment. Participants were generally satisfied with the treatment they received, though significant differences favored the Capitalization and Full duration conditions. There were no differences in trajectories of improvement as a function of sequencing condition. There were also no differences in end-of-study outcomes between brief personalized treatment and full standard treatment. Thus, it may be feasible to deliver CBT for personalized durations, though this may not substantially impact trajectories of change in anxiety or depressive symptoms.

Clinical trials are constantly evolving in the context of increasingly complex research questions and potentially limited resources. In this review article, we discuss the emergence of "adaptive" clinical trials that allow for the preplanned modification of an ongoing clinical trial based on the accumulating evidence with application across translational research. These modifications may include terminating a trial before completion due to futility or efficacy, re-estimating the needed sample size to ensure adequate power, enriching the target population enrolled in the study, selecting across multiple treatment arms, revising allocation ratios used for randomization, or selecting the most appropriate endpoint. Emerging topics related to borrowing information from historic or supplemental data sources, sequential multiple assignment randomized trials (SMART), master protocol and seamless designs, and phase I dose-finding studies are also presented. Each design element includes a brief overview with an accompanying case study to illustrate the design method in practice. We close with brief discussions relating to the statistical considerations for these contemporary designs.

BackgroundObstructive sleep apnea (OSA) and insomnia are commonly co-occurring conditions that amplify morbidity and complicates the management of affected patients. Unfortunately, previous research provides limited guidance as to what constitutes the best and most practical management approach for this comorbid patient group. Some preliminary studies show that when cognitive behavioral insomnia therapy (CBT-I) is combined with standard OSA therapies for these patients, outcomes are improved. However, the dearth of trained providers capable of delivering CBT-I has long served as a pragmatic barrier to the widespread use of this therapy in clinical practice. The emergence of sophisticated online CBT-I (OCBT-I) programs could improve access, showing promising reductions in insomnia severity. Given its putative scalability and apparent efficacy, some have argued OCBT-I should represent a 1st-stage intervention in a broader stepped care model that allocates more intensive and less assessable therapist-delivered CBT-I (TCBT-I) only to those who show an inadequate response to lower intensity OCBT-I. However, the efficacy of OCBT-I as a 1st-stage therapy within a broader stepped care management strategy for insomnia comorbid with OSA has yet to be tested with comorbid OSA/insomnia patients.Methods/designThis dual-site randomized clinical trial will use a Sequential Multiple Assignment Randomized Trial (SMART) design to test a stepped care model relative to standard positive airway pressure (PAP) therapy and determine if (1) augmentation of PAP therapy with OCBT-I improves short-term outcomes of comorbid OSA/insomnia and (2) providing a higher intensity 2nd-stage CBT-I to patients who show sub-optimal short-term outcomes with OCBT-I+PAP improves short and longer-term outcomes. After completing baseline assessment, the comorbid OSA/insomnia patients enrolled will be randomized to a 1st-stage therapy that includes usual care PAP + OCBT-I or UC (usual care PAP + sleep hygiene education). Insomnia will be reassessed after 8 weeks. OCBT-I recipients who meet “remission” criteria (defined as an Insomnia Severity Index score < 10) will continue PAP but will not be offered any additional insomnia intervention and will complete study outcome measures again after an additional 8 weeks and at 3 and 6 month follow-ups. OCBT-I recipients classified as “unremitted” after 8 weeks of treatment will be re-randomized to a 2nd-stage treatment consisting of continued, extended access to OCBT-I or a switch to TCBT-I. Those receiving the 2nd-stage intervention as well as the UC group will be reassessed after another 8 weeks and at 3- and 6-month follow-up time points. The primary outcome will be insomnia remission. Secondary outcomes will include subjective and objective sleep data, including sleep time, sleep efficiency, fatigue ratings, PAP adherence, sleepiness ratings, sleep/wake functioning ratings, and objective daytime alertness.DiscussionThis study will provide new information about optimal interventions for patients with comorbid OSA and insomnia to inform future clinical decision-making processes.Trial registrationClinicalTrials.gov, NCT03109210, registered on April 12, 2017, prospectively registered.

A sequential multiple assignment randomized trial (SMART) protocol for empirically developing an adaptive preventive intervention for college student drinking reduction