Assessing Treatment Effects with Pharmacometric Models: A New Method that Addresses Problems with Standard Assessments

<div>Abstract<p>The FDA Oncology Center of Excellence recently launched a crowdsourcing pilot to request ideas from the scientific community for research questions that FDA could address with pooled analyses of clinical trial data submitted to the agency for regulatory purposes. This effort builds on FDA’s track record of publishing pooled analyses to explore scientific questions that cannot be addressed in a single trial due to limited sample size. The research crowdsourcing pilot tested a new approach for obtaining external input on regulatory science activities, because FDA is generally unable to share patient-level data outside of the agency due to federal disclosure laws and regulations protecting different types of data submitted in regulatory applications. We received 29 submissions over the 28-day crowdsourcing campaign, including one research idea that we are exploring for possible follow up. Based on our experience with this pilot, we learned that crowdsourcing is a promising new approach to gather external input and feedback. We identified opportunities to build understanding in the external oncology community about the types of data typically included in regulatory applications and expand dissemination of published FDA pooled analyses to help inform future drug development and clinical practice.</p></div>

The statistical power of cross-over trials can be increased by taking 'baseline' measurements of the outcome variable at the start of each treatment period. Analysis of covariance (ANCOVA), rather than analysis of change scores, takes best advantage of this. However, ANCOVA can give biased treatment effect estimates in observational studies with true baseline imbalance. While in truth balanced, chance baseline imbalance is possible in individual randomized cross-over studies due to their typically small sample size. Although such chance imbalance does not cause biased estimation on average over repeated trials, this simulation study will aim to confirm the appropriateness of ANCOVA when faced with the analysis of data from an individual trial in which chance baseline imbalance is clearly apparent. Randomized cross-over trials were simulated, varying in sample size and the pattern and strength of correlation between repeated measures. Estimates from ANCOVA, change scores, and post-treatment difference were unbiased on average across each set of simulated data sets. ANCOVA and change scores could use baseline information to improve precision, but change scores could also reduce precision if baseline measures were uninformative. Change scores only were correlated with chance within-subject baseline imbalance. All three estimators could be correlated with chance between-subjects imbalance in the first period baseline measurements, the strongest associations being with the post-treatment difference. Consistent results were obtained from a real data example. In conclusion, ANCOVA took best advantage of baseline measures to improve precision, and avoided bias in the widest set of circumstances with chance imbalance in those baseline measures.

The rate of participant attrition in alcohol clinical trials is often substantial and can cause significant issues with regard to the handling of missing data in statistical analyses of treatment effects. It is common for researchers to assume that missing data is indicative of participant relapse, and under that assumption, many researchers have relied on setting all missing values to the worst-case scenario for the outcome (e.g., missing=heavy drinking). This sort of single-imputation method has been criticized for producing biased results in other areas of clinical research, but has not been evaluated within the context of alcohol clinical trials, and many alcohol researchers continue to use the missing=heavy drinking assumption. Data from the COMBINE study, a multisite randomized clinical trial, were used to generate simulated situations of missing data under a variety of conditions and assumptions. We manipulated the sample size (n=200, 500, and 1,000) and dropout rate (5, 10, 25, 30%) under 3 missing data assumptions (missing completely at random, missing at random, and missing not at random). We then examined the association between receiving naltrexone and heavy drinking during the first 10 weeks following treatment using 5 methods for treating missing data (complete case analysis [CCA], last observation carried forward [LOCF], missing=heavy drinking, multiple imputation [MI], and full information maximum likelihood [FIML]). CCA, LOCF, and missing=heavy drinking produced the most biased naltrexone effect estimates and standard errors under conditions that are likely to exist in randomized clinical trials. MI and FIML produced the least biased naltrexone effect estimates and standard errors. Assuming that missing=heavy drinking produces biased results of the treatment effect and should not be used to evaluate treatment effects in alcohol clinical trials.

BackgroundA nested case-control study is an efficient design that can be embedded within an existing cohort study or randomised trial. It has a number of advantages compared to the conventional case-control design, and has the potential to answer important research questions using untapped prospectively collected data.MethodsWe demonstrate the utility of the matched nested case-control design by applying it to a secondary analysis of the Abnormal Doppler Enteral Prescription Trial. We investigated the role of milk feed type and changes in milk feed type in the development of necrotising enterocolitis in a group of 398 high risk growth-restricted preterm infants.ResultsUsing matching, we were able to generate a comparable sample of controls selected from the same population as the cases. In contrast to the standard case-control design, exposure status was ascertained prior to the outcome event occurring and the comparison between the cases and matched controls could be made at the point at which the event occurred. This enabled us to reliably investigate the temporal relationship between feed type and necrotising enterocolitis.ConclusionsA matched nested case-control study can be used to identify credible associations in a secondary analysis of clinical trial data where the exposure of interest was not randomised, and has several advantages over a standard case-control design. This method offers the potential to make reliable inferences in scenarios where it would be unethical or impractical to perform a randomised clinical trial.

Biased effects of pre-diagnostic physical activity on breast cancer survival: Systematic review and meta-analysis

Bayesian analysis of clinical trial data may provide useful information to aid in study interpretation, especially when trial evidence suggests that the benefits of an intervention are uncertain, such as in a trial that evaluated early extracorporeal membrane oxygenation (ECMO) for severe acute respiratory distress syndrome (ARDS). To demonstrate the potential utility of Bayesian analyses by estimating the posterior probability, under various assumptions, that early ECMO was associated with reduced mortality in patients with very severe ARDS in a randomized clinical trial (RCT). A post hoc Bayesian analysis of data from an RCT (ECMO to Rescue Lung Injury in Severe ARDS [EOLIA]) that included 249 patients with very severe ARDS who had been randomized to receive early ECMO (n = 124; mortality at 60 days, 35%) vs initial conventional lung-protective ventilation with the option for rescue ECMO (n = 125, mortality at 60 days, 46%). The trial was designed to detect an absolute risk reduction (ARR) of 20%, relative risk (RR) of 0.67. Statistical prior distributions were specified to represent varying levels of preexisting enthusiasm or skepticism for ECMO and by Bayesian meta-analysis of previously published studies (with downweighting to account for differences and quality between studies). The RR, credible interval (CrI), ARR, and probability of clinically important mortality benefit (varying from RR less than 1 to RR less than 0.67 and ARR from 2% or more to 20% or more) were estimated with Bayesian modeling. Combining a minimally informative prior distribution with the findings of the EOLIA trial, the posterior probability of RR less than 1 for mortality at 60 days after randomization was 96% (RR, 0.78 [95% CrI, 0.56-1.04]); the posterior probability of RR less than 0.67 was 18%, the probability of ARR of 2% or more was 92%, and the probability of ARR of 20% or more was 2%. With a moderately enthusiastic prior, equivalent to information from a trial of 264 patients with an RR of 0.78, the estimated RR was 0.78 (95% CrI, 0.63-0.96), the probability of RR less than 1 was 99%, the probability of RR less than 0.67 was 8%, the probability of ARR of 2% or more was 97%, and the probability of ARR of 20% or more was 0%. With a strongly skeptical prior, equivalent to information from a trial of 264 patients with an RR of 1.0, the estimated RR was 0.88 (95% CrI, 0.71-1.09), the probability of RR less than 1 was 88%, the probability of RR less than 0.67 was 0%, the probability of ARR of 2% or more was 78%, and the probability of ARR of 20% or more was 0%. If the prior was informed by previous studies, the estimated RR was 0.71 (95% CrI, 0.55-0.94), the probability of RR less than 1 was 99%, the probability of RR less than 0.67 was 48%, the probability of ARR of 2% or more was 98%, and the probability of ARR of 20% or more was 4%. Post hoc Bayesian analysis of data from a randomized clinical trial of early extracorporeal membrane oxygenation compared with conventional lung-protective ventilation with the option for rescue extracorporeal membrane oxygenation among patients with very severe acute respiratory distress syndrome provides information about the posterior probability of mortality benefit under a broad set of assumptions that may help inform interpretation of the study findings.

Background: It is sometimes suggested that newly-approved cancer treatments have only marginal effectiveness, which raises questions concerning their cost-vs-benefit ratio. Such concerns appear at odds with the lower cancer-related hospitalization rate and improved survival. However, such cost-effectiveness analyses rely on population-based averages obtained from the analysis of clinical trial data. By failing to analyze data from longitudinal datasets, such assessments are unable to account for real-world patient conditions and treatment patterns in evaluating clinical and cost-effectiveness. Longitudinally surveyed clinical data has the potential to objectively reveal any association between patient outcomes and new cancer treatment utilization. Methods: We investigated the effect of being prescribed a higher proportion of new oncology drugs on quality of life, medical services use, and productivity measures as reported by the Medical Expenditure Panel Survey (MEPS, 1996–2015). General linear models with Taylor series variance estimation were applied. New oncology drugs were defined as cancer treatments marketed after the year 2000. Included subjects (N=16,677) had a solid or hematologic malignancy diagnosis (CCCodex 11–47) and available prescription data. Individual age and employment status were accounted for as covariates. All analyses were performed using SAS version 9.4 (Cary, NC). Results: Unadjusted regression data show that individuals using newer oncology treatments missed on average 2.5 (±0.3 SE) fewer days of work or school per year as compared to patients using older drugs (43% improved productivity, P<.0001). The effect persisted even after adjusting for the magnitude of the effect (P<.0001). Accounting for age, the use of newer drugs was, on average, associated with ∼35% fewer missed work or school days. Cancer patients using newer treatments had 0.06 (±0.01 SE) fewer hospital admissions/year compared to patients using older treatments (P<.0001) and spent less time in the emergency room (P<.0411) with ∼45% fewer hospitalizations. Patients using newer medicines also had fewer health-related visits (P<.0001). Conclusion: Analysis of longitudinal real-world evidence gives a more comprehensive and reliable view of the clinical and economic impact of new oncology treatments. Our data suggests significant reductions in lost work/school days, hospitalizations, and use of medical services in general.

Probabilities of hospital mortality provide meaningful information in many contexts, such as in discussions of patient prognosis by intensive care physicians, in patient stratification for analysis of clinical trial data by researchers, and in hospital reimbursement analysis by insurers. Use of probabilities as binary predictors based on a cut point can be misleading for making treatment decisions for individual patients, however, even when model performance is good overall. Alternative models for estimating severity of illness in intensive care unit (ICU) patients, while demonstrating good agreement for describing patients in the aggregate, are shown to differ considerably for individual patients. This suggests that identifying patients unlikely to benefit from ICU care by using models must be approached with considerable caution.

For a variety of reasons including poorly designed case report forms (CRFs), incomplete or invalid CRF data entries, and premature treatment or study discontinuations, missing data is a common phenomenon in controlled clinical trials. With the widely accepted use of the intent-to-treat (ITT) analysis dataset as the primary analysis dataset for the analysis of controlled clinical trial data, the presence of missing data could lead to complicated data analysis strategies and subsequently to controversy in the interpretation of trial results. In this article, we review the mechanisms of missing data and some common approaches to analyzing missing data with an emphasis on study dropouts. We discuss the importance of understanding the reasons for study dropouts with ways to assess the mechanisms of missingness. Finally, we discuss the results of a comparative Monte Carlo investigation of the performance characteristics of commonly utilized statistical methods for the analysis of clinical trial data with dropouts. The methods investigated include the mixed effects model for repeated measurements (MMRM), weighted and unweighted generalized estimating equations (GEE) method for the available case data, multiple-imputation-based GEE (MI-GEE), complete case (CC) analysis of covariance (ANCOVA), and last observation carried forward (LOCF) ANCOVA. Simulation experiments for the repeated measures model with missing at random (MAR) dropout, under varying dropout rates and intrasubject correlation, show that the LOCF, ANCOVA, and weighted GEE methods perform poorly in terms of percent relative bias for estimating a difference in means effect, while the MI-GEE and weighted GEE methods both have less power for rejecting a zero difference in means hypothesis.

Estimating treatment effects from longitudinal clinical trial data with missing values: comparative analyses using different methods

The debate about what constitutes a valid analysis of clinical trial data is longstanding. While the intention-to-treat (ITT) principle seems to be widely accepted in the context of controlled clinical trials aiming to show superiority of an experimental treatment over a control, the best choice for a noninferiority trial is still under discussion. In this article, it is argued that the definition of analysis sets and the purpose of ITT and per-protocol analyses proposed in the International Conference on Harmonisation biostatistics guideline E9 should be revised to allow for more appropriate analyses, given that statistical methodology has been developed since the guideline was issued.

Statistical methods are fundamental to designing and executing cancer clinical trials, which present unique challenges such as patient diversity, incomplete data, and ethical considerations. This article underscores the pivotal role of advanced statistical techniques in ensuring reliable outcomes, with a focus on sample size estimation, randomization, and endpoint selection. Key methodologies discussed include Cox regression, Bayesian modeling, and machine learning applications for predictive analytics and real-time data processing. Practical solutions to challenges like treatment effect assessment, bias reduction, and ethical compliance are highlighted through real-world examples. The paper concludes by envisioning future innovations that enhance accuracy, efficiency, and accessibility in cancer research. By integrating rigorous statistical methods with clinical relevance, this work aims to propel oncology into a new era of precision and patient-centered care.

MSQ, the 16-item Migraine-Specific Quality-of-Life Questionnaire (Version 1.0), was developed by Glaxo Wellcome Inc. to assess the effect of migraine and its treatment on patients' health-related quality of life (HR-QOL). The MSQ was hypothesised to measure 3 meaningful dimensions: (i) Role Function-Restrictive; (ii) Role Function-Preventive; and (iii) Emotional Function. The objective of this research was to further investigate the number of dimensions as well as the items contained in each dimension through principal components factor analysis of clinical trial data. Secondary objectives were to determine whether the factor structure changed in post-treatment visits compared with screening visits, to make recommendations for coding the MSQ when the patient did not have a migraine in the previous 4 weeks, and to modify the MSQ if so indicated by this research. Results supported the existence of 3 distinct factors which agreed strongly with the hypothesised dimensions. The analysis of post-treatment data suggested that the underlying factor structure of the MSQ varies as a result of treatment. Based on evaluations of the 'did not have a migraine' response, it was concluded that it be dropped from the MSQ. All these changes have been incorporated into MSQ (Version 2.0) which is being evaluated in studies to determine if its psychometric properties are different than the properties of the previous version.

BackgroundTraditionally rheumatoid arthritis (RA) trials classify patients as responders and non-responders; they ignore the potential range of treatment responses. Group Based Trajectory Models (GBTMs) provide a more refined approach. They identify patient subgroups with similar outcome trajectories. We used GBTMs to classify patients into subgroups of varying responses and explore factors associated with different responses to intensive treatment in a secondary analysis of intensive treatment in the TITRATE clinical trial.MethodsThe TITRATE trial enrolled 335 patients with RA: 168 patients were randomised to receive intensive management, which comprised monthly assessments including measures of the disease activity score for 28 joints (DAS28), treatment escalation when patients were not responding sufficiently and psychosocial support; 163 of these patients completed the trial. We applied GBTMs to monthly DAS28 scores over one year to these patients who had received intensive management. The control group had standard care and were assessed every 6 months; they had too few DAS28 scores for applying GBTMs.ResultsGBTMs identified three distinct trajectories in the patients receiving intensive management: good (n = 40), moderate (n = 76) and poor (n = 47) responders. Baseline body mass index (BMI), disability, fatigue and depression levels were significantly different between trajectory groups. Few (10%) good responders were obese, compared to 38% of moderate, and 43% of poor responders (P = 0.002). Few (8%) good responders had depression, compared to 14% moderate responders, and 38% poor responders (P < 0.001). The key difference in treatments was using high-cost biologics, used in only 5% of good responders but 30% of moderate and 51% of poor responders (P < 0.001). Most good responders had endpoint remissions and low disability, pain, and fatigue scores; few poor responders achieved any favourable outcomes.ConclusionGBTMs identified three trajectories of disease activity progression in patients receiving intensive management for moderately active RA. Baseline variables like obesity and depression predicted different treatment responses. Few good responders needed biologic drugs; they responded to conventional DMARDs alone. GBTMs have the potential to facilitate precision medicine enabling patient-oriented treatment strategies based on key characteristics.RegistrationTITRATE Trial ISRCTN 70160382.

Achieving a target intraocular pressure (IOP) in patients with ocular hypertension and open-angle glaucoma has many barriers, some treatment specific and others patient specific. Nonadherence to IOP-lowering eyedrops is one of the major barriers to effective glaucoma treatment. There is a wide range of patients who may benefit from non-topical treatment modalities such as selective laser trabeculoplasty and sustained-release drug delivery, including those who face challenges with topical eyedrops, and those who find benefit in independence from daily medication administration. Sustained-release drug delivery options for lowering IOP include the bimatoprost intracameral implant 10 µg (Bim-I), a relatively new treatment option. This article provides expert opinion on the use of the implant based on available literature, supplementary analysis of clinical trial data, and the experience of the authors who have performed thousands of Bim-I administrations. A single administration of Bim-I has demonstrated the potential to reduce IOP and the need for topical IOP-lowering medication for up to 2 years. Evidence suggests that the safety profile of the implant is most favorable in patients with wide-open (Shaffer grade 4) angles. Recommendations regarding the administration procedure and patient follow-up are provided. The objective of this expert opinion is to help optimize patient selection for Bim-I administration, the implantation procedure, follow-up management, and clinical outcomes while minimizing and addressing potential complications.