Some issues with composite endpoints in clinical trials

Composite Endpoints in Clinical Trials

Composite endpoints can encode multiple pieces of information and are increasingly adopted in clinical trials. Advocacy for using composite endpoints began decades ago in cardiovascular trials, leading to incorporation of patient-oriented outcomes and consideration of a hierarchical ranking system. The use of composite endpoints in coronavirus disease (COVID-19) trials has evolved similarly. We conducted a literature review to investigate the use of composite endpoints in acute heart failure and COVID-19 clinical trials. The results showed more frequent use of patient-oriented outcomes and ordinal composite endpoints in COVID-19 trials, which might be driven by global consensus on a set of common outcome measures.

The primary analysis plays a key role in clinical trials to determine efficacy. By providing a single P-value, the primary analysis provides the basis for controlling the overall type I error rate. However, it is often desirable to consider multiple important patient outcomes in the primary analysis in order to use the most comprehensive and most clinically meaningful assessment of the patient. These properties also tend to increase the power of the resulting statistical tests. Composite endpoints are an effective way of combining the information from multiple endpoints when identification of a single primary outcome is not possible. We discuss a variety of ways for forming such composites. The ideal is an existing composite that has been extensively studied, validated, and used in clinical practice, and where a meaningful effect size has been identified. Statistical alternatives that may be useful when such an existing composite is not available include a rank sum approach and least squares methods and other methods described in global assessment variables. Keywords: Composite endpoints; primary endpoint; primary analysis; Bonferroni; multiple testing

assumption is often untenable, necessitating a conceptual framework with greater specificity.

To The Editors: The US Food and Drug Administration (FDA) has held several meetings discussing clinical trial design for studies of acute otitis media (AOM), 1 including discussions of appropriate endpoints for such trials. The results of the study by Piglansky et al. 2 are based on bacteriologic outcomes as the primary endpoint. The premise of utilizing a bacteriologic outcome as the primary endpoint is based on an understanding of the correlation between clinical and bacteriologic outcomes. Although antimicrobials have a direct effect on microorganisms, this effect should ultimately translate into diminution of the symptoms of the infection for which patients seek treatment. In this study the investigators evaluated 50 children with AOM in a double tympanocentesis study using high dose amoxicillin (80 mg/kg/day 3 times a day for 10 days). The bacteriologic success rate at Day 4 to 6 was 72% (36 of 50), whereas the clinical success rate at Day 4 of 6 was 94% (47 of 50). This translates into a number of children designated as bacteriologic failures on the second tympanocentesis who were clinically cured. The paper does not provide a patient-by-patient listing relating each patient’s bacterial and clinical outcomes; however, there were three patients described as clinical failures who were also bacteriologic failures. Using these numbers we calculated the correlation between the bacteriologic and clinical outcomes (Table 1). 3 The kappa coefficient for the study was 0.28 with a standard deviation of 0.13, showing only fair agreement between the two endpoints.TABLE 1: Comparison of bacteriologic and clinical outcomes in three double tympanocentesis acute otitis media studiesAt a recent FDA meeting on clinical trial study considerations for AOM, 1 several presentations advocated the use of bacteriologic endpoints in double tympanocentesis studies as the primary endpoint. However, the kappa coefficient for the two studies 4, 5 discussed at the advisory meeting (kappa = 0.34 and kappa = 0.35) also showed less than substantial 3 correlation between bacteriologic and clinical endpoints. The cited studies supported a “correlation”4, 5 between children who clinically failed treatment and those who were culture-positive on the second tympanocentesis. To evaluate a true correlation, however, one must also look at the number of children who were clinical successes who remained culture-positive. In both studies as well as the current study by Piglansky et al., 2 substantial numbers of children were clinically cured despite positive cultures on the second tympanocentesis. The chi square test and the associated P value presented in these studies show an association of the two variables but do not provide a quantitative measure of the degree of the relationship between clinical and bacteriologic outcomes. The significant association of the two outcomes is driven by the high sensitivity (number of clinical and bacteriologic success patients/total bacteriologic success patients) of clinical failure for predicting bacteriologic failure. However, the specificity of bacteriologic failure for predicting clinical failure (number of clinical and bacteriologic failures/total bacteriologic failure patients) is too low for use in a clinical trial setting when attempting to differentiate outcomes between treatment arms (Table 1). Although high sensitivity is desirable in both clinical practice and clinical trials, high specificity is also important in clinical trials attempting to assess antimicrobial efficacy. Double tympanocentesis studies in clinical trials for AOM are important for assessment of the bacteriology and the natural history of the disease. However, the less than substantial correlation between bacteriologic and clinical outcomes raises significant questions about using bacteriologic outcomes as the sole measure of drug efficacy. Authors have claimed that clinical endpoints do not sufficiently discriminate between the efficacies of various therapies in AOM. 6 However, it may be that clinical resolution measured at a fixed time point well beyond the expected time of natural resolution of the disease (Day 10 or Day 28) may not provide the most useful information. Time-to-resolution of symptoms, used in other indications with high spontaneous cure rates, 7 may be more meaningful clinical endpoints in AOM clinical trials. Perhaps the intrapatient correlation between the two endpoints would be improved with use of such clinical endpoints. Rosemary Johann-Liang, M.D. Jyoti Zalkikar, Ph.D. John H. Powers, M.D.

Reply: Composite endpoints in clinical trials - simplicity or perfection?

Using GFR, Albuminuria, and Their Changes in Clinical Trials and Clinical Care

Objective: To describe the current status of registration and design characteristics of clinical trials of new drugs for curing hepatitis B through domestic and foreign websites, so as to provide references for the follow-up clinical trials of new hepatitis B drugs. Methods: A search was conducted on the US Clinical Trials Database and the Chinese Clinical Trial Registry Center. The search date was from the establishment of the database to May 26, 2020, and the registration trials of new drugs for curing hepatitis B at home and abroad were included. Two researchers independently searched and screened the literature and extracted the data. Results: A total of 106 registered clinical trials of new drugs for curing hepatitis B were included (94 English registration websites and 12 Chinese registration websites), and the number of registrations had increased year by year. Among them, the proportion of therapeutic vaccines and core protein inhibitors were the highest, accounting for 27.4% (n = 29) and 22.6% (n = 24), respectively. The vast majority of clinical trials (n = 96, 90.6%) were in the early stages (Phase I and II). The subjects in phase I clinical trial were mainly healthy people and treated CHB patients, while the subjects in phase II clinical trial were mainly CHB patients who had achieved viral suppression after initial or post-treatment. The main evaluation indicators of Phase I clinical trials were the safety and tolerability of new drugs. The main evaluation indicators in about half of Phase II clinical trials were HBsAg negative conversion/quantitative decline. Overall, the number of clinical trials with the new design was small, accounting for 3.8% (4 / 106). There were relatively few trials of new drugs for curing hepatitis B on domestic registration websites, and the information provided was incomplete. Conclusion: The number of clinical trials of new hepatitis B drugs at home and abroad is increasing year by year, but most of them are in phase I and II, with few adopting new designs. In addition, the information integrity of the domestic website registration center needs to be improved.

Consider the choice of outcome for overall treatment benefit in a clinical trial which measures the first time to each of several clinical events. We describe several new variants of the win ratio that incorporate the time spent in each clinical state over the common follow-up, where clinical state means the worst clinical event that has occurred by that time. One version allows restriction so that death during follow-up is most important, while time spent in other clinical states is still accounted for. Three other variants are described; one is based on the average pairwise win time, one creates a continuous outcome for each participant based on expected win times against a reference distribution and another that uses the estimated distributions of clinical state to compare the treatment arms. Finally, a combination testing approach is described to give robust power for detecting treatment benefit across a broad range of alternatives. These new methods are designed to be closer to the overall treatment benefit/harm from a patient's perspective, compared to the ordinary win ratio. The new methods are compared to the composite event approach and the ordinary win ratio. Simulations show that when overall treatment benefit on death is substantial, the variants based on either the participants' expected win times (EWTs) against a reference distribution or estimated clinical state distributions have substantially higher power than either the pairwise comparison or composite event methods. The methods are illustrated by re-analysis of the trial heart failure: a controlled trial investigating outcomes of exercise training.

Conventional analysis of vascular prevention trials assigns equal weight to disparate vascular events in a composite end point at variance with the public's perception of their differential impact on health outcome. This study sought to apply the disability-adjusted life-year (DALY) metric to differential weighting individual vascular end points in trial analyses. DALY values for the most common major end points in vascular prevention trials (nonfatal myocardial infarction, nonfatal stroke, and vascular death), were derived by using World Health Organization Global Burden of Disease Project methodology. The standardized DALYs for each event were applied to recent major primary and secondary vascular prevention trials and to hypothetical model trials. Standardized DALYs lost were 7.63 for nonfatal stroke, 5.14 for nonfatal myocardial infarction, and 11.59 for vascular death. In the published trials analyses, the direction of treatment effects was consistent between DALY and standard event analysis, but the rank order of treatment effect changed for 10 of 18 trials. The DALY analysis also permitted derivation of number-needed-to-treat values to gain 1 DALY: 2.1 for anticoagulation in atrial fibrillation, 2.7 for carotid endarterectomy in symptomatic stenosis, and 4.7 for clopidogrel added to aspirin in acute coronary syndrome. Hypothetical trial analyses demonstrated that the DALY metric more finely discriminates treatment effects. Compared with a nonfatal myocardial infarction, a nonfatal stroke causes a 1.48-fold greater loss and vascular death a 2.25-fold greater loss of DALY. DALY analysis integrates these valuations in a summary metric reflecting the net impact of therapy on patient and societal health, complementing conventional end point analyses.

Composite endpoints in clinical trials: the “win ratio”

Treating the Black Hypertensive in 2010: Achieve Lower Targets While Awaiting More Definitive Evidence

CKJ-5 - Investigator-Initiated Clinical Trials for New Drugs in Japan: Challenges and Perspectives

Quality of life assessment using patient-reported outcome (PRO) measures: still a Cinderella outcome?

Japanese guideline for clinical research of antimicrobial agents on urogenital infections: Second edition.