• Home
  • Search
  • Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error
  • Cite Icon24
  • https://doi.org/10.1186/s13063-020-4076-yCopy DOI Icon

Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error

  • Feb 10, 2020
  • Trials
  • James A Watson +1 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

BackgroundRetrospective exploratory analyses of randomised controlled trials (RCTs) seeking to identify treatment effect heterogeneity (TEH) are prone to bias and false positives. Yet the desire to learn all we can from exhaustive data measurements on trial participants motivates the inclusion of such analyses within RCTs. Moreover, widespread advances in machine learning (ML) methods hold potential to utilise such data to identify subjects exhibiting heterogeneous treatment response.MethodsWe present a novel analysis strategy for detecting TEH in randomised data using ML methods, whilst ensuring proper control of the false positive discovery rate. Our approach uses random data partitioning with statistical or ML-based prediction on held-out data. This method can test for both crossover TEH (switch in optimal treatment) and non-crossover TEH (systematic variation in benefit across patients). The former is done via a two-sample hypothesis test measuring overall predictive performance. The latter is done via ‘stacking’ the ML predictors alongside a classical statistical model to formally test the added benefit of the ML algorithm. An adaptation of recent statistical theory allows for the construction of a valid aggregate p value. This testing strategy is independent of the choice of ML method.ResultsWe demonstrate our approach with a re-analysis of the SEAQUAMAT trial, which compared quinine to artesunate for the treatment of severe malaria in Asian adults. We find no evidence for any subgroup who would benefit from a change in treatment from the current standard of care, artesunate, but strong evidence for significant TEH within the artesunate treatment group. In particular, we find that artesunate provides a differential benefit to patients with high numbers of circulating ring stage parasites.ConclusionsML analysis plans using computational notebooks (documents linked to a programming language that capture the model parameter settings, data processing choices, and evaluation criteria) along with version control can improve the robustness and transparency of RCT exploratory analyses. A data-partitioning algorithm allows researchers to apply the latest ML techniques safe in the knowledge that any declared associations are statistically significant at a user-defined level.

Loading PDF

Similar Papers
  • Research Article
  • Citations27

Parameter importance assessment improves efficacy of machine learning methods for predicting snow avalanche sites in Leh-Manali Highway, India

  • Jun 29, 2021
  • Science of the Total Environment
  • Anuj Tiwari +2
  • Abstract

P125. Development of a novel ensemble machine learning algorithm for prediction of complications and readmission after anterior cervical spinal fusion

  • Aug 10, 2021
  • The Spine Journal
  • Akash A Shah +7
  • Research Article
  • Citations33

Performance evaluation of machine learning techniques in predicting cumulative absolute velocity

  • Aug 10, 2023
  • Soil Dynamics and Earthquake Engineering
  • Fahrettin Kuran +2
  • Research Article
  • Citations37

Machine Learning for Active Portfolio Management

  • Jul 12, 2021
  • The Journal of Financial Data Science
  • Söhnke M Bartram +3
  • Research Article

Machine Learning Approaches to Forecast the Realized Volatility of Crude Oil Prices

  • Jan 22, 2026
  • Journal of Forecasting
  • Talha Omer +3
  • Research Article
  • Citations144

Advances in Machine Learning Algorithms for Hate Speech Detection in Social Media: A Review

  • Jan 01, 2021
  • IEEE Access
  • Nanlir Sallau Mullah +1
  • Research Article
  • Citations3

Failure in Stock Price Prediction: A Comparison bettwen the Curve-Shape-Feature and Non-Curve-Shape-Feature Modes of Existing Machine Learning Algorithms

  • Dec 03, 2021
  • INTERNATIONAL JOURNAL OF COMPUTERS COMMUNICATIONS & CONTROL
  • Ping Zhang +5
  • Research Article
  • Citations1

Research Methods in Machine Learning: A Content Analysis

  • May 03, 2022
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Mayank Shukla
  • Book Chapter

Foundation of Machine Learning-Based Data Classification Techniques for Health Care

  • May 25, 2021
  • Bindu Babu +2
  • PDF
  • Research Article
  • Citations118

Review: Application and Prospective Discussion of Machine Learning for the Management of Dairy Farms.

  • Sep 18, 2020
  • Animals
  • Marianne Cockburn
  • Research Article
  • Citations747

A review of machine learning applications in wildfire science and management

  • Jul 28, 2020
  • Environmental Reviews
  • Piyush Jain +5
  • PDF
  • Research Article
  • Citations7

Integrating Bioinformatics and Machine Learning for Genomic Prediction in Chickens.

  • May 26, 2024
  • Genes
  • Xiaochang Li +4
  • Research Article
  • Citations3

Prediction of dose deposition matrix using voxel features driven machine learning approach.

  • Mar 06, 2023
  • The British Journal of Radiology
  • Shengxiu Jiao +2
  • Research Article
  • Citations1327

A systematic literature review of machine learning methods applied to predictive maintenance

  • Sep 05, 2019
  • Computers & Industrial Engineering
  • Thyago P Carvalho +5
  • Research Article
  • Citations2

Machine learning applications of network security enhancement: review

  • Oct 14, 2024
  • Computer Science & IT Research Journal
  • Abbas A Mahdi
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.