Well Completion Optimization in Canada Tight Gas Fields Using Ensemble Machine Learning

ObjectivesTo develop and identify machine learning (ML) models using pretreatment 2-deoxy-2-[18F]fluoro-D-glucose ([18F]-FDG)-positron emission tomography (PET)-based radiomic features to differentiate benign from malignant parotid gland diseases (PGDs).Materials and methodsThis retrospective study included 62 patients with 63 PGDs who underwent pretreatment [18F]-FDG-PET/computed tomography (CT). The lesions were assigned to the training (n = 44) and testing (n = 19) cohorts. In total, 49 [18F]-FDG-PET-based radiomic features were utilized to differentiate benign from malignant PGDs using five different conventional ML algorithmic models (random forest, neural network, k-nearest neighbors, logistic regression, and support vector machine) and the deep learning (DL)-based ensemble ML model. In the training cohort, each conventional ML model was constructed using the five most important features selected by the recursive feature elimination method with the tenfold cross-validation and synthetic minority oversampling technique. The DL-based ensemble ML model was constructed using the five most important features of the bagging and multilayer stacking methods. The area under the receiver operating characteristic curves (AUCs) and accuracies were used to compare predictive performances.ResultsIn total, 24 benign and 39 malignant PGDs were identified. Metabolic tumor volume and four GLSZM features (GLSZM_ZSE, GLSZM_SZE, GLSZM_GLNU, and GLSZM_ZSNU) were the five most important radiomic features. All five features except GLSZM_SZE were significantly higher in malignant PGDs than in benign ones (each p < 0.05). The DL-based ensemble ML model had the best performing classifier in the training and testing cohorts (AUC = 1.000, accuracy = 1.000 vs AUC = 0.976, accuracy = 0.947).ConclusionsThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can be useful for differentiating benign from malignant PGDs.Second abstractThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can overcome the previously reported limitation of [18F]-FDG-PET/CT scan for differentiating benign from malignant PGDs. The DL-based ensemble ML approach using [18F]-FDG-PET-based radiomic features can provide useful information for managing PGD.

Harmful Algal Blooms (HABs), driven by environmental pollution, pose significant threats to water quality, public health, and aquatic ecosystems. This study aims to enhance the prediction of HABs in Lake Erie, part of the Great Lakes system, by utilizing ensemble machine learning (ML) models coupled with explainable artificial intelligence (XAI) for interpretability. Using water quality data from 2013 to 2020, various physical, chemical, and biological parameters were analyzed to predict chlorophyll-a (Chl-a) concentrations, a proxy for algal blooms. The study employed multiple ensemble ML models, including Random Forest (RF), Deep Forest (DF), Gradient Boosting (GB), and XGBoost, and compared their performance against individual models such as Support Vector Machine (SVM), Decision Tree (DT), and Multi-Layer Perceptron (MLP). The findings reveal that ensemble models, particularly XGBoost and Deep Forest (DF), achieve superior predictive accuracy with R² values of 0.8517 and 0.8544, respectively. The application of SHapley Additive exPlanations (SHAP) provided insights into the relative importance of input features, identifying Particulate Organic Nitrogen (PON), Particulate Organic Carbon (POC), and Total Phosphorus (TP) as critical factors influencing Chl-a concentrations. This research demonstrates the effectiveness of integrating ensemble ML models with XAI to improve HAB prediction accuracy and interpretability. The results support the development of proactive water quality management strategies and highlight the potential of advanced ML techniques in environmental monitoring.

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

The identification of an effective inhibitor is an important starting step in drug development. Unfortunately, many issues such as the characterization of protein binding sites, the screening library, materials for assays, etc., make drug screening a difficult proposition. As the size of screening libraries increases, more resources will be inefficiently consumed. Thus, new strategies are needed to preprocess and focus a screening library towards a targeted protein. Herein, we report an ensemble machine learning (ML) model to generate a CDK8-focused screening library. The ensemble model consists of six different algorithms optimized for CDK8 inhibitor classification. The models were trained using a CDK8-specific fragment library along with molecules containing CDK8 activity. The optimized ensemble model processed a commercial library containing 1.6 million molecules. This resulted in a CDK8-focused screening library containing 1,672 molecules, a reduction of more than 99.90%. The CDK8-focused library was then subjected to molecular docking, and 25 candidate compounds were selected. Enzymatic assays confirmed six CDK8 inhibitors, with one compound producing an IC50 value of ≤100 nM. Analysis of the ensemble ML model reveals the role of the CDK8 fragment library during training. Structural analysis of molecules reveals the hit compounds to be structurally novel CDK8 inhibitors. Together, the results highlight a pipeline for curating a focused library for a specific protein target, such as CDK8.

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

The evolution of Electronic Health Records (EHR) has facilitated comprehensive patient record-keeping, enhancing healthcare delivery and decision-making processes. Despite these advancements, analyzing EHR data using ensemble machine learning methods poses unique challenges. These challenges include data dimensionality, imbalanced class distributions, and the need for effective hyperparameter tuning to optimize model performance. The study conducted a thorough comparative analysis of various ensemble machine learning (EML) models using Electronic Health Record (EHR) datasets. After addressing data imbalance and reducing dimensionality, the accuracy of the EML models showed significant improvement. Notably, the Gradient Boosting Machine (GBM) and CatBoost models exhibited superior performance with an accuracy of 73%, achieved through experiments involving dimensionality reduction and handling of imbalanced data. Furthermore, optimization techniques such as Grid Search and Random Search were employed to enhance the EML models. The results of model optimization revealed that the GBM + Random Search model performed the best, achieving an accuracy of 74%, followed by the XGBoost + Grid Search model with an accuracy of 73%. The GBM model also excelled in distinguishing between positive and negative classes, boasting the highest Area under Curve (AUC) value of 0.78, indicative of its superior classification capabilities compared to other models. This study emphasizes the significance of incorporating cutting-edge EML techniques into clinical workflows and emphasizes the revolutionary potential of GBM in classification modeling for patient severity conditions. Future research should focus on deep learning (DL) applications and the integration of these models.

Ensemble machine learning prediction of compressive strength in waste-derived sulfoaluminate cement paste.

Ensemble machine learning (ML) demonstrated potential for improving predictions based on big health care data. We developed and validated interpretable ensemble ML models in evaluating the nonadherence and nonpersistence of biological or targeted synthetic disease-modifying antirheumatic drugs (b/tsDMARDs) in rheumatoid arthritis (RA). This retrospective study used 5% Medicare claims data including older (aged ≥65 years) patients initiating b/tsDMARDs (the index date) between 2013 and 2019, who were diagnosed with RA (the International Classification of Diseases, Ninth and Tenth revision codes). Nonadherence, defined as medication possession ratio <80%, was evaluated during the 12-month follow-up. Nonpersistence was defined as duration (in days) from the index date until treatment gap ≥60 days during the follow-up period, with death and insurance disenrollment as censoring criteria. Data were split into 75% for model training and 25% for evaluation, with patients' demographics and clinical characteristics measured in the baseline period. Machine learning ensemble classification models (random forest, eXtreme Gradient Boosting [XGBoost], and prediction rule ensembles [PRE]) evaluated the binary nonadherence outcome and ML ensemble survival models (random survival forest, XGBoost survival, and PRE survival) assessed time until nonpersistence. Using the testing cohort, concordance index (C-index) was reported for the ML survival models and AUC was reported for ML classification models. We identified 3927 eligible patients with RA (mean age 73 ± 6 years; 75% female). About 53.65% of them were adherent (medication possession ratio 0.72 [0.31]; 0.01-1) and 18.5% of them had the risk of nonpersistence (mean time to nonpersistence 1110 ± 670 days) in the 12-month follow-up. Compared with PRE (AUC, 0.6271; 95% CI, 0.5923-0.6618), random forest (0.6315; 95% CI, 0.5969-0.6661; P = 0.7127) and XGBoost (0.6277; 95% CI, 0.5930-0.6624; P = 0.9643) had comparable performance in predicting nonadherence. Age, claims-based index for RA severity, type of b/tsDMARDs, frailty score, cancer and Elixhauser comorbidity index were ranked commonly as the top features for nonadherence. With reference to PRE survival (C-index, 0.634; 95% CI, 0.611-0.658), random survival forest (C-index, 0.661; 95% CI, 0.665-0.675; P = 0.0276) and XGBoost Survival (C-index, 0.670; 95% CI, 0.666-0.676; P = 0.0033) had improved performance for predicting nonpersistence. Age, Elixhauser score, frailty score, and claims-based index for RA severity were commonly found as the top predictors for nonpersistence. We developed ML models predicting nonadherence and nonpersistence of b/tsDMARDs for older adults with RA. There is potential to leverage ML for understanding patients' behavior of utilizing b/tsDMARDs treatment with a goal towards personalized medicine in RA.

Understanding the spatial distribution of soil organic carbon (SOC) content over different climatic regions will enhance our knowledge of carbon gains and losses due to climatic change. However, little is known about the SOC content in the contrasting arid and sub-humid regions of Iran, whose complex SOC–landscape relationships pose a challenge to spatial analysis. Machine learning (ML) models with a digital soil mapping framework can solve such complex relationships. Current research focusses on ensemble ML models to increase the accuracy of prediction. The usual ensemble method is boosting or weighted averaging. This study proposes a novel ensemble technique: the stacking of multiple ML models through a meta-learning model. In addition, we tested the ensemble through rescanning the covariate space to maximize the prediction accuracy. We first applied six state-of-the-art ML models (i.e., Cubist, random forests (RF), extreme gradient boosting (XGBoost), classical artificial neural network models (ANN), neural network ensemble based on model averaging (AvNNet), and deep learning neural networks (DNN)) to predict and map the spatial distribution of SOC content at six soil depth intervals for both regions. In addition, the stacking of multiple ML models through a meta-learning model with/without rescanning the covariate space were tested and applied to maximize the prediction accuracy. Out of six ML models, the DNN resulted in the best modeling accuracies, followed by RF, XGBoost, AvNNet, ANN, and Cubist. Importantly, the stacking of models indicated a significant improvement in the prediction of SOC content, especially when combined with rescanning the covariate space. For instance, the RMSE values for SOC content prediction of the upper 0–5 cm of the soil profiles of the arid site and the sub-humid site by the proposed stacking approaches were 17% and 9% respectively, less than that obtained by the DNN models—the best individual model. This indicates that rescanning the original covariate space by a meta-learning model can extract more information and improve the SOC content prediction accuracy. Overall, our results suggest that the stacking of diverse sets of models could be used to more accurately estimate the spatial distribution of SOC content in different climatic regions.

Managed aquifer recharge (MAR) replenishes groundwater by artificially entering water into subsurface aquifers. This technology improves water storage, reduces over-extraction, and ensures water security in water-scarce or variable environments. MAR systems are complex, encompassing various components such as water storage, soil, meteorological factors, groundwater management (GWM), and receiving bodies. Over the past decade, the utilization of machine learning (ML) methodologies for MAR modeling and prediction has increased significantly. This review evaluates all supervised, semi-supervised, unsupervised, and ensemble ML models employed to predict MAR factors and parameters, rendering it the most comprehensive contemporary review on this subject. This study presents a concise and integrated overview of MAR’s most effective ML approaches, focusing on design, suitability for water quality (WQ) applications, and GWM. The paper examines performance measures, input specifications, and the variety of ML functions employed in GWM, and highlights prospects. It also offers suggestions for utilizing ML in MAR, addressing issues related to physical aspects, technical advancements, and case studies. Additionally, previous research on ML-based data-driven and soft sensing techniques for MAR is critically evaluated. The study concludes that integrating ML into MAR systems holds significant promise for optimizing WQ management and enhancing the efficiency of groundwater replenishment strategies.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11356-024-35529-3.

BackgroundSepsis-associated acute kidney injury (S-AKI) is a major contributor to mortality in intensive care units (ICU). Early prediction of mortality risk is crucial to enhance prognosis and optimize clinical decisions. This study aims to develop a 28-day mortality risk prediction model for S-AKI utilizing an explainable ensemble machine learning (ML) algorithm.MethodsThis study utilized data from the Medical Information Mart for Intensive Care IV (MIMIC-IV 2.0) database to gather information on patients with S-AKI. Univariate regression, correlation analysis and Boruta were combined for feature selection. To construct the four ML models, hyperparameters were tuned via random search and five-fold cross-validation. To evaluate the performance of all models, ROC, K-S, and LIFT curves were used. The discrimination of ML models and traditional scoring systems was compared using area under the receiver operating characteristic curve (AUC). Additionally, the SHapley Additive exPlanation (SHAP) was utilized to interpret the ML model and identify essential variables. To investigate the relationship between the top nine continuous variables and the risk of 28-day mortality. COX regression-restricted cubic splines were utilized while controlling for age and comorbidities.ResultsThe study analyzed data from 9,158 patients with S-AKI, dividing them into a 28-day mortality group of 1,940 and a survival group of 7,578. The results showed that XGBoost was the best performing model of the four ML models with AUC of 0.873. All models outperformed APS-III 0.713 and SAPS-II 0.681. The K-S and LIFT curves indicated XGBoost as the most effective predictor for 28-day mortality risk. The model’s performance was evaluated using ROCpr curves, calibration curves, accuracy, precision, and F1 scores. SHAP force plots were utilized to interpret and visualize the personalized predictive power of the 28-day mortality risk model. Additionally, COX regression restricted cubic splines revealed an interesting non-linear relationship between the top nine variables and 28-day mortality.ConclusionThe use of ensemble ML models has shown to be more effective than the LR model and conventional scoring systems in predicting 28-day mortality risk in S-AKI patients. By visualizing the XGBoost model with the best predictive performance, clinicians are able to identify high-risk patients early on and improve prognosis.

Background: The rapidly expanding concept, the Internet of Things (IoT), connects a large number of devices to the internet for effective real-time data sharing and communication. But as IoT technology advances, new security risks also arise, and for this reason, machine learning (ML)-based intrusion detection systems (IDS), particularly anomaly-based ones, have emerged as a crucial defence against these new dangers to IoT networks. Objective: The primary goal of this study is to develop and assess anomaly-based IDS based on ML techniques for use in Internet of Things environments. The goal is to improve the models' performance in terms of accuracy and usefulness. Methods: The first phase of the study is a thorough review of the recent studies in the field to support the model design. The steps of the experimental stage include data preprocessing, encoding, and normalization. Data balancing was achieved using the SMOTEENN technique. The experiments and validation studies were performed on the UNSW-NB15 dataset. The employed ML algorithms for the study include DT, DNN, RF, XGBoost, and KNN, and the performance was in terms of precision, recall, accuracy, and F1-score. Results: XGBoost recorded the best detection accuracy (96.37%), followed by RF and DNN. The models were evaluated on both normal and attack traffic, and the suggested model outperformed many existing approaches in recent literature. Conclusion: The experimental findings confirm the suitability of hybrid and ensemble ML models in improving intrusion detection performance in IoT systems. Future research should consider integrating real-time datasets and combining deep learning with ensemble methods to develop more robust and adaptive models.

BackgroundThe prevalence of mental health problems in adolescents is a global public health concern. Machine learning (ML) can analyse multidimensional data collected by phone sensors (passive tracking) and symptom self-reporting (active tracking) to model and predict mental health states. However, research in adolescents is lacking. This study investigated the feasibility of using ML to predict mental health in a non-clinical population of adolescents from active and passive data collected through the Mindcraft mobile app.MethodsWe recruited 103 secondary students (aged 14-18) who first completed the Strengths and Difficulties Questionnaire (SDQ) and then used the Mindcraft app for two weeks. The app integrates sensor data (location, steps, noise, light, battery, app usage) and self-reports. We extracted 85 sensor-based features and 19 from questionnaires to develop a gradient-boosted ensemble ML model. This model identifies students with abnormal SDQ scores and was evaluated using leave-one-out cross-validation.ResultsAll students completed the active questionnaires; 67 also provided passive sensor data. 31 had abnormal SDQ scores. Using passive data, the ML model reached a balanced accuracy of 0.64 and an AUC score of 0.57. Incorporating both data types improved accuracy to 0.67 and AUC to 0.70. Feature importance scores showed step count and questionnaire on racing thoughts and hopefulness as most significant.ConclusionsOur findings affirm the potential of integrating passive and active data collection to monitor mental health in real-world settings. Our ML model demonstrated moderate accuracy in predicting adolescents’ mental health risk measured with a behavioural screening questionnaire. This study will guide the development of a ML-driven, personalised mobile intervention for early symptom detection and mental health self-management, which might constitute an innovative and scalable tool for mental health prevention and intervention in adolescents.Key messages• This study shows the feasibility of using active and passive data to predict mental health risk in adolescents.• This study informs the development of a personalised machine learning-driven mobile intervention.

A study on road accident prediction and contributing factors using explainable machine learning models: analysis and performance

Investigating soil swelling potential is indeed a critical research area in geotechnical engineering, given its significant influence on the stability and longevity of civil structures. This study aims to predict and assess the one-dimensional vertical free swelling potential of soils using ensemble machine learning models. Within the study context, a large dataset encompassing a wide array of soil parameters from 210 soil samples, including moisture content, unit weight, plasticity, and clay content, will be used. These parameters are critical in understanding the swelling behavior of soils under varying environmental and load conditions. The novel approach of this research lies in the application of ensemble machine learning techniques, which offer a robust framework to analyze complex, nonlinear relationships within soil properties. Another key aspect of this research is the parametric assessment, where the influence of individual soil properties on swelling potential is investigated using feature importance and partial dependence analyses. These analyses provide valuable insights into the relative importance of different soil parameters on soil behavior. The outcomes of this study contribute to soil mechanics and machine learning applications in geotechnical engineering and offer practical implications for engineers and practitioners. Besides, the predictive models developed in this study aid in more informed decision-making in the design and construction of civil structures, particularly in swelling-prone areas.