Use of Tree-Based Machine Learning Methods for Stratigraphic Classification in 3D Geological Modelling

A study on road accident prediction and contributing factors using explainable machine learning models: analysis and performance

Modeling of necking area reduction of carbon steel in hydrogen environment using machine learning approach

Characterizing the 3D distribution of hydraulic properties in glacial sediments is challenging due to fine‐scale heterogeneity and complexity. Borehole lithological data provide high vertical resolution but low horizontal resolution. Geophysical methods can fill gaps between boreholes, providing improved horizontal resolution but low vertical resolution. Machine learning can combine borehole and geophysical data to overcome these challenges. However, few studies have compared multiple machine learning methods for predicting hydrofacies in glacial aquifer systems. This study uses colocated airborne electromagnetic resistivity and borehole lithology data to train multiple machine learning models and predict the 3D distribution of hydrofacies in glacial deposits of eastern Nebraska, USA. Random Forest, Gradient Boosting Classifier, Extreme Gradient Boosting, Multilayer Perceptron, and Stacking Classifier were used to model 3D probabilistic distributions of hydrofacies (sand and clay) at a grid size of 200 m × 200 m × 3 m. Comparison of the predicted 3D hydrofacies models shows that the probability distributions and the contrasts between hydrofacies vary. The classification metrics show that the Stacking Classifier model performed better than other machine learning models in predicting hydrofacies. Multi‐Layer Perceptron and Stacking Classifier models show sharp vertical transitions between the low and high sand probability while other machine learning models show gradual transitions. K‐means clustering was used to translate the Stacking Classifier model into a 4‐class hydraulic conductivity model. This study shows that machine learning methods advance our understanding of glacial hydrogeology by improving the vertical and horizontal resolution of hydrofacies distribution and resolving aquifer‐aquifer and stream‐aquifer connections.

Estimating whether to treat the rupture risk of small intracranial aneurysms (IAs) with size ≤ 7mm in diameter is difficult but crucial. We aimed to construct and externally validate a convenient machine learning (ML) model for assessing the rupture risk of small IAs. One thousand four patients with small IAs recruited from two hospitals were included in our retrospective research. The patients at hospital 1 were stratified into training (70%) and internal validation set (30%) randomly, and the patients at hospital 2 were used for external validation. We selected predictive features using the least absolute shrinkage and selection operator (LASSO) method and constructed five ML models applying diverse algorithms including random forest classifier (RFC), categorical boosting (CatBoost), support vector machine (SVM) with linear kernel, light gradient boosting machine (LightGBM), and extreme gradient boosting (XGBoost). The Shapley Additive Explanations (SHAP) analysis provided interpretation for the best ML model. The training, internal, and external validation cohorts included 658, 282, and 64 IAs, respectively. The best performance was presented by SVM as AUC of 0.817 in the internal [95% confidence interval (CI), 0.769-0.866] and 0.893 in the external (95% CI, 0.808-0.979) validation cohorts, which overperformed compared with the PHASES score significantly (all P < 0.001). SHAP analysis showed maximum size, location, and irregular shape were the top three important features to predict rupture. Our SVM model based on readily accessible features presented satisfying ability of discrimination in predicting the rupture IAs with small size. Morphological parameters made important contributions to prediction result.

For effective management of water quantity and quality, it is absolutely essential to estimate the pollution level of the existing surface water. This case study aims to evaluate the performance of twelve machine learning (ML) models, including five boosting-based algorithms (adaptive boosting, gradient boosting, histogram-based gradient boosting, light gradient boosting, and extreme gradient boosting), three decision tree-based algorithms (decision tree, extra trees, and random forest), and four ANN-based algorithms (multilayer perceptron, radial basis function, deep feed-forward neural network, and convolutional neural network), in estimating the surface water quality of the La Buong River in Vietnam. Water quality data at four monitoring stations alongside the La Buong River for the period 2010–2017 were utilized to calculate the water quality index (WQI). Prediction performance of the ML models was evaluated by using two efficiency statistics (i.e., R2 and RMSE). The results indicated that all twelve ML models have good performance in predicting the WQI but that extreme gradient boosting (XGBoost) has the best performance with the highest accuracy (R2 = 0.989 and RMSE = 0.107). The findings strengthen the argument that ML models, especially XGBoost, may be employed for WQI prediction with a high level of accuracy, which will further improve water quality management.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

Multiple sclerosis (MS) is a heterogeneous immune-mediated disorder of the central nervous system, and primary progressive MS (PPMS) is characterized by insidious injury independent of clinical relapses from disease onset. Predicting disease progression remains an unmet need across the MS disease spectrum. We evaluated statistical and machine learning (ML) models for predicting progression in 596 patients with PPMS from ORATORIO, a randomized, placebo (PBO)-controlled, phase 3 clinical study of ocrelizumab (OCR). Using baseline clinical, magnetic resonance imaging, and biomarker (neurofilament light chain [NfL]) data, we trained 6 ML models to predict a progression event (logistic regression, support vector machines, random forest, neural network, gradient boosting, and extreme gradient boosting) and the time to progression (Cox proportional hazards [PH]) on upper limb function, as measured by 20% confirmed worsening on the 9-hole peg test (9HPT). Predictive performance was similar across the six ML models (area under the curve = 0.63 to 0.67) and was modest using the Cox PH model (C-index = 0.76, an improvement over previous studies). In the Cox PH model, treatment arm, baseline 9HPT, neurologic functional system scores, and serum NfL levels were identified as factors associated with progression; model performance was similar across the OCR, PBO, and combined OCR + PBO treatment arms. Accounting for potential interactions with treatment arm resulted in similar predictive performance. To our knowledge, this is the first study to systematically evaluate ML models and statistical methods for predicting progression in PPMS using a well-characterized clinical trial cohort.

Shale caprocks, often organic-rich, play a significant role in hydrogen sequestration, but their wettability is typically altered from water-wet to H2-wet due to the organic matter, compromising their sealing performance. Silica nanofluids can restore water-wet conditions, which is vital for effective containment of H2 in geological formations. However, laboratory experiments to test the effects of nanomaterials on wettability and interfacial tension (IFT) are resource-intensive and time-consuming. To address this challenge, data-driven machine learning (ML) models offer a promising alternative, allowing for the prediction of key UHS parameters—such as advancing and receding contact angles without the need for extensive laboratory testing. We have applied multiple ML models, including gradient boosting, extreme gradient boosting (XGboost), Catboost, random forest, and extra trees to predict the wettability of organic rich shale samples treated with silica nanofluids. The ML models were trained on a comprehensive dataset generated through laboratory experiments conducted under a wide range of pressure (0.5 – 1600psi) and temperature (298-323 K) conditions, using brine (1 wt.% KCl + 2 wt.% NaCl) and varying silica nanofluid concentrations (0.05–1.0 wt.%). Advanced visualization techniques, including swarm maps, box plots, and heatmap plots, were utilized to analyze the experimental dataset thoroughly. To enhance the generalization capabilities of the ML models, we employed Bayesian optimization for hyperparameter tuning, along with k-fold cross-validation and grid-search optimization techniques. Both statistical and graphical analyses were performed to show the reliability and performance of the models. The results demonstrated that the implemented ML models, particularly the XGboost and Catboost, accurately predicted the wettability behavior under various operating conditions. The training and testing average R² scores were above 0.998 and 0.949, respectively, confirming the high accuracy of the predictions. These findings highlight the reliability of the ML algorithms in predicting the complex interaction between shale, brine, and H2. Further investigation into the feature importance revealed that nanomaterials had the greatest influence on the wettability of the ternary system. The silica nanofluid concentrations played a key role in modifying wettability, with varying impacts on the receding contact angles, emphasizing the effect of nanomaterial treatment. The model's predictions were consistent with experimental results, suggesting that geological conditions, including pressure and temperature, have a profound influence on rock mineral wettability, directly impacting H2 geo-storage capacity. This study is the first to employ data-driven ML models to predict critical UHS parameters in organic-rich shales treated with silica nanofluids. The ability of the ML models to predict UHS parameters across a wide range of conditions provides a valuable tool for optimizing UHS system designs, offering a more efficient and scalable approach compared to traditional experimental methods.

Discovery of new antibiotics is the need of the hour to treat infectious diseases. An ever-increasing repertoire of multidrug-resistant pathogens poses an imminent threat to human lives across the globe. However, the low success rate of the existing approaches and technologies for antibiotic discovery remains a major bottleneck. In silico methods like machine learning (ML) deem more promising to meet the above challenges compared with the conventional experimental approaches. The goal of this study was to create ML models that may be used to successfully predict new antimicrobial compounds. In this article, we employed eight different ML algorithms namely, extreme gradient boosting, random forest, gradient boosting classifier, deep neural network, support vector machine, multilayer perceptron, decision tree, and logistic regression. These models were trained using a dataset comprising 312 antibiotic drugs and a negative set of 936 non-antibiotic drugs in a five-fold cross validation approach. The top four ML classifiers (extreme gradient boosting, random forest, gradient boosting classifier and deep neural network) were able to achieve an accuracy of 80 per cent and above during the evaluation of testing and blind datasets. We aggregated the top performing four models through a soft-voting technique to develop an ensemble-based ML method and incorporated it into a freely accessible online prediction server named ABDpred ( http://clinicalmedicinessd.com.in/abdpred/ ).

P125. Development of a novel ensemble machine learning algorithm for prediction of complications and readmission after anterior cervical spinal fusion

The prediction of torque capacity in circular Concrete-Filled Double-Skin Tubular (CFDST) members under pure torsion is considered vital for structural design and analysis. In this study, torque capacity is predicted using machine learning (ML) algorithms, such as Categorical Boosting (CatBoost), Extreme Gradient Boosting (XGBoost), Gradient Boosting Machine (GBM), Random Forest (RF), and Decision Tree (DT), which are employed. The interpretation of the results is conducted using Shapley Additive Explanations (SHAPs). The performance of these ML models is evaluated against two traditional analytical formulas that have been proposed and are available in the literature. Through comprehensive analysis, it is shown that superior predictive capabilities are possessed by the CatBoost and XGBoost models, characterized by high R2 values and minimal mean errors. Additionally, insights into the influence of input features are provided by SHAP interpretation, with an emphasis on key parameters such as concrete compressive strength and steel tube dimensions. The gap between empirical models and ML techniques is bridged by this study, offering engineers a more accurate and efficient tool for CFDST structural design. Significant implications for optimizing CFDST column designs and advancing structural engineering practices are presented by these findings. Directions for future research include the further refinement of ML models and the integration of probabilistic analyses for enhanced structural resilience. Overall, the transformative potential of ML and SHAP interpretation in advancing the field of structural engineering is showcased by this study.

Mild cognitive impairment (MCI) is a clinical condition characterized by a decline in cognitive ability and progression of cognitive impairment. It is often considered a transitional stage between normal aging and Alzheimer's disease (AD). This study aimed to compare deep learning (DL) and traditional machine learning (ML) methods in predicting MCI using plasma proteomic biomarkers. A total of 239 adults were selected from the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort along with a pool of 146 plasma proteomic biomarkers. We evaluated seven traditional ML models (support vector machines (SVMs), logistic regression (LR), naïve Bayes (NB), random forest (RF), k-nearest neighbor (KNN), gradient boosting machine (GBM), and extreme gradient boosting (XGBoost)) and six variations of a deep neural network (DNN) model-the DL model in the H2O package. Least Absolute Shrinkage and Selection Operator (LASSO) selected 35 proteomic biomarkers from the pool. Based on grid search, the DNN model with an activation function of "Rectifier With Dropout" with 2 layers and 32 of 35 selected proteomic biomarkers revealed the best model with the highest accuracy of 0.995 and an F1 Score of 0.996, while among seven traditional ML methods, XGBoost was the best with an accuracy of 0.986 and an F1 Score of 0.985. Several biomarkers were correlated with the APOE-ε4 genotype, polygenic hazard score (PHS), and three clinical cerebrospinal fluid biomarkers (Aβ42, tTau, and pTau). Bioinformatics analysis using Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) revealed several molecular functions and pathways associated with the selected biomarkers, including cytokine-cytokine receptor interaction, cholesterol metabolism, and regulation of lipid localization. The results showed that the DL model may represent a promising tool in the prediction of MCI. These plasma proteomic biomarkers may help with early diagnosis, prognostic risk stratification, and early treatment interventions for individuals at risk for MCI.

Prediction of axial load capacity of cold formed lipped channel section using machine learning

Efficient seismic fragility analysis method utilizing ground motion clustering and probabilistic machine learning

Prediction of rockhead using a hybrid N-XGBoost machine learning framework