Fundamental error in tree-based machine learning model selection for reservoir characterisation

Accurate estimation of water saturation (Sw) is critical for hydrocarbon exploration reservoir management, as it reveals the proportion of pore spaces filled with hydrocarbons and water. Determining Sw could be challenging in the absence of core data or resistivity logs. This informed the use of machine learning (ML) techniques to predict Sw in the "TM" Field, Niger Delta, where missing resistivity log data poses a challenge. Five ML models (XGBoost, AdaBoost, CatBoost, LightGBM, and Gradient Boost) were deployed using well log data (caliper, gamma-ray, neutron, porosity, density, and shale volume) to estimate Sw at “TM” Field, Niger-Delta. The dataset includes 61,253 observations, which were split into training (70%) and testing (30%) sets. After preprocessing and correcting inconsistencies in the data, the five ML models were trained and hyperparameters tuned to optimize performance. The models were evaluated using standard statistical metrics: Root Mean Square Error (RMSE), Mean Squared Error (MSE), Mean Absolute Error (MAE), and R-squared (R²). To validate the performance of these models, predicted Sw values were compared with those estimated from resistivity logs. Likewise the predicted Sw from the five ML models were plotted against the Sw estimated from the resistivity data not used in the prediction process to validate and determine the quality of the predicted Sw from the ML models. Among the five ML models tested, XGBoost exhibited the best performance, with the highest R² value of 0.9992 and the lowest RMSE of 0.0071. Other models, such as CatBoost, LightGBM, and Gradient Boost, showed strong performance with correlation coefficients of 0.9785, 0.9732, and 0.9299, respectively, but were less accurate than XGBoost. AdaBoost, on the other hand, demonstrated the poorest performance with a correlation coefficient of 0.4381 and the highest RMSE of 0.2082. The cross plot of the predicted Sw from XGBoost's model and actual Sw from Archie's equation had the highest correlation coefficient of 0.9, providing quality prediction thereby aligning with the statistical metrics. Hence, this study has identified Xgboost to be a promising ML tool that could be used to efficiently predict Sw without the use of resistivity data at “TM” Field Niger Delta and this could be applied in other similar geological settings.

In this study, three machine learning techniques, the XGBoost (Extreme Gradient Boosting), LSTM (Long Short-Term Memory Networks), and ARIMA (Autoregressive Integrated Moving Average Model), are utilized to deal with the time series prediction tasks for coastal bridge engineering. The performance of these techniques is comparatively demonstrated in three typical cases, the wave-load-on-deck under regular waves, structural displacement under combined wind and wave loads, and wave height variation along with typhoon/hurricane approaching. To enhance the prediction accuracy, a typical data preprocessing method is adopted and an improved prediction framework for the LSTM model after the rolling forecast prediction is proposed. The obtained results show that: (a) When making a prediction on data featured with periodic regularity, both the XGBoost and ARIMA models perform well, and the XGBoost model can make predictions multi-step ahead, (b) The ARIMA model can predict just one step ahead based on aperiodic dataset with limited amplitude more accurately, while the XGBoost and LSTM models can predict multi-step ahead with appropriate data preprocessing, and (c) All the three models can predict the data tendency with model updating over time, but the prediction accuracy of the LSTM model is more favorable. The successful application of these three machine learning techniques can provide guidance to resolve engineering problems with time-history prediction requirements.

Background This study aimed to establish and validate a machine learning (ML) model for predicting in-hospital mortality in critically ill patients with chronic kidney disease (CKD). Methods This study collected data on CKD patients from 2008 to 2019 using the Medical Information Mart for Intensive Care IV. Six ML approaches were used to build the model. Accuracy and area under the curve (AUC) were used to choose the best model. In addition, the best model was interpreted using SHapley Additive exPlanations (SHAP) values. Results There were 8527 CKD patients eligible for participation; the median age was 75.1 (interquartile range: 65.0–83.5) years, and 61.7% (5259/8527) were male. We developed six ML models with clinical variables as input factors. Among the six models developed, the eXtreme Gradient Boosting (XGBoost) model had the highest AUC, at 0.860. According to the SHAP values, the sequential organ failure assessment score, urine output, respiratory rate, and simplified acute physiology score II were the four most influential variables in the XGBoost model. Conclusions In conclusion, we successfully developed and validated ML models for predicting mortality in critically ill patients with CKD. Among all ML models, the XGBoost model is the most effective ML model that can help clinicians accurately manage and implement early interventions, which may reduce mortality in critically ill CKD patients with a high risk of death.

ObjectivesA radiomics-based explainable eXtreme Gradient Boosting (XGBoost) model was developed to predict central cervical lymph node metastasis (CCLNM) in patients with papillary thyroid carcinoma (PTC), including positive and negative effects.MethodsA total of 587 PTC patients admitted at Binzhou Medical University Hospital from 2017 to 2021 were analyzed retrospectively. The patients were randomized into the training and test cohorts with an 8:2 ratio. Radiomics features were extracted from ultrasound images of the primary PTC lesions. The minimum redundancy maximum relevance algorithm and the least absolute shrinkage and selection operator regression were used to select CCLNM positively-related features and radiomics scores were constructed. Clinical features, ultrasound features, and radiomics score were screened out by the Boruta algorithm, and the XGBoost model was constructed from these characteristics. SHapley Additive exPlanations (SHAP) was used for individualized and visualized interpretation. SHAP addressed the cognitive opacity of machine learning models.ResultsEleven radiomics features were used to calculate the radiomics score. Five critical elements were used to build the XGBoost model: capsular invasion, radiomics score, diameter, age, and calcification. The area under the curve was 91.53% and 90.88% in the training and test cohorts, respectively. SHAP plots showed the influence of each parameter on the XGBoost model, including positive (i.e., capsular invasion, radiomics score, diameter, and calcification) and negative (i.e., age) impacts. The XGBoost model outperformed the radiologist, increasing the AUC by 44%.ConclusionsThe radiomics-based XGBoost model predicted CCLNM in PTC patients. Visual interpretation using SHAP made the model an effective tool for preoperative guidance of clinical procedures, including positive and negative impacts.

The green concretes industry benefits from utilizing gel to replace parts of the cement in concretes. However, measuring the compressive strength of geo-polymer concretes (CSGPoC) needs a significant amount of work and expenditure. Therefore, the best idea is predicting CSGPoC with a high level of accuracy. To do this, the base learner and super learner machine learning models were proposed in this study to anticipate CSGPoC. The decision tree (DT) is applied as base learner, and the random forest and extreme gradient boosting (XGBoost) techniques are used as super learner system. In this regard, a database was provided involving 259 CSGPoC data samples, of which four-fifths of is considered for the training model and one-fifth is selected for the testing models. The values of fly ash, ground-granulated blast-furnace slag (GGBS), Na2SiO3, NaOH, fine aggregate, gravel 4/10 mm, gravel 10/20 mm, water/solids ratio, and NaOH molarity were considered as input of the models to estimate CSGPoC. To evaluate the reliability and performance of the decision tree (DT), XGBoost, and random forest (RF) models, 12 performance evaluation metrics were determined. Based on the obtained results, the highest degree of accuracy is achieved by the XGBoost model with mean absolute error (MAE) of 2.073, mean absolute percentage error (MAPE) of 5.547, Nash-Sutcliffe (NS) of 0.981, correlation coefficient (R) of 0.991, R2 of 0.982, root mean square error (RMSE) of 2.458, Willmott's index (WI) of 0.795, weighted mean absolute percentage error (WMAPE) of 0.046, Bias of 2.073, square index (SI) of 0.054, p of 0.027, mean relative error (MRE) of -0.014, and a20 of 0.983 for the training model and MAE of 2.06, MAPE of 6.553, NS of 0.985, R of 0.993, R2 of 0.986, RMSE of 2.307, WI of 0.818, WMAPE of 0.05, Bias of 2.06, SI of 0.056, p of 0.028, MRE of -0.015, and a20 of 0.949 for the testing model. By importing the testing set into trained models, values of 0.8969, 0.9857, and 0.9424 for R2 were obtained for DT, XGBoost, and RF, respectively, which show the superiority of the XGBoost model in CSGPoC estimation. In conclusion, the XGBoost model is capable of more accurately predicting CSGPoC than DT and RF models.

Background:Systemic lupus erythematosus (SLE) is a chronic autoimmune disease that affects multiple organs and systems. However, in addition to physical damage, patients with SLE have an increased risk of mental...

Optimized machine learning models for predicting crown convergence of plateau mountain tunnels

This study develops predictive models for Chinese female patients with VL utilizing machine learning techniques. The aim is to create an effective model that can assist in clinical diagnosis and treatment of vaginal relaxation, thereby enhancing women’s pelvic floor health. In total, 1184 women with VL have been randomly selected and categorized into groups using the finger measurement method. Among them, there are 383 cases of mild VL, 405 cases of moderate VL, and 396 cases of severe VL. Concurrently, 396 healthy women without VL who underwent routine health examinations have been chosen at random and assigned to the non-VL group. Based on 1580 cases, we have established LightGBM, Random Forest, XGBoost, and AdaBoost models based on training dataset using 5-fold cross-validation and GridSearch, and analyzed the performance of the models on the hold-out test dataset. The confusion matrix, precision, recall, F1 score, overall accuracy, and ROC curve of the models on the hold-out test dataset are compared. The overall accuracy of LightGBM model, RF model, XGBoost model, and AdaBoost model are 0.8987, 0.8987, 0.8987, and 0.8457, respectively. The average AUC of LightGBM model is 0.976, the one of RF model is 0.9763, the one of XGBoost model is 0.9775, and the one of AdaBoost model is 0.928. The XGBoost model has the more comprehensive and reasonable performance among the four prediction models, which can accurately distinguish between healthy, mild VL, as well as moderate VL and severe VL, which can assist doctors in diagnosing persons’ conditions more accurately, devising personalized treatment plans, avoiding unnecessary surgeries, reducing persons’ psychological stress, improving patient compliance and treatment outcomes, thus enhancing overall treatment results.

Compared with coronary artery bypass grafting (CABG) under cardiopulmonary bypass, off-pump coronary artery bypass (OPCAB) is minimally invasive and reduces the risk of intraoperative blood transfusion and acute kidney injury. Nonetheless, OPCAB-related complications still pose a threat. Machine learning technology can analyze a large number of clinical data, establish risk prediction models and help clinicians make early and correct clinical decisions. Risk prediction models are available for mortality and morbidity after cardiac surgery, but they are not specific to OPCAB. This study aimed to develop a predictive model of severe complications after OPCAB, based on machine learning. Anesthesia records of OPCAB from the General Hospital of the Northern Theater Command (Shenyang, China) collected between January 1, 2019, and June 15, 2020, were analyzed. The endpoint of the study was the occurrence of serious complications after OPCAB (postoperative unplanned intra-aortic balloon pump, secondary surgery and death). The features entered into the models were as follows: intraoperative ventricular fibrillation, number of saphenous vein grafts, nerve block (NeB), venous oxygen saturation (SvO2), skin incision-bypass time, and hypertension. A total of 8 machine learning algorithms were tested: logistic regression analysis (LRA), k-nearest neighbor (KNN), naïve Bayes (NB), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical features gradient boosting (CatBoost). Among the 506 patients found in the records, 27 met the endpoint. The highest area under the curve (AUC) value was achieved with the XGBoost model (AUC = 0.94), and the lowest with the SVM model (AUC = 0.75). The highest and lowest accuracy were observed with the XGBoost and NB models, respectively, while the highest and lowest precision were achieved using the SVM and NB models, respectively. Based on the receiver operating characteristic (ROC) curves, the XGBoost model was selected as the most useful in this study. This study suggests using the XGBoost model to predict the risk of complications after OPCAB.

BackgroundHyperkalemia is a common complication of chronic kidney disease (CKD). Hyperkalemia is associated with mortality, CKD progression, hospitalization, and high healthcare costs in patients with CKD. We developed a machine learning model to predict hyperkalemia in patients with advanced CKD at an outpatient clinic.MethodsThis retrospective study included 1,965 advanced CKD patients between January 1, 2010, and December 31, 2020 in Taiwan. We randomly divided all patients into the training (75%) and testing (25%) datasets. The primary outcome was to predict hyperkalemia (K+ > 5.5 mEq/L) in the next clinic vist. Two nephrologists were enrolled in a human-machine competition. The area under the receiver operating characteristic curves (AUCs), sensitivity, specificity, and accuracy were used to evaluate the performance of XGBoost and conventional logistic regression models with that of these physicians.ResultsIn a human-machine competition of hyperkalemia prediction, the AUC, PPV, and accuracy of the XGBoost model were 0.867 (95% confidence interval: 0.840–0.894), 0.700, and 0.933, which was significantly better than that of our clinicians. There were four variables that were chosen as high-ranking variables in XGBoost and logistic regression models, including hemoglobin, the serum potassium level in the previous visit, angiotensin receptor blocker use, and calcium polystyrene sulfonate use.ConclusionsThe XGBoost model provided better predictive performance for hyperkalemia than physicians at the outpatient clinic.

In this work, a study was conducted to investigate the effects of different biodiesel blends with hydrogen peroxide additive on the performance and emissions of an internal combustion engine under various operating parameters. A CI engine was operated with diesel, four dissimilar biodiesels, and H2O2 at various proportions. The biodiesel blends used were Jatropha (D60JB30A10, D60JB34A6, D60JB38A2, D60JB40), Honge (D60HB30A10, D60HB34A6, D60HB38A2, D60HB40), Simarouba (D60SB30A10, D60SB34A6, D60SB38A2, D60SB40), and Neem (D60NB30A10, D60NB34A6, D60NB38A2, D60NB40). The engine was tested at different injection operating pressures (200, 205, and 210 bar), a speed of 1500 rpm, and a CR of 17.5:1. From the experiments conducted, it was highlighted that, under specific conditions, i.e., with an injection pressure of 205 bar, 80% load, a compression ratio of 17.5, an injection timing set at 230 before top dead center, and an engine speed of 1500 rpm, the biodiesel blends D60JB30A10, D60HB30A10, D60SB30A10, and D60NB30A10 achieved the highest brake thermal efficiencies of 24%, 23.9675%, 23.935%, and 23.9025%, respectively. Notably, the blend D60JB30A10 stood out with the highest brake thermal efficiency of 24% among these tested blends. Similarly, when evaluating emissions under the same operational conditions, the D60JB30A10 blend exhibited the lowest emissions levels: CO (0.16% Vol), CO2 (7.8% Vol), HC (59 PPM), and Smoke (60 HSU), while NOx (720 PPM) emissions showed a relative increase with higher concentrations of the hydrogen-based additive. The D60HB30A10, D60SB30A10, and D60NB30A10 blends showed higher emissions in comparison. Additionally, the study suggests that machine learning techniques can be employed to predict engine performance and emission characteristics, thereby cutting down on time and costs associated with traditional engine trials. Specifically, machine learning methods, like XG Boost, random forest regressor, decision tree regressor, and linear regression, were utilized for prediction purposes. Among these techniques, the XG Boost model demonstrated highly accurate predictions, followed by the random forest regressor, decision tree regressor, and linear regression models. The accuracy of the predictions for XG Boost model was assessed through evaluation metrics such as R2-Score (0.999), Root Mean Squared Error (0.540), Mean Squared Error (0.248), and Mean Absolute Error (0.292), which allowed for a thorough analysis of the algorithm’s performance compared to actual values.

Data Driven Framework with Graphical User Interface for Predicting Flexural Behavior of FRCM Strengthened RC Beams

    The increasing intensity and severity of droughts, coupled with the challenge of generating reliable forecasts, significantly contribute to food insecurity in developing countries. Enhanced agricultural drought forecasts offering early, impact-based warnings can assist communities in preparing for the detrimental effects of drought, thus diminishing the number of individuals impacted. Drought Early Warning Systems (DEWS) convert physical drought observations into actionable insights, guiding decisions about water-intensive agricultural practices and alternative irrigation water sources. These systems require extended lead times, ideally spanning months or seasons, due to the time-sensitive nature of decisions aimed at mitigating drought vulnerability.      Prior research on impact-based forecasting primarily concentrated on precipitation indicators, frequently neglecting the integration of vegetation health, climatic, and hydrological factors, which are crucial in determining crop conditions. This study addresses the limitations of current DEWS by investigating the application of machine learning models, specifically Fast and Frugal Trees (FFTs) and Extreme Gradient Boosting (XGBoost), to identify agricultural impact triggers well before the harvest period. These algorithms forecast agricultural drought impacts by constructing decision trees from a comprehensive set of drought indicators. We illustrate the efficacy of machine learning models in predicting maize yield failures and anomalies in the Water Requirement Satisfaction Index (WRSI), a vital measure of crop performance during the growth season.      This paper presents the outcomes of a pilot DEWS project, InfoSequia-4CAST, executed by the World Food Programme (WFP) in Mozambique in 2022. The user-end model successfully provided monthly DEWS bulletins for Tete and Gaza provinces with satisfactory reliability. End users had the flexibility to modify various parameters, including input variable selection, lead times of interest (up to six months), optimization statistical metrics, the geographic scale for model development, and the threshold method or value for alert initiation. Furthermore, we discuss prospects for enhancing the reliability of these impact-based forecasts by contrasting the applied and transparent FFT model with the more complex, 'black box' XGBoost model. While simpler approaches like FFTs may be more appealing to decision-makers due to their straightforwardness, their potential for oversimplification might limit their effectiveness, stemming from their restricted use of variables. XGBoost exhibits superior performance over FFTs in identifying drivers of agricultural drought and providing more reliable, data-driven forecasts. Nevertheless, FFTs hold value in contexts where transparency is required, as this can strengthen trust in the forecasts, which is equally crucial for their adoption and the initiation of responsive actions. 

The antioxidant properties of Ainsliaea acerifolia, a wild edible plant, were examined by ultrasonic-assisted ethanol extraction methods. The primary objective was to optimize the extraction conditions and accurately predict antioxidant activities using advanced machine learning models. The extraction conditions were optimized using Response Surface Methodology (RSM). Various parameters, including temperature, extraction time, and ethanol concentration, were adjusted to maximize antioxidant activity. The optimal conditions identified were a temperature of 68 °C, an extraction time of 86 min, and an ethanol concentration of 57%. Under these conditions, the extracts exhibited the highest antioxidant activity. To enhance the predictive accuracy of antioxidant activity, an XGBoost (XGB) model was employed. The XGB model performance was evaluated and compared with the RSM model. The XGB model achieved an R² value of 94.71%, significantly outperforming the RSM model by 12.8%. This highlights the superiority of the XGB model in predicting antioxidant activities based on the given extraction parameters. Additionally, the study developed a graphical user interface (GUI). This GUI allows researchers and industry experts to input extraction conditions and obtain quick, accurate predictions of antioxidant activity.

Development and validation of interpretable machine learning model for pre-treatment predicting the response to targeted and immune therapy in hepatocellular carcinoma.