Learning from Imbalanced Educational Data Using Ensemble Machine Learning Algorithms

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

Groundwater is a primary source of drinking water for billions worldwide. It plays a crucial role in irrigation, domestic, and industrial uses, and significantly contributes to drought resilience in various regions. However, excessive groundwater discharge has left many areas vulnerable to potable water shortages. Therefore, assessing groundwater potential zones (GWPZ) is essential for implementing sustainable management practices to ensure the availability of groundwater for present and future generations. This study aims to delineate areas with high groundwater potential in the Bankura district of West Bengal using four machine learning methods: Random Forest (RF), Adaptive Boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), and Voting Ensemble (VE). The models used 161 data points, comprising 70% of the training dataset, to identify significant correlations between the presence and absence of groundwater in the region. Among the methods, Random Forest (RF) and Extreme Gradient Boosting (XGBoost) proved to be the most effective in mapping groundwater potential, suggesting their applicability in other regions with similar hydrogeological conditions. The performance metrics for RF are very good with a precision of 0.919, recall of 0.971, F1-score of 0.944, and accuracy of 0.943. This indicates a strong capability to accurately predict groundwater zones with minimal false positives and negatives. Adaptive Boosting (AdaBoost) demonstrated comparable performance across all metrics (precision: 0.919, recall: 0.971, F1-score: 0.944, accuracy: 0.943), highlighting its effectiveness in predicting groundwater potential areas accurately; whereas, Extreme Gradient Boosting (XGBoost) outperformed the other models slightly, with higher values in all metrics: precision (0.944), recall (0.971), F1-score (0.958), and accuracy (0.957), suggesting a more refined model performance. The Voting Ensemble (VE) approach also showed enhanced performance, mirroring XGBoost's metrics (precision: 0.944, recall: 0.971, F1-score: 0.958, accuracy: 0.957). This indicates that combining the strengths of individual models leads to better predictions. The groundwater potentiality zoning across the Bankura district varied significantly, with areas of very low potentiality accounting for 41.81% and very high potentiality at 24.35%. The uncertainty in predictions ranged from 0.0 to 0.75 across the study area, reflecting the variability in groundwater availability and the need for targeted management strategies.In summary, this study highlights the critical need for assessing and managing groundwater resources effectively using advanced machine learning techniques. The findings provide a foundation for better groundwater management practices, ensuring sustainable use and conservation in Bankura district and beyond.

Although the goal of rectal cancer treatment is to restore gastrointestinal continuity, some patients with rectal cancer develop a permanent stoma (PS) after sphincter-saving operations. Although many studies have identified the risk factors and causes of PS, few have precisely predicted the probability of PS formation before surgery. To validate whether an artificial intelligence model can accurately predict PS formation in patients with rectal cancer after sphincter-saving operations. Patients with rectal cancer who underwent a sphincter-saving operation at Taipei Medical University Hospital between January 1, 2012, and December 31, 2021, were retrospectively included in this study. A machine learning technique was used to predict whether a PS would form after a sphincter-saving operation. We included 19 routinely available preoperative variables in the artificial intelligence analysis. To evaluate the efficiency of the model, 6 performance metrics were utilized: accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiving operating characteristic curve. In our classification pipeline, the data were randomly divided into a training set (80% of the data) and a validation set (20% of the data). The artificial intelligence models were trained using the training dataset, and their performance was evaluated using the validation dataset. Synthetic minority oversampling was used to solve the data imbalance. A total of 428 patients were included, and the PS rate was 13.6% (58/428) in the training set. The logistic regression (LR), Gaussian Naïve Bayes (GNB), Extreme Gradient Boosting (XGB), Gradient Boosting (GB), random forest, decision tree and light gradient boosting machine (LightGBM) algorithms were employed. The accuracies of the logistic regression (LR), Gaussian Naïve Bayes (GNB), Extreme Gradient Boosting (XGB), Gradient Boosting (GB), random forest (RF), decision tree (DT) and light gradient boosting machine (LightGBM) models were 70%, 76%, 89%, 93%, 95%, 79% and 93%, respectively. The area under the receiving operating characteristic curve values were 0.79 for the LR model, 0.84 for the GNB, 0.95 for the XGB, 0.95 for the GB, 0.99 for the RF model, 0.79 for the DT model and 0.98 for the LightGBM model. The key predictors that were identified were the distance of the lesion from the anal verge, clinical N stage, age, sex, American Society of Anesthesiologists score, and preoperative albumin and carcinoembryonic antigen levels. Integration of artificial intelligence with available preoperative data can potentially predict stoma outcomes after sphincter-saving operations. Our model exhibited excellent predictive ability and can improve the process of obtaining informed consent.

A study on road accident prediction and contributing factors using explainable machine learning models: analysis and performance

This paper is concerned with various machine Learning (ML) techniques to estimate the crop yield based on various factors, followed by an averaging technique. Multiple models are utilised by this ensemble machine learning technique called Bagging, taking weak learners as the base model to make the prediction. The pertinent ML algorithms that support crop yield prediction are Support Vector machines (SVM), Random trees (RT), Linear Regression (LR), K Nearest Neighbour (KNN), Random Forest (RF), Multiple Regression (MR), Support Vector Regression (SVR), Artificial Neural Networks (ANN) and Deep Neural Networks (DNN). Of these, in this paper, Classification and Regression Tree (CART), Ridge Regression (RR), eXtreme Gradient Boosting (XGBoost), and Artificial Neural Network (ANN) were implemented along with Bagging as the ensemble technique. The output of the entity models was thus averaged to give a final prediction.

P125. Development of a novel ensemble machine learning algorithm for prediction of complications and readmission after anterior cervical spinal fusion

Modeling of necking area reduction of carbon steel in hydrogen environment using machine learning approach

Bone marrow is the source of many blood-related diseases, such as blood cancers, and Bone Marrow Transplantation (BMT), also known as Hematopoietic Stem Cell Transplantation (HSCT), is a life-saving surgical procedure. However, this treatment is associated with a high risk of mortality. Predicting survival after BMT is therefore essential for effective and accurate treatment. BMT is considered a treatment-related mortality due to several primary causes of death such as infections, toxicity, and Graft-versus-Host Disease (GvHD) that occur after treatment. In addition, several risk factors affect the success of BMT and long-term survival after treatment. Therefore, there is a need for a prediction system based on machine learning techniques that can predict whether the patient will survive after BMT or not, which will definitely help the physicians to make the right decisions before performing the surgery for the patient. In this paper, using a publicly available BMT dataset from the University of California, Irvine ML repository (UCI ML repository), different machine learning models were investigated to predict the survival status of children undergoing BMT treatment. In particular, Random Forest (RF), Bagging Classifier, Extreme Gradient Boost (XGBoost), Adaptive Boosting (AdaBoost), Decision Tree (DT), Gradient Boost (GB), and K-Nearest Neighbors (KNN) were trained on the given dataset. The dataset consists of 45 variables after applying a series of preprocessing steps and removing the multicollinearity features based on the correlation heat map. Then, a feature engineering and modelling step was applied to identify the most significant features, followed by the use of machine learning models to simplify the overall classification process. It’s important to note that the most important features obtained by DT and those obtained by GB were the most suitable for training the Bagging classifier and the KNN model, respectively. In addition to that, hyper-parameters optimization using Grid Search Cross-Validation (GSCV) was applied to both approaches to improve the accuracy of the survival prediction. RF, AdaBoost, GB, and Bagging techniques have achieved the best accuracy of 97.37%

With the emergence of the massive educational data, education data mining techniques have extensively drawn considerable interest from scholars to explore the relationship between students’ achievements and other factors. In this study, the data set about the students’ achievements of Portuguese in two secondary education schools in Portugal is selected for education data mining, which involves the personal information, social and school related factors. To analyze the relationship between the students' achievements and other factors, this study proposed an ensemble model based on weighted voting for predicting the students’ achievements of Portuguese in the final period. First, the raw data is preprocessed using some basic methods, including dummy coding, correlation analysis, standardization, and normalization. Second, the isolation forest algorithm-based outlier adaption is applied to deal with the data set to enhance the robustness of the ensemble model. Finally, two base classifiers, i.e. gradient boosting decision tree and extreme gradient boosting, are integrated to form the ensemble model. The experiments are presented for verifying the superiority of the proposed model by comparing with five base classifiers, including gradient boosting decision tree, adaptive boosting, extreme gradient boosting, random forest, and decision tree. The experimental results demonstrate that the ensemble model performs better than other base classifiers in classification, and prove the validity of the outlier adaption based on isolation forest algorithm.

Fiber-reinforced polymer (FRP)–concrete bonding is widely adopted for structural strengthening, yet its durability is highly vulnerable to freeze–thaw (FT) degradation. This study combines experimental testing with interpretable machine learning (ML) to reveal the degradation mechanism and predict the interfacial behavior of FRP–concrete systems under FT exposure. Single-lap shear tests showed that all specimens failed through interfacial debonding accompanied by partial concrete peeling. The ultimate bond strength decreased by 6.0–18.5%, and the peak shear stress dropped by 53–80%, indicating a pronounced loss of ductility and adhesion. To extend the analysis, experimental data were integrated with literature datasets, and three ensemble ML algorithms—AdaBoost, Random Forest (RF), and Extreme Gradient Boosting (XGBoost)—were employed to predict key bond–slip parameters including ultimate bond strength, local maximum bond stress, slip values, and interfacial fracture energy. Among them, XGBoost achieved the highest predictive accuracy, with R2 values exceeding 0.94 for most output parameters and consistently low RMSE values. Shapley Additive exPlanations (SHAP) and Partial Dependence Plots (PDPs) further identified adhesive tensile strength, fiber modulus, FRP thickness, and concrete strength as dominant factors and defined their optimal ranges. The findings offer a scientific foundation for evaluating and predicting the long-term bond durability of FRP–concrete systems and support the development of reliable reinforcement strategies for infrastructure in cold and severe environments.

The potential of generalized deep learning models developed for crop water estimation was examined in the current study. This study was conducted in a semiarid region of India, i.e., Karnataka, with daily climatic data (maximum and minimum air temperatures, maximum and minimum relative humidity, wind speed, sunshine hours, and rainfall) of 44 years (1976–2020) for twelve locations. The Extreme Gradient Boosting (XGBoost), Gradient Boosting (GB), and Random Forest (RF) are three ensemble deep learning models that were developed using all of the climatic data from a single location (Bengaluru) from January 1976 to December 2017 and then immediately applied at eleven different locations (Ballari, Chikmaglur, Chitradurga, Devnagiri, Dharwad, Gadag, Haveri, Koppal, Mandya, Shivmoga, and Tumkuru) without the need for any local calibration. For the test period of January 2018–June 2020, the model’s capacity to estimate the numerical values of crop water requirement (Penman-Monteith (P-M) ETo values) was assessed. The developed ensemble deep learning models were evaluated using the performance criteria of mean absolute error (MAE), average absolute relative error (AARE), coefficient of correlation (r), noise to signal ratio (NS), Nash–Sutcliffe efficiency (ɳ), and weighted standard error of estimate (WSEE). The results indicated that the WSEE values of RF, GB, and XGBoost models for each location were smaller than 1 mm per day, and the model’s effectiveness varied from 96% to 99% across various locations. While all of the deep learning models performed better with respect to the P-M ETo approach, the XGBoost model was able to estimate ETo with greater accuracy than the GB and RF models. The XGBoost model’s strong performance was also indicated by the decreased noise-to-signal ratio. Thus, in this study, a generalized mathematical model for short-term ETo estimates is developed using ensemble deep learning techniques. Because of this type of model’s accuracy in calculating crop water requirements and its ability for generalization, it can be effortlessly integrated with a real-time water management system or an autonomous weather station at the regional level.

Utilizing an intelligent machine learning strategy for predicting the tensile strength and ductility properties of nitrogen doped CoCrFeMnNi based high entropy alloys

A literature survey of corporate failure prediction models

Postpartum depression (PPD) presents a significant mental health challenge for mothers following childbirth. While the precise cause of this condition remains unknown, preventive measures and treatments are available. This study aims to employ ensemble learning techniques, utilizing C4.5 decision tree (DT), gradient boosting tree (GBT), and extreme gradient boosting (XGBoost), to predict the occurrences of PPD in the Banjarmasin, South Kalimantan, Indonesia. The predictive model developed encompasses a dataset comprising 317 records gathered from postpartum mothers in hospitals, community health services, and midwifery clinics (referred to as Model 1). Furthermore, resampling techniques (Model 2) were employed to address class imbalance. Additionally, feature selection including forward selection and backward elimination (Model 3) were implemented to enhance model performance. The findings reveal that XGBoost, combined with resampling methods, achieved the highest accuracy rate at 87.57%. Feature selection identified five crucial factors associated with PPD incidence: marital status, number of living children, history of depression, fear of delivery, and family relationships. The utilization of ensemble learning strategies for PPD prediction yields reliable outcomes that can be applied within clinical settings. Exploring alternative ensemble learning strategies such as random forest and adaptive boosting could further optimize model performance and warrant consideration in future research endeavours.

Pulse wave velocity (PWV) can evaluate potential atherosclerosis (AS) and ultrafast pulse wave velocity (ufPWV) is a new technique to accurately assess PWV. However, few studies have examined the predictive value of ufPWV for AS risk. We aimed to establish a classification model for AS risk diagnosis based on ufPWV, so that AS can be diagnosed and prevented in advance. We collected imaging data, as well as clinical and laboratory data. A total of 613 patients with 20 attributes were admitted in this study. There were 392 patients with hyperlipidemia (AS risk group) and 221 healthy adults as the control group. In order to build AS risk prediction models, we considered decision tree, five different ensemble learning (EL) models [random forest (RF), adaptive boosting (AdaBoost), gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost) and light gradient boosting machine (LGBM)] and two different feature selection methods [statistical analysis and RF]. Accuracy and the area under the ROC curve (AUC) were used as the main criterion for model evaluation. In the prediction of AS risk with statistical analysis as the feature selection method, the performances of XGBoost (accuracy: 0.851; AUC: 0.884) and RF (accuracy: 0.844; AUC: 0.889) were better than other models. Besides, in the prediction of AS risk with RF as the feature selection method, the performances of LGBM (accuracy: 0.870; AUC: 0.903) and XGBoost (accuracy: 0.857; AUC: 0.903) were better than other models. In conclusions, EL models with RF as the feature selection method might provide accurate results in predicting AS risk. Besides, ufPWV, especially PWV of left common carotid artery at the end of systole, was an important feature in the AS risk prediction models.