Ensemble Learning via Extreme Learning Machines for Imbalanced Data

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

The exponential rise in digital transactions has made credit card fraud a critical threat to financial institutions and consumers worldwide. Traditional rule-based fraud detection systems are increasingly ineffective against sophisticated and evolving fraudulent patterns due to their inability to adapt to new tactics and handle large-scale transaction data. This paper proposes an end-to-end FraudX that classifies transactions as legitimate or fraudulent using ensemble machine learning and real-time predictive analytics. The system utilizes historical transaction data in CSV format, applies advanced preprocessing and feature engineering techniques, and employs XGBoost as the primary classifier, alongside comparative models such as Logistic Regression and Random Forest]. Experimental results on a publicly available credit card fraud dataset demonstrate that XGBoost achieves superior performance with an accuracy of 99.8%, precision of 92.5%, and recall of 88.3%. The system is implemented as a modular web application with a Flask backend and SQLite database, offering an interactive dashboard for transaction screening, detailed reporting, and administrator-led model retraining. This work provides a scalable, efficient, and reliable solution for real-time fraud detection in financial ecosystems. Keywords: Credit card fraud detection, ensemble learning, XGBoost, machine learning, Flask web application, financial security, predictive analytics, imbalanced data.

Since cyberbullying impacts both individual victims and entire society, research on abusive language and its detection has attracted attention in recent years. Because social media sites like Facebook, Instagram, Twitter, and others are so widely accessible, hate speech, bullying, sexism, racism, aggressive material, harassment, poisonous comments, and other types of abuse have all substantially increased. Due to the critical requirement to detect, regulate, and limit the spread of harmful content on social networking sites, we conducted this study to automate the detection of offensive language or cyberbullying. We created a new Arabic balanced data set to be used in the offensive detection process because having a balanced data set for a model would result in improved accuracy models. Recently, the performance of single classifiers has been improved using ensemble machine learning. The purpose of this study is to examine the effectiveness of several single and ensemble machine learning algorithms in identifying Arabic text that contains foul language and cyberbullying. Applying them to three Arabic datasets, we have selected three machine learning classifiers and three ensemble models for this aim. Two of them are offensive datasets that are readily accessible in the public, while the third one was created. The results showed that the single learner machine learning strategy is inferior to the ensemble machine learning methodology. Voting performs is the best performing trained ensemble machine learning classifier, outperforming the best single learner classifier (65.1%, 76.2%, and 98%) for the same datasets with accuracy scores of (71.1%, 76.7%, and 98.5%) for each of the three datasets used. Finally, we improve the voting technique’s performance through hyperparameter tuning on the Arabic cyberbullying data set.

ObjectivesTo develop and identify machine learning (ML) models using pretreatment 2-deoxy-2-[18F]fluoro-D-glucose ([18F]-FDG)-positron emission tomography (PET)-based radiomic features to differentiate benign from malignant parotid gland diseases (PGDs).Materials and methodsThis retrospective study included 62 patients with 63 PGDs who underwent pretreatment [18F]-FDG-PET/computed tomography (CT). The lesions were assigned to the training (n = 44) and testing (n = 19) cohorts. In total, 49 [18F]-FDG-PET-based radiomic features were utilized to differentiate benign from malignant PGDs using five different conventional ML algorithmic models (random forest, neural network, k-nearest neighbors, logistic regression, and support vector machine) and the deep learning (DL)-based ensemble ML model. In the training cohort, each conventional ML model was constructed using the five most important features selected by the recursive feature elimination method with the tenfold cross-validation and synthetic minority oversampling technique. The DL-based ensemble ML model was constructed using the five most important features of the bagging and multilayer stacking methods. The area under the receiver operating characteristic curves (AUCs) and accuracies were used to compare predictive performances.ResultsIn total, 24 benign and 39 malignant PGDs were identified. Metabolic tumor volume and four GLSZM features (GLSZM_ZSE, GLSZM_SZE, GLSZM_GLNU, and GLSZM_ZSNU) were the five most important radiomic features. All five features except GLSZM_SZE were significantly higher in malignant PGDs than in benign ones (each p < 0.05). The DL-based ensemble ML model had the best performing classifier in the training and testing cohorts (AUC = 1.000, accuracy = 1.000 vs AUC = 0.976, accuracy = 0.947).ConclusionsThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can be useful for differentiating benign from malignant PGDs.Second abstractThe DL-based ensemble ML model using [18F]-FDG-PET-based radiomic features can overcome the previously reported limitation of [18F]-FDG-PET/CT scan for differentiating benign from malignant PGDs. The DL-based ensemble ML approach using [18F]-FDG-PET-based radiomic features can provide useful information for managing PGD.

Machine Learning is a branch of Artificial Intelligence that predicts several naturally occurring events by training a model with some data and then using unseen data to test it. This paper seeks to analyze the performances of single and ensemble machine learning algorithms on the Cleveland Heart disease data set. Experimental study proves that the accuracy score and area under the ROC curve in the ensemble machine learning model is higher than the single machine learning model in predicting non-CVD and CVD patients.

Radon diffusion through thin plastics

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

Ensemble machine learning prediction of posttraumatic stress disorder screening status after emergency room hospitalization

The evolution of Electronic Health Records (EHR) has facilitated comprehensive patient record-keeping, enhancing healthcare delivery and decision-making processes. Despite these advancements, analyzing EHR data using ensemble machine learning methods poses unique challenges. These challenges include data dimensionality, imbalanced class distributions, and the need for effective hyperparameter tuning to optimize model performance. The study conducted a thorough comparative analysis of various ensemble machine learning (EML) models using Electronic Health Record (EHR) datasets. After addressing data imbalance and reducing dimensionality, the accuracy of the EML models showed significant improvement. Notably, the Gradient Boosting Machine (GBM) and CatBoost models exhibited superior performance with an accuracy of 73%, achieved through experiments involving dimensionality reduction and handling of imbalanced data. Furthermore, optimization techniques such as Grid Search and Random Search were employed to enhance the EML models. The results of model optimization revealed that the GBM + Random Search model performed the best, achieving an accuracy of 74%, followed by the XGBoost + Grid Search model with an accuracy of 73%. The GBM model also excelled in distinguishing between positive and negative classes, boasting the highest Area under Curve (AUC) value of 0.78, indicative of its superior classification capabilities compared to other models. This study emphasizes the significance of incorporating cutting-edge EML techniques into clinical workflows and emphasizes the revolutionary potential of GBM in classification modeling for patient severity conditions. Future research should focus on deep learning (DL) applications and the integration of these models.

BackgroundMetabolic predictors and potential mediators of survival in sepsis have been incompletely characterized. We examined whether machine learning (ML) tools applied to the human plasma metabolome could consistently identify and prioritize metabolites implicated in sepsis survivorship, and whether these methods improved upon conventional statistical approaches.MethodsPlasma gas chromatography–liquid chromatography mass spectrometry quantified 411 metabolites measured ≤ 72 h of ICU admission in 60 patients with sepsis at a single center (Brigham and Women’s Hospital, Boston, USA). Seven ML approaches were trained to differentiate survivors from non-survivors. Model performance predicting 28 day mortality was assessed through internal cross-validation, and innate top-feature (metabolite) selection and rankings were compared across the 7 ML approaches and with conventional statistical methods (logistic regression). Metabolites were consensus ranked by a summary, ensemble ML ranking procedure weighing their contribution to mortality risk prediction across multiple ML models.ResultsMedian (IQR) patient age was 58 (47, 62) years, 45% were women, and median (IQR) SOFA score was 9 (6, 12). Mortality at 28 days was 42%. The models’ specificity ranged from 0.619 to 0.821. Partial least squares regression-discriminant analysis and nearest shrunken centroids prioritized the greatest number of metabolites identified by at least one other method. Penalized logistic regression demonstrated top-feature results that were consistent with many ML methods. Across the plasma metabolome, the 13 metabolites with the strongest linkage to mortality defined through an ensemble ML importance score included lactate, bilirubin, kynurenine, glycochenodeoxycholate, phenylalanine, and others. Four of these top 13 metabolites (3-hydroxyisobutyrate, indoleacetate, fucose, and glycolithocholate sulfate) have not been previously associated with sepsis survival. Many of the prioritized metabolites are constituents of the tryptophan, pyruvate, phenylalanine, pentose phosphate, and bile acid pathways.ConclusionsWe identified metabolites linked with sepsis survival, some confirming prior observations, and others representing new associations. The application of ensemble ML feature-ranking tools to metabolomic data may represent a promising statistical platform to support biologic target discovery.

Disastrous natural hazards, such as landslides, floods, and forest fires cause a serious threat to natural resources, assets and human lives. Consequently, landslide risk assessment has become requisite for managing the resources in future. This study was designed to develop four ensemble metaheuristic machine learning algorithms, such as grey wolf optimized based artificial neural network (GW-ANN), grey wolf optimized based random forest (GW-RF), particle swarm optimization optimized based ANN (PSO-ANN), and PSO optimized based RF for modeling rainfall-induced landslide susceptibility (LS) in Aqabat Al-Sulbat, Asir region, Saudi Arabia, which observes landslide frequently. To obtain very high precision and robust prediction from machine learning algorithms, the grey wolf and PSO optimization algorithms were integrated to develop new ensemble machine learning techniques. Subsequently, LS maps produced by training dataset were validated using the receiver operating characteristics (ROC) curve based on the testing dataset. Based on the area under curve (AUC) value of ROC curve, the best method for LS modeling was selected. We developed ROC curve-based sensitivity analysis to investigate the influence of the parameters for LS modeling. The Gumble extreme value distribution was employed to estimate the rainfall at 2, 5, 10, 20, 50, and 100 year return periods. Then, the landslide hazard maps were prepared at different return periods by integrating the best LS model and estimated rainfall at different return periods. The theory of danger pixels was employed to prepare a final risk assessment of the resources, which have been exposed to the landslide. The results showed that 27–42 and 6–15 km2 were predicted as the very high and high LS zones using four ensemble metaheuristic machine learning algorithms. Based on the area under curve (AUC) of ROC, GR-ANN (AUC-0.905) appeared as the best model for LS modeling. The areas under high and very high landslide hazard were gradually increased over the progression of time (26 km2 at the 2 year return period and 40 km2 at the 100 year return period for the high landslide hazard zone, and 6 km2 at the 2 year return period and 20 km2 at the 100 year return period for the very high landslide hazard zone). Similarly, the areas of danger pixel also increased gradually from the 2 to 100 year return periods (37 km2 to 62 km2). Various natural resources, such as scrubland, built up, and sparse vegetation, were identified under risk zone due to landslide hazards. In addition, these resources would be exposed extensively to landslides over the advancement of return periods. Therefore, the outcome of the present study will help planners and scientists to propose high precision management plans for protecting natural resources, which have been exposed to landslides.

We recently reported the importance of deep learning (DL) of pelvic magnetic resonance imaging in predicting the degree of urinary incontinence (UI) following robot-assisted radical prostatectomy (RARP). However, our results were limited because the prediction accuracy was approximately 70%. To develop a more precise prediction model that can inform patients about UI recovery post-RARP surgery using a DL model based on intraoperative video images. The study cohort comprised of 101 patients with localized prostate cancer undergoing RARP. Three snapshots from intraoperative video recordings showing the pelvic cavity (prior to bladder neck incision, immediately following prostate removal, and after vesicourethral anastomosis) were evaluated, including pre- and intraoperative parameters. We evaluated the DL model plus simple or ensemble machine learning (ML), and the area under the receiver operating characteristic curve (AUC) was analyzed through sensitivity and specificity. Of 101, 64 and 37 patients demonstrated "early continence (using 0 or 1 safety pad at 3 months post-RARP)" and "late continence (others)," respectively, at 3 months postoperatively. The combination of DL and simple ML using intraoperative video snapshots with clinicopathological parameters had a notably high performance (AUC, 0.683-0.749) to predict early recovery from UI after surgery. Furthermore, combining DL with ensemble artificial neural network using intraoperative video snapshots had the highest performance (AUC, 0.882; sensitivity, 92.2%; specificity, 78.4%; overall accuracy, 85.3%) to predict early recovery from post-RARP incontinence, with similar results by internal validation. The addition of clinicopathological parameters showed no additive effects for each analysis using DL, EL and simple ML. Our findings suggest that the DL algorithm with intraoperative video imaging is a reliable method for informing patients about the severity of their recovery from UI after RARP, although it is not clear if our methods are reproducible for predicting long-term UI and pad-free continence.

Ensemble machine learning prediction of compressive strength in waste-derived sulfoaluminate cement paste.

Worm gearboxes are popular across various industrial applications since they offer significant gear ratios in small installation spaces. Despite having multiple advantages, worm gearboxes are subjected to higher friction due to sliding design and are prone to damage and increased transmission error over the period of operation. Delayed diagnosis of worm gearbox degradation can lead to low-quality products and/or unnecessary production line downtime. Using vibration characteristics of worm gearbox, it is possible to determine the fault and transmission error at a given period in time. In this paper, an ensemble machine learning model is trained and deployed to monitor the transmission error of worm gearbox and classify between new, operational and old conditions. 1D CNN (one dimensional convolutional neural network) model is used to automatically extract features in vibration signal of X, Y, and Z axes and predict the relevant state of worm gear. The proposed technique uses ensemble machine learning technique fusion of features extracted by multi-layer 1D CNN for three axes vibration data. The proposed method could achieve 96% accuracy and performs significantly better than traditional sequential and ensemble machine learning models on a dataset of with 7,870 samples with 800 samples labeled as new condition, 4,740 samples as operational and 2,330 as old.

Student’s academic performance is one of the most important parameters for evaluating the standard of any institute. It has become a paramount importance for any institute to identify the student at risk of underperforming or failing or even drop out from the course. Machine Learning techniques may be used to develop a model for predicting student’s performance as early as at the time of admission. The task however is challenging as the educational data required to explore for modelling are usually imbalanced. We explore ensemble machine learning techniques namely bagging algorithm like random forest (rf) and boosting algorithms like adaptive boosting (adaboost), stochastic gradient boosting (gbm), extreme gradient boosting (xgbTree) in an attempt to develop a model for predicting the student’s performance of a private university at Meghalaya using three categories of data namely demographic, prior academic record, personality. The collected data are found to be highly imbalanced and also consists of missing values. We employ k-nearest neighbor (knn) data imputation technique to tackle the missing values. The models are developed on the imputed data with 10 fold cross validation technique and are evaluated using precision, specificity, recall, kappa metrics. As the data are imbalanced, we avoid using accuracy as the metrics of evaluating the model and instead use balanced accuracy and F-score. We compare the ensemble technique with single classifier C4.5. The best result is provided by random forest and adaboost with F-score of 66.67%, balanced accuracy of 75%, and accuracy of 96.94%.