Using an Ensemble Learning Approach on Traditional Machine Learning Methods to Solve a Multi-Label Classification Problem

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Child mortality is a big problem around the world, especially in low- and middle-income nations where there are big differences in health care and social conditions. This investigation seeks to create a predictive model for child mortality and pinpoint the key factors that significantly contribute to it, employing machine learning (ML) methodologies. The dataset includes various features such as parental age, maternal education, birth weight, wealth index, and access to healthcare services. Thirteen machine learning classifiers were used, categorized into four model groups: Traditional Models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Naive Bayes), Tree-Based Models (Decision Tree, Random Forest, Extra Trees), Boosting Models (AdaBoost, Gradient Boosting, XGBoost), and Ensemble Learning Models (Soft Voting, Hard Voting, Stacking). The efficacy of each model was assessed using classification metrics, including Accuracy, Precision, Recall, and F1-Score within a 10-fold cross-validation framework to guarantee robustness. Results indicate that ensemble models, particularly AdaBoost, achieved the highest predictive accuracy, with perfect scores across all metrics (1.00). XGBoost and Stacking also demonstrated strong and consistent performance. The findings indicate that ensemble learning methods are effective in predicting child mortality and can assist policymakers and healthcare planners in identifying high-risk populations and implementing targeted interventions to reduce child mortality.

Worm gearboxes are popular across various industrial applications since they offer significant gear ratios in small installation spaces. Despite having multiple advantages, worm gearboxes are subjected to higher friction due to sliding design and are prone to damage and increased transmission error over the period of operation. Delayed diagnosis of worm gearbox degradation can lead to low-quality products and/or unnecessary production line downtime. Using vibration characteristics of worm gearbox, it is possible to determine the fault and transmission error at a given period in time. In this paper, an ensemble machine learning model is trained and deployed to monitor the transmission error of worm gearbox and classify between new, operational and old conditions. 1D CNN (one dimensional convolutional neural network) model is used to automatically extract features in vibration signal of X, Y, and Z axes and predict the relevant state of worm gear. The proposed technique uses ensemble machine learning technique fusion of features extracted by multi-layer 1D CNN for three axes vibration data. The proposed method could achieve 96% accuracy and performs significantly better than traditional sequential and ensemble machine learning models on a dataset of with 7,870 samples with 800 samples labeled as new condition, 4,740 samples as operational and 2,330 as old.

Accurate classifications of land use/land cover (LULC) in arid regions are vital for analyzing changes in climate. We propose an ensemble learning approach for improving LULC classification accuracy in Xinjiang, northwest China. First, multisource geographical datasets were applied, and the study area was divided into Northern Xinjiang, Tianshan, and Southern Xinjiang. Second, five machine learning algorithms—k-nearest neighbor, support vector machine (SVM), random forest (RF), artificial neural network (ANN), and C4.5—were chosen to develop different ensemble learning strategies according to the climatic and topographic characteristics of each subregion. Third, stratified random sampling was used to obtain training samples and optimal parameters for each machine learning algorithm. Lastly, each derived approach was applied across Xinjiang, and subregion performance was evaluated. The results showed that the LULC classification accuracy achieved across Xinjiang via the proposed ensemble learning approach was improved by ≥6.85% compared with individual machine learning algorithms. By specific subregion, the accuracies for Northern Xinjiang, Tianshan, and Southern Xinjiang increased by ≥6.70%, 5.87%, and 6.86%, respectively. Moreover, the ensemble learning strategy combining four machine learning algorithms (i.e., SVM, RF, ANN, and C4.5) was superior across Xinjiang and Tianshan; whereas, the three-algorithm (i.e., SVM, RF, and ANN) strategy worked best for the Northern and Southern Xinjiang. The innovation of this study is to develop a novel ensemble learning approach to divide Xinjiang into different subregions, accurately classify land cover, and generate a new land cover product for simulating climate change in Xinjiang.

Lately, multi-label classification (MLC) problems have drawn a lot of attention in a wide range of fields including medical, web, and entertainment. The scale and the diversity of MLC problems is much larger than single-label classification problems. Especially we have to face all possible combinations of labels. To solve MLC problems more efficiently, we focus on three kinds of locality hidden in a given MLC problem. In this paper, first we show how large degree of locality exists in nine datasets, then examine how closely they are related to labels, and last propose a method of reducing the problem size using one kind of locality.

Hepatocellular Carcinoma (HCC) is the most common type of liver cancer which accounts for around 75% of all liver cancer cases. From statistical data, it has been found that fatality due to liver cancer is higher regardless of improved screening and discoveries in medicines, HCC escalate fatality rate. This paper presents an ensemble learning model for HCC survival prediction. The input predictors for the proposed model consist of geographical information, risk factors and clinical trial information of HCC patients. Fifteen different models are presented to evaluate the prediction. These models present data pre-processing, feature reduction/elimination and survival classification phase. For feature evaluation, LASSO Regression (L-1 penalization), Ridge Regression (L-2 penalization), Genetic Algorithm (GA) Optimization and Random Forest (RF) are proposed for weight valuation of features wherein features with significant weights are selected for prediction. With the aid of feature evaluators, L-1 penalized Nu-Support Vector Classification (Nu-SVC) model, L-2 penalized Nu-SVC model, GA optimized Nu-SVC model, RF-NuSVC model, L-1 penalized RidgeCV (RCV) model, L-2 penalized RCV model, GA optimized RCV model, RF-RCV model, L-1 penalized Gradient Boosting Ensemble Learning (GBEL) model, L-2 penalized GBEL model, GA optimized GBEL model and RFGBEL model are presented for survival prediction. The prediction performances of models were measured in terms of accuracy, recall/sensitivity, F-1 score, Log-Loss score, Jaccard score and Area Under Receiver Operating Curves (AUROC). The results indicate that RFGBEL model shows excellent performance in contrast to other proposed models. The proposed RFGBEL model achieves an accuracy of 93.92%, sensitivity of 94.73%, F-1 score of 0.93, Log-Loss/Cross entropy score of 5.89 and Jaccard score of 0.72. RFGBEL estimates value of area under the curve as 0.932. Comparison of RFGBEL model with other existing state of the art models are presented for performance assessment. Overall, the RFGBEL model has a capability to predict the result with more accuracy and sensitivity by means of machine learning and data mining approach.

Landslides are among the most hazardous geomorphological processes, especially in mountainous terrains where they threaten lives, disrupt infrastructure, and degrade ecosystems. The Chamoli district in Uttarakhand, India, with its rugged topography, erratic rainfall, and anthropogenic disturbances, is highly susceptible to such events. This research focuses on generating a detailed landslide susceptibility map for Chamoli using four machine learning algorithms Naïve Bayes (NB), K-Nearest Neighbors (KNN), Random Forest (RF), and Extreme Gradient Boosting (XGBoost). A total of sixteen causative factors were selected and processed using Geographic Information System (GIS) techniques. To avoid multicollinearity and ensure the reliability of input variables, statistical validation was performed. The landslide inventory, consisting of 778 past events, was split into 70% training and 30% testing datasets for model development and evaluation. The models were assessed using statistical indicators such as sensitivity, specificity, precision, accuracy, F1-score, Matthews Correlation Coefficient (MCC), and the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC). Among the models, XGBoost showed the highest performance (AUC = 0.95), followed by RF (0.83), KNN (0.79), and NB (0.78). The susceptibility analysis revealed that XGBoost categorized 17.53% of the area as highly vulnerable, outperforming RF (14.08%), KNN (14.00%), and NB (4.55%). The enhanced accuracy of XGBoost and RF stems from their ensemble learning approach, which effectively captures nonlinear relationships and mitigates overfitting. The resulting susceptibility maps are crucial tools for risk management, infrastructure planning, and sustainable development in hazard-prone Himalayan regions. This study reinforces the importance of machine learning in natural hazard assessment and suggests that future efforts should incorporate real-time data, finer-resolution remote sensing, and deep learning methods to further refine prediction accuracy.

The aaccurate prediction of enzymes with environment detoxification functions is crucial, not only to achieve a better understanding of bioremediation strategies, but also to alleviate environmental pollution. In the present study, a novel machine learning model was introduced which classifies enzymes by their toxin degradation ability. In this model, two different sets of data were used which include enzymes that can catalyze the toxin degradation as a positive dataset and non-toxin-degrading enzymes as a negative dataset. Further, a comparison of multiple classifiers was performed to find the best model and a Random Forest (RF) classifier was selected due to its strong performance. To enhance the accuracy, we combined RF with a Deep Neural Network (DNN), forming an ensemble model which effectively integrated both techniques. This combination achieved 95% precision, surpassing individual models. Our ensemble model not only ensures high prediction accuracy but also reliably differentiates toxin-degrading enzymes from non-degrading ones. This study highlights the power of combining classical machine learning with deep learning to advance prediction. Our model represents a significant step in enzyme classification and serves as a valuable resource for environmental biotechnology, food nutrition, and health applications.

The increasing number of space debris and space-active targets makes the space environment more and more complex. Space target recognition, a crucial component of space situational awareness, is of paramount importance to space security. Firstly, this paper elucidates the fundamental principles of ensemble learning, analyzes its characteristics and fusion method, and provides a comprehensive comparison of three common ensemble learning methods. Secondly, this paper analyzes the basic attributes and characteristics of space targets and categorizes the hierarchy of space target recognition. Again, the paper reviews the advances in the application of ensemble learning in space target recognition, focusing on three aspects: space target recognition datasets, the ensemble of traditional machine learning models, and ensemble deep learning. Subsequently, classical machine learning and ensemble learning algorithms are tested on a self-built space target simulation dataset, and we find that Stacking performs well on this dataset. Finally, the paper discusses future research directions.

Exploring interpretable ensemble learning to predict mechanical strength and thermal conductivity of aerogel-incorporated concrete

This study addresses the accuracy challenges in e-commerce sentiment classification and thus provides valuable insight for businesses to enrich strategies toward the interpretation of customer feedback and improvement of product development. This article elaborately contrasts long short-term memory (LSTM)-based models with traditional machine learning models, like support vector machines (SVM), random forest, and Naive Bayes classifiers. The authors have used a large dataset of customer reviews from famous e-commerce websites, pre-processed for noise reduction and standardization of the input. Our LSTM models were implemented using state-of-the-art deep learning frameworks, with special consideration while performing hyperparameter tuning. The results showed that the bidirectional LSTM model had the best performance of 92.1% in accuracy, 91.8% in precision, 92.0% in recall, and an F1-score of 91.9%. In comparison, traditional machine learning approaches gave accuracies: 86.5% for SVM, 85.1% for random forest, and 82% for Naive Bayes. Hyperparameter tuning returned the best configurations for the LSTM models: 128 LSTM units, a dropout of 0.3, with learning rates 0.001 for LSTM and 0.0001 for bidirectional LSTM. These optimizations contributed hugely to the performance improvements observed in these models. Error analysis gave insights into the challenges that the models faced. Sarcasm and irony accounted for 22% of the classification errors, while mixed sentiment accounted for 18%, and implicit accounted for 15%. To sum up, this research has shown the efficiency of LSTM models on e-commerce user review sentiment analysis, especially bidirectional LSTM. These models show a much better result in sentiment classification compared to traditional machine learning techniques, which proves them to have huge potential for improving the accuracy of the task in real-world applications.

The new global pandemic caused by the 2019 novel coronavirus (COVID-19), novel coronavirus pneumonia, has spread rapidly around the world, causing enormous damage to daily life, public health security, and the global economy. Early detection and treatment of COVID-19 infected patients are critical to prevent the further spread of the epidemic. However, existing detection methods are unable to rapidly detect COVID-19 patients, so infected individuals are not detected in a timely manner, which complicates the prevention and control of COVID-19 to some extent. Therefore, it is crucial to develop a rapid and practical COVID-19 detection method. In this work, we explored the application of deep learning in COVID-19 detection to develop a rapid COVID-19 detection method. Existing studies have shown that novel coronavirus pneumonia has significant radiographic performance. In this study, we analyze and select the features of chest radiographs. We propose a chest X-Ray (CXR) classification method based on the selected features and investigate the application of transfer learning in detecting pneumonia and COVID-19. Furthermore, we combine the proposed CXR classification method based on selected features with transfer learning and ensemble learning and propose an ensemble deep learning model based on transfer learning called COVID-ensemble to diagnose pneumonia and COVID-19 using chest x-ray images. The model aims to provide an accurate diagnosis for binary classification (no finding/pneumonia) and multivariate classification (COVID-19/No findings/ Pneumonia). Our proposed CXR classification method based on selection features can significantly improve the CXR classification accuracy of the CNN model. Using this method, DarkNet19 improved its binary and triple classification accuracies by 3.5% and 5.78%, respectively. In addition, the COVIDensemble achieved 91.5% accuracy in the binary classification task and 91.11% in the multi-category classification task. The experimental results demonstrate that the COVID-ensemble can quickly and accurately detect COVID-19 and pneumonia automatically through X-ray images and that the performance of this model is superior to that of several existing methods. Our proposed COVID-ensemble can not only overcome the limitations of the conventional COVID-19 detection method RT-PCR and provide convenient and fast COVID-19 detection but also automatically detect pneumonia, thereby reducing the pressure on the medical staff. Using deep learning models to automatically diagnose COVID-19 and pneumonia from X-ray images can serve as a fast and efficient screening method for COVID-19 and pneumonia.

With the accelerating advancement of biomedical research, it has been widely accepted that genetic variation plays a critical role in the pathogenesis of human inherited diseases. As an important type of genetic variation, nonsynonymous single nucleotide polymorphisms (nsSNPs) that occur in protein coding regions lead to amino acid substitutions in proteins, affecting structures and functions of proteins, and potentially causing human diseases. Hence, identifying disease-associated nsSNPs against neutral ones by machine learning approaches plays an important role in the understanding of genetic bases of human diseases and further promoting the prevention, diagnosis, and treatment of these diseases. In this paper, we formulate the task of identifying disease-associated nsSNPs as a binary classification problem. Based on a set of 26 numeric features derived from protein sequence information, we compare the performance of five popular ensemble learning approaches (AdaBoost, LogitBoost, Random forests, L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> boosting and stochastic gradient regression) with two traditional classification methods (decision trees and support vector machines) in this classification problem. Systematic validation demonstrates that ensemble learning approaches are in general more effective in identifying the disease-associated nsSNPs, while LogitBoost can achieve the highest performance among all the methods compared.

Due to the vast and rapid increase in data, data mining has become an increasingly important tool for the purpose of knowledge discovery in order to prevent the presence of rich data but poor knowledge. Data mining tasks can be undertaken in two ways, namely, manual walkthrough of data and use of machine learning approaches. Due to the presence of big data, machine learning has thus become a powerful tool to do data mining in intelligent ways. A popular approach of machine learning is inductive learning, which can be used to generate a rule set (a set of rules) using a particular algorithm. Inductive learning can involve a single base algorithm learning from a single data set following a standard learning approach. In this approach, the learning algorithm can generate a single rule set such as decision trees. On the other hand, the inductive learning can also involve a single base algorithm learning from multiple data sets following an ensemble learning approach. In this approach, the learning algorithm can generate multiple rule sets such as random forests. The latter approach is usually designed to reduce overfitting of models that usually arises when the former approach is adopted. In this context, the ensemble learning approach usually enables the improvement of the overall accuracy in prediction. The aim of this paper is to introduce a new approach of ensemble learning called Collaborative Rule Generation. In the new approach, the inductive learning involves multiple base algorithms learning from a single data set to generate a single rule set, which aims to enable each rule to have a higher quality. This paper also includes an experimental study validating the Collaborative Rule Generation approach and discusses the results in both quantitative and qualitative ways.

Lung cancer is one of the three deadliest diseases in the world and has rapidly developed. Based on this, researchers conducted research to predict the factors that influence lung cancer. One method to identify this is using data mining methods and classification techniques. Researchers used several popular algorithms in classification to make comparisons of the most accurate algorithms for lung cancer classification. The algorithms used include K-Nearest Neighbor, Random Forest Classifier, Logistic Regression, Linear SVM, Naïve Bayes, Decision Tree, Random Forest, Gradient Boosting, Kernel SVM, and MLPClassifier. The researcher used this algorithm because, in the research that the researcher found on the Kaggle platform, he examined the comparison of the algorithm using the breast cancer dataset. In previous studies, their researchers used SVM, which obtained an accuracy of 96.47%, Neural Networks of 97.06%, and Naïve Bayes with an accuracy of 91.18% to study breast cancer. The difference from previous research is that this study uses several existing algorithms in Machine Learning such as K-Nearest Neighbor, Random Forest Classifier, Logistic Regression, Linear SVM, Naïve Bayes, Decision Tree, Random Forest, Gradient Boosting, Kernel SVM, and MLPClassifier. In addition, this research was conducted to see whether the results of the accuracy of the algorithm that the researchers carried out using the lung cancer dataset had different results. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood.