Optimizing Heart Disease Prediction with Random Forest and Ensemble Methods

Heart disease has long been a major threat to human health, causing about one-third of all deaths each year. Therefore, there is a need for accurate and effective prediction of cardiac disease using machine learning techniques. This paper uses data from 1,319 patients with heart disease and applies several machine-learning methods to study the relationship between a total of eight factors. The algorithms used in this paper include neural networks, decision trees, random forests, and so on. The best model was established by the random forest method, with an accuracy of 96.97%, precision of 97.03%, recall of 96.97%, and f1-Score of 0.9. It was found that troponin and CK-MB indicators had the highest influence weights, and the sum of the weights of each model accounted for more than 75% of the total weight, which demonstrated their significance in the prediction of heart disease, and the results could be utilized for the future prediction of heart disease. In addition, it also plays an essential role in the prediction of heart disease. To sum up, this is really important for the prediction of future heart disease.

Background: Heart diseases continue to be among the primary causes of morbidity and mortality worldwide, highlighting the need for novel diagnostic methods to accurately, early, and economically predict and diagnose heart diseases. Current physical explores upon expert knowledge, possibly missing intricate and non-linear associations between heart disease risk factors. This is where machine learning (ML)methods emerge as promising alternatives for prediction and diagnosis of heart diseases by emphasizing data availability. Methods: This paper attempts to find out how six popular machine learning algorithms namely, Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting (GB), K-Nearest Neighbors (KNN), and eXtreme Gradient Boosting (XGB) perform on a publicly available heart disease dataset collected from Kaggle. For better performance and to make them comparable to each other, basic preprocessing techniques have also been implemented. The performance of all these algorithms will be measured by their respective definitions of accuracy, precision, recall, and F1 score. Result: With an F1-score of 98.52%, Random Forest and XGBoost surpassed other models, yielding a superior accuracy in predictions, measuring 98.53%. The models recorded a perfect level of precision, 100%, coupled with high values of recall, exceeding 97%. Gradient Boosting recorded impressive results, receiving an accuracy score of 93.17%. Conclusion: The research findings revealed that the Random Forest and XGBoost models are very effective in the prediction and diagnosis of heart disease. These two models have the capacity to support a clinical decision. Novelty: That is to say, the novelty of this research is that it evaluates multiple machine learning models with comparison to each other. The role of machine learning in enhancing cardiovascular disease diagnosis as well as decision support systems in healthcare and educating and aware the people about heart care and health.

In the rapidly evolving landscape of retail analytics, the accurate prediction of sales figures holds paramount importance for informed decision-making and operational optimization. Leveraging diverse machine learning methodologies, this study aims to enhance the precision of Walmart sales forecasting, utilizing a comprehensive dataset sourced from Kaggle. Exploratory data analysis reveals intricate patterns and temporal dependencies within the data, prompting the adoption of advanced predictive modeling techniques. Through the implementation of linear regression, ensemble methods such as Random Forest, Gradient Boosting Machines (GBM), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM), this research endeavors to identify the most effective approach for predicting Walmart sales. Comparative analysis of model performance showcases the superiority of advanced machine learning algorithms over traditional linear models. The results indicate that XGBoost emerges as the optimal predictor for sales forecasting, boasting the lowest Mean Absolute Error (MAE) of 1226.471, Root Mean Squared Error (RMSE) of 1700.981, and an exceptionally high R-squared value of 0.9999900, indicating near-perfect predictive accuracy. This model's performance significantly surpasses that of simpler models such as linear regression, which yielded an MAE of 35632.510 and an RMSE of 80153.858. Insights from bias and fairness measurements underscore the effectiveness of advanced models in mitigating bias and delivering equitable predictions across temporal segments. Our analysis revealed varying levels of bias across different models. Linear Regression, Multiple Regression, and GLM exhibited moderate bias, suggesting some systematic errors in predictions. Decision Tree showed slightly higher bias, while Random Forest demonstrated a unique scenario of negative bias, implying systematic underestimation of predictions. However, models like GBM, XGBoost, and LGB displayed biases closer to zero, indicating more accurate predictions with minimal systematic errors. Notably, the XGBoost model demonstrated the lowest bias, with an MAE of -7.548432 (Table 4), reflecting its superior ability to minimize prediction errors across different conditions. Additionally, fairness analysis revealed that XGBoost maintained robust performance in both holiday and non-holiday periods, with an MAE of 84273.385 for holidays and 1757.721 for non-holidays. Insights from the fairness measurements revealed that Linear Regression, Multiple Regression, and GLM showed consistent predictive performance across both subgroups. Meanwhile, Decision Tree performed similarly for holiday predictions but exhibited better accuracy for non-holiday sales, whereas, Random Forest, XGBoost, GBM, and LGB models displayed lower MAE values for the non-holiday subgroup, indicating potential fairness issues in predicting holiday sales. The study also highlights the importance of model selection and the impact of advanced machine learning techniques on achieving high predictive accuracy and fairness. Ensemble methods like Random Forest and GBM also showed strong performance, with Random Forest achieving an MAE of 12238.782 and an RMSE of 19814.965, and GBM achieving an MAE of 10839.822 and an RMSE of 1700.981. This research emphasizes the significance of leveraging sophisticated analytics tools to navigate the complexities of retail operations and drive strategic decision-making. By utilizing advanced machine learning models, retailers can achieve more accurate sales forecasts, ultimately leading to better inventory management and enhanced operational efficiency. The study reaffirms the transformative potential of data-driven approaches in driving business growth and innovation in the retail sector.


 
 
 
 Abstract
 The classification task known as MNIST digit recognition involves identifying handwritten numbers into their corresponding values. Although there are numerous approaches proposed for this type of task, they typically face issues in achieving high accuracy. One method that can improve single models' performance is through ensemble learning. The goal of this study is to explore the use of various learning techniques, such as boosting and bagging, in combination with random forest models and decision trees, to improve the performance of MNIST digit recognition with regard to accuracy. We then perform evaluations on these methods using various metrics, such as recall, precision, accuracy, and F1. The findings of this study provide valuable insight into the various advantages of ensemble methods for the MNIST digit recognition task. It also highlights the need to explore these techniques in the context of machine learning. The objective of this study is to investigate the use of ensembles in improving the accuracy of MNIST digit recognition. We performed evaluations on two popular methods, namely boosting and bagging, with random forest and decision tree models. The evaluation parameters included F1 score, recall, accuracy, and precision. The results of the evaluations revealed that both boosting and bagging methods performed well in terms of their evaluation metrics. In most cases, the decision tree performed better than the random forest. However, the random forest method was able to achieve the highest accuracy, which is 99 percent. The findings of the evaluation revealed that ensembles can help improve single models' accuracy in MNIST digit recognition. On the other hand, the random forest method is a promising option for this task. The exact results of the evaluations will vary depending on the evaluation and implementation metrics. More research is needed to confirm their generalizability. The study emphasizes the value of exploring ensembles in machine learning systems, as well as the potential advantages of performing MNIST digit recognition using them.
 
 
 

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

In recent years, forest fires have become an important issue in Central Europe. To model the probability of the occurrence of forest fires in the Lower Silesian Voivodeship of Poland, historical fire data and several types of predictors were collected or generated, including topographic, vegetation, climatic, and anthropogenic features. The main objectives of this study were to determine the importance of the predictors of forest fire occurrence and to map the probability of forest fire occurrence. The H2O driverless artificial intelligence (DAI) cloud platform was used to model forest fire probability. The gradient boosted machine (GBM) and random forest (RF) methods were applied to assess the probability of forest fire occurrence. Evaluation the importance of the variables was performed using the H2O platform permutation method. The most important variables were the presence of coniferous forest and the distance to agricultural land according to the GBM and RF methods, respectively. Model validation was conducted using receiver operating characteristic (ROC) analysis. The areas under the curve (AUCs) of the ROC plots from the GBM and RF models were 83.3% and 81.3%, respectively. Based on the results obtained, the GBM model can be recommended for the mapping of forest fire occurrence in the study area.

Software defects and quality assurance are crucial aspects of software development that should be considered during the software development cycle. To ensure high-quality software, it is essential to have a robust quality assurance process in place. System reliability and quality are very key components that must be considered during software development, and this can only be achieved when software undergoes a thorough test process for errors, anomalies, defects, omissions, and bugs. Early software defect prediction and detection play an essential role in ensuring the reliability and quality of software systems, ensuring that software companies discover errors or defects early enough and allocate more resources to defect-prone modules. This study proposes the development of an enhanced classifier model for software defect prediction and detection. The aim is to harness the collective intelligence of selected base classifiers like Support Vector Machine, Logistic regression, Decision Trees, Random Forest, AdaBoost, Gradient Boosting, K-Nearest Neighbor, GaussianNB, and Multi-Layer Perception to improve accuracy, robustness, and generalization in identifying potential defects using a soft voting ensemble technique. The ensemble model leveraged the confidence probability of the soft voting technique and the generalization advantage of cross-validation leading to a more robust and dynamic model. The performance of the model with existing classifiers was evaluated using accuracy, F1 score, Precision, and area under the ROC curve (ROC- AUC) as the evaluation metrics. The results of the experiment revealed that the Proposed Classifier produced an overall Accuracy rate of 93%, and ROC AUC of 98%. The results demonstrate the effectiveness of our enhanced ensemble classifier in software defect detection and prediction. By harnessing the strengths of diverse base classifiers, our approach provides a robust and adaptive solution to the challenges of early detection and mitigating defects in software systems. This research contributes to the advancement of reliable software development practices and lays the foundation for future enhancements in ensemble-based defect detection methodologies.

Bank telemarketing campaigns often experience low subscription rates due to customer heterogeneity and severe class imbalance, which pose challenges for reliable predictive modeling. This study investigates a data-driven approach that integrates synthetic minority oversampling and cost-sensitive learning to improve the prediction of telemarketing outcomes. Experiments are conducted using the Portuguese Bank Marketing dataset, comprising 41,188 instances with a positive response rate of 11.3%. Eight machine learning models are evaluated under a unified preprocessing pipeline and five-fold stratified cross-validation, including Logistic Regression, Decision Tree, Random Forest, and Ensemble methods. The results show that Ensemble models, particularly CatBoost, XGBoost, and LightGBM, achieve improved performance compared with traditional baselines, with notable gains in minority-class recall and overall discrimination ability. The best-performing model attains an F1-score of 0.540, a recall of 0.812 for the positive class, and a ROC–AUC of 0.908. To enhance interpretability, SHAP-based analysis is applied to quantify feature contributions, identifying campaign duration, previous contact outcomes, and selected macroeconomic indicators as key predictors. These findings indicate that combining resampling strategies with cost-sensitive optimization provides a robust and transparent approach for learning from imbalanced telemarketing data, thereby supporting reproducible and data-driven financial decision-making by explicitly addressing difficulty in minority-class identification under imbalance and class imbalance under cross-entropy training in imbalanced banking data.

In an unstable oil market with volatile prices due to various natural and geopolitical factors, it is crucial for oil‐producing companies to enhance the value of their assets by improving the recovery factors of petroleum reservoirs. Primary recovery through natural depletion or artificial lift and secondary recovery using waterflooding and immiscible gas injection typically recover no more than 10%–40% of the available reserves. A significant portion of the hydrocarbons remain unproduced if enhanced oil recovery (EOR) methods are not implemented. EOR projects are extremely costly, complex, and usually have long lead times from the decision‐making and design phases to pilot and full‐field implementations. Therefore, oil and gas operator companies need reliable insights into the best possible EOR options from the early stages of any field development planning. Since screening potential EOR choices is the first step in deciding future production scenarios, a smart EOR screening tool can add significant value by streamlining the EOR decision‐making process. In this study, we developed an EOR screening tool based on two advanced machine learning classification algorithms, random forest and light gradient boosting machine (LightGBM). These tree‐based ensemble learning classifiers were trained on an extensive dataset of 1384 worldwide EOR implementations, encompassing various reservoir conditions and reservoir rock and fluid properties as the feature space, to predict the EOR type as the class label. Considering EOR screening as a classification problem, an essential aspect of model development would be addressing the data imbalance of EOR datasets. To tackle this issue, the adaptive synthetic (ADASYN) sampling method was used to reduce classification bias by oversampling the training sets to achieve uniform class distributions. We designed an iterative model development procedure in which the classifiers were trained and tested on various training and test subsets split by stratified random sampling. For each classifier, the classification results at each iteration were used to build the confusion matrix and calculate model evaluation metrics (accuracy, precision, recall, and F1–score), which were then averaged over all independent runs to provide a fair assessment of classification performance. Moreover, binary receiver operating characteristic (ROC) curves were used to evaluate the classifier predictions and improvements obtained by oversampling. The results showed that both random forest and LightGBM classifiers made accurate class predictions, with LightGBM achieving slightly better classification performance in each modelling scenario (with or without oversampling). In both cases, the oversampling of the training dataset resulted in significant improvement of the classifiers, as evidenced by higher values of the evaluation metrics, leading to considerably more accurate EOR type predictions; specifically, oversampling boosted the prediction accuracy of the random forest model from 78.3% to 89.5% and the LightGBM model from 77.5% to 90.2%. Additionally, feature importance rankings provided valuable insights into which input variables had the greatest impact on model development.

Background/Objectives: Smartphone addiction (SA) and gaming addiction (GA) have become risk factors for individuals of all ages in recent years. Especially during adolescence, it has become very difficult for parents to control this situation. Physical activity and the effective use of free time are the most important factors in eliminating such addictions. This study aimed to test a new machine learning method by combining routine regression analysis with the gradient-boosting machine (GBM) and random forest (RF) methods to analyze the relationship between SA and GA with leisure time management (LTM) and the enjoyment of physical activity (EPA) among adolescents. Methods: This study presents the results obtained using our developed GBM + RF hybrid model, which incorporates LTM and EPA scores as inputs for predicting SA and GA, following the preprocessing of data collected from 1107 high school students aged 15-19 years. The results were compared with those obtained using routine regression results and the lasso, ElasticNet, RF, GBM, AdaBoost, bagging, support vector regression (SVR), K-nearest neighbors (KNN), multi-layer perceptron (MLP), and light gradient-boosting machine (LightGBM) models. In the GBM + RF model, probability scores obtained from GBM were used as input to RF to produce final predictions. The performance of the models was evaluated using the R2, mean absolute error (MAE), and mean squared error (MSE) metrics. Results: Classical regression analyses revealed a significant negative relationship between SA scores and both LTM and EPA scores. Specifically, as LTM and EPA scores increased, SA scores decreased significantly. In contrast, GA scores showed a significant negative relationship only with LTM scores, whereas EPA was not a significant determinant of GA. In contrast to the relatively low explanatory power of classical regression models, ML algorithms have demonstrated significantly higher prediction accuracy. The best performance for SA prediction was achieved using the Hybrid GBM + RF model (MAE = 0.095, MSE = 0.010, R2 = 0.9299), whereas the SVR model showed the weakest performance (MAE = 0.310, MSE = 0.096, R2 = 0.8615). Similarly, the Hybrid GBM + RF model also showed the highest performance for GA prediction (MAE = 0.090, MSE = 0.014, R2 = 0.9699). Conclusions: These findings demonstrate that classical regression analyses have limited explanatory power in capturing complex relationships between variables, whereas ML algorithms, particularly our GBM + RF hybrid model, offer more robust and accurate modeling capabilities for multifactorial cognitive and performance-related predictions.

Although machine learning has been extensively used in various fields, it has only recently been applied to soil erosion pin modeling. To improve upon previous methods of quantifying soil erosion based on erosion pin measurements, this study explored the possible application of ensemble machine learning algorithms to the Shihmen Reservoir watershed in northern Taiwan. Three categories of ensemble methods were considered in this study: (a) Bagging, (b) boosting, and (c) stacking. The bagging method in this study refers to bagged multivariate adaptive regression splines (bagged MARS) and random forest (RF), and the boosting method includes Cubist and gradient boosting machine (GBM). Finally, the stacking method is an ensemble method that uses a meta-model to combine the predictions of base models. This study used RF and GBM as the meta-models, decision tree, linear regression, artificial neural network, and support vector machine as the base models. The dataset used in this study was sampled using stratified random sampling to achieve a 70/30 split for the training and test data, and the process was repeated three times. The performance of six ensemble methods in three categories was analyzed based on the average of three attempts. It was found that GBM performed the best among the ensemble models with the lowest root-mean-square error (RMSE = 1.72 mm/year), the highest Nash-Sutcliffe efficiency (NSE = 0.54), and the highest index of agreement (d = 0.81). This result was confirmed by the spatial comparison of the absolute differences (errors) between model predictions and observations using GBM and RF in the study area. In summary, the results show that as a group, the bagging method and the boosting method performed equally well, and the stacking method was third for the erosion pin dataset considered in this study.

How to identify suspected infection for sepsis surveillance purposes remains a well-recognised challenge. This study aimed to operationalise suspected infection for sepsis surveillance by developing an interpretable machine learning (ML) model for retrospective identification of patients with sepsis. This multicentre cohort and machine learning study was conducted in two Dutch tertiary care hospitals. Adult patients with a quick Sequential Organ Failure assessment (qSOFA) ≥2 were included. Exclusion criteria included admission to the intensive care unit, transfer to or from another hospital, or patient refusal to reuse data. Cohort one consisted of patients admitted to the Emergency Department (ED) of hospital A between 01/01/2019 and 12/31/2019, to investigate community-onset sepsis. An external validation cohort of ED patients was obtained from hospital B between 01/01/2021 and 06/03/2022. Cohort two included hospitalised patients from hospital A between 01/01/2021 and 06/01/2022, to investigate hospital-onset sepsis. Objective data were extracted from electronic health records. Seven ML methods, including gradient boosting, random forest, logistic regression, decision trees, support vector machines, K nearest neighbours and stochastic gradient descent, were trained to identify sepsis with manual chart review as reference standard. The F1 score (harmonic mean of precision and recall), sensitivity and specificity were used as evaluation metrics. The best performing ML method was compared with other commonly used suspected infection proxies, including the Sepsis-3 definition, an adapted Adult Sepsis Event (ASE) definition and International Classification of Diseases (ICD) codes. In the ED cohort, 655 patients were included (male: 355 (54.2%), female: 300 (45.8%)) and 240 (36.6%) had sepsis. For community-onset sepsis, gradient boosting performed best with an F1 score of 85.9%, a sensitivity of 91.1% (95%-CI 83.4-95.4%) and a specificity of 89.0% (95%-CI 83.4-92.8%). Most model features reflected either the inflammatory response (CRP, body temperature) or actions taken when an infection is suspected (antibiotic administration, microbial culture). In the external validation cohort, 185 patients were included (male: 94 (50.8%), female: 91 (49.2%)) and 54 (29.2%) had sepsis. External validation yielded an F1 score of 85.7%, a sensitivity of 87.5% (95%-CI 75.3-94.1%) and a specificity of 92.5% (95%-CI 85.9-96.2%). The gradient boosting model outperformed other commonly used proxies for suspected infection in terms of sensitivity, achieving 91.1% (95% CI: 83.4-95.4%), compared to Sepsis-3 with 78.9% (95% CI: 69.4-86.0%), the adapted ASE with 85.6% (95% CI: 76.8-91.4%), and ICD codes with 33.3% (95% CI: 24.5-43.6%). In the hospitalised cohort, 493 patients were included (male: 265 (53.8%), female: 228 (46.2%)) and 129 (26.2%) had sepsis. For hospital-onset sepsis, logistic regression had the highest F1 score (52.2%). Sensitivity was 58.1% (95%-CI 40.6-75.5%) and specificity was 82.9% (95%-CI 76.0-89.8%). ED patients meeting ≥2 qSOFA criteria can be accurately classified as having suspected infection or not by a gradient boosting algorithm, outperforming common suspected infection definitions for sepsis surveillance. Including the inflammatory response in the suspected infection surveillance definition may enhance the accuracy and objectivity of sepsis surveillance. Future research is needed to validate the algorithm using other organ dysfunction criteria and in international settings. None.

Gender classification is one of the major challenges in the domain of speech analysis now-a-days. Identification of gender from acoustic properties of voice such as mean, median, and frequency, is of high importance. Machine learning is used to solve this problem because it gives promising results for classification techniques. There are several algorithms that can be used to predict the gender using acoustic properties. In this work, classifiers are evaluated using 6 different machine learning algorithms. These algorithms include K-Nearest Neighbor (KNN), Decision Tree (DT), Random Forest (RF), and Support Vector Machine (SVM), Support Vector Machine Using Poly kernel and Gradient Boosting (GB). For the same voice sample dataset, the accuracy, recall, precision and F1 Score are computed for all the above-mentioned classifiers. The accuracy obtained from KNN is 96.71, DT is 93.33%, Random Forest is 97.72%, SVM is 98.48%, SVM using Poly Kernel is 97.72%, Gradient Boosting is 96.96%. The predicted values of the considered evaluation metrices on test data for all above classifiers evidenced the supremacy of SVM classifier as it is best fitted model for gender identification of acoustic. SVM attained the accuracy of 98.48% which is the highest among all.

Optimizing the bank marketing strategy is major area of concern for the economic growth of many financial companies. In this paper, a study on the performance of different machine learning models considering multiple metrics for banking systems has been performed. In particular, the results of the study predict that the attitudes of the customers vary according to multiple factors and marketing campaigns. The results suggest that LightGBM showed the best accuracy result of (90.66%), succeeded by Random Forest and SVM with (89.33%), Gradient Boosting, and another Adaboost with (89.38%), Logistic Regression with (90.15%), XGBoost(90.54%) and Decision Tree with (87.44%), and finally Naive Bayes with (85.08%). This data set includes some level of preprocessing to handle missing data, encode categorical variables, and perform feature engineering, which is likely to improve model performance. In terms of evaluation metrics, the accuracy and precision, recall and the F1 score were analyzed. The results indicate that ensemble models, particularly Random Forest and Gradient Boosting, yield the best overall performance.

The accelerated adoption of bitcoin and other cryptocurrencies has facilities swift, decentralized transfer of value while simultaneously opening venues for illicit behavior such as money laundering, fraud, and the financing of terrorism. Because block chain records are pseudonymous and generate high dimensional data, effective detection demands sophisticated machine-learning (ML) models combined with network and graph-analysis methods to achieve reliable classification. This study offers a comparative assessment of ML classifiers to identify illegal activity in cryptocurrency transactions based on the Elliptic Bitcoin Transaction dataset. A two-stage feature selection approach including variance threshold method and random forest based feature ranking method was utilized to extract most informative features. Six ML classifiers, namely Random Forest (RF), K-Nearest Neighbor (KNN), Logistic Regression (LR), Support Vector Classifier (SVC), Gradient Boosting (GB), and Multilayer Perceptron (MLP), were chosen and compared on a stratified train-test split based on accuracy, precision, recall, f1 score, confusion matrix, and ROCU-AUC score. The performance results illustrate how RF, GB, and the rest of the methods consistently produced the best 98% accuracy rate with almost flawless classification scores of (0.997, 0.994, and 0.982, respectively). However, the KNN achieved an AUC value of 0.963, and the margin-based SVC achieved an AUC value of 0.981. The results shows that using ensemble methods can greatly improve how intrusions are detection in cryptocurrency network, these models are able to spot subtle patterns in blockchain transactions, which has practical value for making Bitcoin networks more secure.