Heart Disease Prediction Using Stacking Model With Balancing Techniques and Dimensionality Reduction

Class imbalance in vehicle payment method classification can cause predictive models to become biased toward the majority. This study aims to build a classification model for automotive consumer payment methods using Extreme Gradient Boosting (XGBoost), with class balancing handled through the Synthetic Minority Over-sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN), and model interpretability performed using SHAP (SHapley Additive Explanations). The dataset consisted of 11,011 records and 13 attributes derived from Toyota vehicle delivery order transactions. Results show that the XGBoost model without balancing achieved 67.37% accuracy but only 0.24 recall for the Cash class. After applying SMOTE, the recall for the Cash class improved to 0.58, while ADASYN produced a similar improvement at 0.59, with overall accuracy maintained at around 61–62% and a stable ROC-AUC of 0.65. Feature importance and SHAP analysis identified c_vehicle_model and c_city as the most influential factors in predicting the payment method. From a business perspective, the improved ability to detect cash customers reduces the risk of misclassification and enables dealers to better segment customer payment preference. This supports more effective marketing campaigns, sales strategies, and financing risk management. The combination of XGBoost, SMOTE, ADASYN, and SHAP has proven effective in handling imbalanced data while offering transparent interpretability of predictions, making it a practical foundation for data-driven decision-making in the automotive industry.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

Zero-day attacks pose a persistent threat to computing infrastructure by exploiting previously unknown software vulnerabilities that evade traditional signature-based network intrusion detection systems (NIDSs). To address this limitation, machine learning (ML) techniques offer a promising approach for enhancing anomaly detection in network traffic. This study evaluates several ML models on a labeled network traffic dataset, with a focus on zero-day attack detection. Ensemble learning methods, particularly eXtreme gradient boosting (XGBoost), achieved perfect classification, identifying all 6231 zero-day instances without false positives and maintaining efficient training and prediction times. While classical support vector machines (SVMs) performed modestly at 64% accuracy, their performance improved to 98% with the use of the borderline synthetic minority oversampling technique (SMOTE) and SMOTE + edited nearest neighbours (SMOTEENN). To explore quantum-enhanced alternatives, a quantum SVM (QSVM) is implemented using three-qubit and four-qubit quantum circuits simulated on the aer_simulator_statevector. The QSVM achieved high accuracy (99.89%) and strong F1-scores (98.95%), indicating that nonlinear quantum feature maps (QFMs) can increase sensitivity to zero-day exploit patterns. Unlike prior work that applies standard quantum kernels, this study introduces a parameterised quantum feature encoding scheme, where each classical feature is mapped using a nonlinear function tuned by a set of learnable parameters. Additionally, a sparse entanglement topology is derived from mutual information between features, ensuring a compact and data-adaptive quantum circuit that aligns with the resource constraints of noisy intermediate-scale quantum (NISQ) devices. Our contribution lies in formalising a quantum circuit design that enables scalable, expressive, and generalisable quantum architectures tailored for zero-day attack detection. This extends beyond conventional usage of QSVMs by offering a principled approach to quantum circuit construction for cybersecurity. While these findings are obtained via noiseless simulation, they provide a theoretical proof of concept for the viability of quantum ML (QML) in network security. Future work should target real quantum hardware execution and adaptive sampling techniques to assess robustness under decoherence, gate errors, and dynamic threat environments.

Stroke is one of the leading causes of death worldwide, creating an urgent need for effective early detection systems, particularly because conventional methods often struggle with class imbalance and produce biased evaluations. Previous studies have primarily focused on accuracy while overlooking model consistency, data pre-processing quality, and probability-based evaluation. This study evaluates model performance under three conditions: original data using extreme gradient boosting (XGBoost) with scale_pos_weight, original data using the easy ensemble classifier, and class-balanced data generated using random oversampling (ROS), adaptive synthetic sampling (ADASYN), and synthetic minority over-sampling technique (SMOTE). Each model underwent missing value handling, normalization, feature preparation, and hyperparameter optimization using grid search. Performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), confidence intervals, calibration curves, Shapley additive explanations (SHAP), decision curve analysis (DCA), and external validation. The results demonstrate that data resampling significantly improves performance, with the XGBoost-SMOTE combination achieving the best results, including an accuracy of 0.99, AUROC of 0.998, and AUPRC of 0.986, outperforming the other approaches. This method provides more consistent and balanced predictions, supporting the application of artificial intelligence for early stroke risk identification.

Cardiovascular diseases remain one of the leading causes of mortality worldwide, particularly in low- and middle-income countries. Early and accurate prediction of heart disease is essential for timely intervention and improved patient outcomes. Machine learning techniques offer promising solutions; however, challenges such as class imbalance, lack of interpretability, and limited real-world validation persist. In this study, a machine learning–based heart disease prediction framework was developed using a real-world clinical dataset comprising 5000 patient records collected from healthcare facilities in Bangladesh. Data preprocessing included cleaning, feature encoding, train–test splitting, and class imbalance handling using the Synthetic Minority Oversampling Technique (SMOTE). Multiple machine learning models—Logistic Regression, Decision Tree, Support Vector Machine, and Random Forest—were evaluated using 10-fold stratified cross-validation. Model performance was assessed using accuracy, precision, recall, and F1-score. SHAP (SHapley Additive exPlanations) was employed to enhance model interpretability. The best-performing model was deployed as a web-based decision support system. Among the evaluated models, the Random Forest classifier achieved the best performance, with an accuracy of 98%, recall of 96%, and F1-score of 96%. Ablation studies demonstrated the effectiveness of SMOTE, feature integration, and ensemble modeling. SHAP analysis identified clinically relevant features contributing to heart disease prediction, enhancing transparency and trust in model decisions. The proposed framework provides an accurate, interpretable, and practical solution for heart disease prediction using real-world clinical data. The integration of explainable machine learning and web-based deployment highlights its potential for clinical decision support. Future work will focus on multi-center prospective validation and adaptive model updating to further improve generalizability and real-world applicability. • A large real-world clinical dataset of 5000 patients was used for robust heart disease prediction. • A Random Forest model with SMOTE achieved high performance (98% accuracy, 96% recall). • Comprehensive ablation and model comparison analyses validate methodological robustness. • SHAP-based explainability provides transparent and clinically interpretable predictions. • The proposed system is deployed as a web-based clinical decision support tool.

Background: Stroke remains a critical global health concern, disproportionately affecting individuals with hypertension which is a well-established, modifiable risk factor. Traditional risk scoring systems often fall short in accurately predicting stroke onset due to their reliance on fixed clinical thresholds and limited variable interaction modeling. As the complexity of health data increases, machine learning (ML) and explainable artificial intelligence (XAI) present powerful tools to uncover hidden patterns and enable precision risk stratification. Objectives: This study proposes a novel, interpretable machine learning framework that uses ensemble learning and SHapley Additive exPlanations (SHAP) to enhance stroke risk prediction in hypertensive patients. The objective is twofold: to improve the predictive power of stroke models and to provide clinically relevant insights that support real-time, data-driven decisions in preventive care. Methods: We utilized a real-world clinical dataset encompassing demographic, physiological, and behavioral variables associated with stroke. Data preprocessing included k-nearest neighbor imputation for missing values, normalization of continuous features, and class balancing via Synthetic Minority Oversampling Technique (SMOTE). A hybrid feature selection pipeline, combining the sparsityenforcing capabilities of LASSO regression with the iterative refinement of Recursive Feature Elimination (RFE), was employed to identify the most salient predictors. Multiple ML models, including logistic regression, deep neural networks, random forests, and gradient boosting machines, were trained and validated using cross-validation. SHAP values were computed post-training to enable individualized, interpretable model outputs. Results: Ensemble models, particularly Gradient Boosting and Random Forest, demonstrated superior discriminative performance, achieving AUC-ROC scores above 0.78 following class balancing. The integrated LASSO-RFE approach revealed age, hypertension status, and average glucose levels as dominant predictors across models. SHAP visualizations confirmed the influence of these features, while also highlighting nuanced interactions involving lifestyle and socioeconomic variables. Logistic Regression, when optimized for recall, achieved the highest balanced accuracy (0.77), reinforcing the clinical utility of simpler models when interpretability is paramount. Conclusion: This study introduces a transparent and high-performing machine learning framework for stroke risk prediction in hypertensive individuals. By integrating ensemble learning, hybrid feature selection, and explainable AI, the framework bridges the gap between predictive modeling and clinical applicability. These findings support the deployment of interpretable ML tools in routine care, enabling proactive interventions, personalized patient education, and ultimately, reduced stroke incidence.

Binary datasets are considered imbalanced when one of their two classes has less than 40% of the total number of the data instances (i.e., minority class). Existing classification algorithms are biased when applied on imbalanced binary datasets, as they misclassify instances of minority class. Many techniques are proposed to minimize the bias and to increase the classification accuracy. Synthetic Minority Oversampling Technique (SMOTE) is a well-known approach proposed to address this problem. It generates new synthetic data instances to balance the dataset. Unfortunately, it generates these instances randomly, leading to the generation of useless new instances, which is time and memory consuming. Different SMOTE derivatives were proposed to overcome this problem (such as Borderline SMOTE), yet the number of generated instances slightly changed. To overcome such problem, this paper proposes a novel approach for generating synthesized data instances known as Hybrid Clustered Affinitive Borderline SMOTE (HCAB-SMOTE). It managed to minimize the number of generated instances while increasing the classification accuracy. It combines undersampling for removing majority noise instances and oversampling approaches to enhance the density of the borderline. It uses k-means clustering on the borderline area and identify which clusters to oversample to achieve better results. Experimental results show that HCAB-SMOTE outperformed SMOTE, Borderline SMOTE, AB-SMOTE and CAB-SMOTE approaches which were developed before reaching HCAB-SMOTE, as it provided the highest classification accuracy with the least number of generated instances.

Having a healthy baby is a dream for mothers. However, the high rate of maternal and fetal mortality is still a serious problem, so more accurate fetal health monitoring is needed to prevent pregnancy complications. One of the devices used is Cardiotocography (CTG), which produces data on fetal conditions. The CTG dataset used in this study faces challenges in the form of class imbalance and a high number of features, which can reduce classification performance. This study aims to overcome these challenges by implementing the Random Forest algorithm combined with the Synthetic Minority Oversampling Technique (SMOTE) technique for class balancing and Recursive Feature Elimination (RFE) for feature selection. The dataset used is "Fetal Health Classification" from the Kaggle platform, which consists of 2,126 data with three classes: Normal, Suspect, and Pathological. The test results show that the RFE method is able to reduce the number of features from 22 to 18, while SMOTE increases the proportion of minority data. The model built produces good classification performance with an accuracy value of 95%, precision 93%, recall 89%, and F1-score 91%. The ROC-AUC value for the Normal class is 0.9881, Suspect 0.9789, and Pathological 0.9985. Although the model is able to predict the Normal and Pathological classes with high accuracy, the performance on the Suspect class still needs to be improved. Overall, the integration of Random Forest with SMOTE and RFE has proven effective in improving the accuracy of fetal health classification.

One of the most complex and prevalent diseases is heart disease (HD). It is among the main causes of death around the globe. With changes in lifestyles and the environment, its prevalence is rising rapidly. The prediction of the disease in its early stages is crucial, as delays in diagnosis can cause serious complications and even death. Machine learning (ML) can be effective in this regard. Many researchers have used different techniques for the efficient detection of the disease and to overcome the drawbacks of existing models. Several ensemble models have also been applied. We proposed a stacking ensemble model named NCDG, which uses Naive Bayes, Categorical Boosting, and Decision Tree as base learners, with Gradient Boosting serving as the meta-learner classifier. We performed preprocessing using a factorization method to convert string columns into integers. We employ the Synthetic Minority Oversampling TEchnique (SMOTE) and BorderLineSMOTE balancing techniques to address the issue of data class imbalance. Additionally, we implemented hard and soft voting using voting classifier and compared the results with the proposed stacking model. For the Artificial Intelligence-based eXplainability of our proposed NCDG model, we use the SHapley Additive exPlanations (SHAP) technique. The outcomes show that our suggested stacking model, NCDG, performs better than the benchmark existing techniques. The experimental results of our proposed stacking model achieved the highest accuracy, F1-Score, precision and recall of 0.91, 0.91, 0.91 and 0.91, respectively, and an execution time of 653 s. Moreover, we have also utilized K-Fold Cross-Validation method to validate our predicted results. It is worth mentioning that our prediction results and their validation strongly coincide with each other which proves our approach to be symmetric.

Parkinson's disease (PD) is the fastest-growing neurodegenerative disorder, with subtle gait changes such as reduced vertical ground-reaction forces (VGRF) often preceding motor symptoms. These gait abnormalities, measurable via wearable VGRF sensors, offer a non-invasive means for early PD detection. However, current computational approaches often suffer from redundant features and class imbalance, limiting both accuracy and generalizability. We propose CRISP (Correlation-filtered Recursive Feature Elimination and Integration of SMOTE Pipeline for Gait-Based Parkinson's Disease Screening), a lightweight multistage framework that sequentially applies correlation-based feature pruning, recursive feature elimination (RFE), and Synthetic Minority Oversampling Technique (SMOTE) based class balancing. To ensure clinically meaningful evaluation, a novel subject-wise protocol was also introduced that assigns one prediction per individual enhancing patient-level variability capture and better aligning with diagnostic workflows. Using 306 VGRF recordings (93 PD, 76 controls), five classifiers, i.e., k-Nearest Neighbours (KNN), Decision Tree (DT), Random Forest (RF), Gradient boosting (GB), and Extreme Gradient Boosting (XGBoost) were evaluated for both binary PD detection and multiclass severity grading. CRISP consistently improved performance across all models under 5-fold cross-validation. XGBoost achieved the highest performance, increasing subject-wise PD detection accuracy from 96.1 ± 0.8% to 98.3 ± 0.8%, and severity grading accuracy from 96.2 ± 0.7% to 99.3 ± 0.5%. CRISP is the first VGRF-based pipeline to combine correlation-filtered feature pruning, recursive feature elimination, and SMOTE to enhance PD detection performance, while also introducing a subject-wise evaluation protocol that captures patient-level variability for truly personalized diagnostics. These twin novelties deliver clinically significant gains and lay the foundation for real-time, on-device PD detection and severity monitoring.

Endoscopic disease activity monitoring is important for the long-term management of patients with ulcerative colitis (UC), there is currently no widely accepted non-invasive method that can effectively predict endoscopic disease activity. We aimed to develop and validate machine learning (ML) models for predicting it, which are desired to reduce the frequency of endoscopic examinations and related costs. The patients with a diagnosis of UC in two hospitals from January 2016 to January 2021 were enrolled in this study. Thirty nine clinical and laboratory variables were collected. All patients were divided into four groups based on MES or UCEIS scores. Logistic regression (LR) and four ML algorithms were applied to construct the prediction models. The performance of models was evaluated in terms of accuracy, sensitivity, precision, F1 score, and area under the receiver-operating characteristic curve (AUC). Then Shapley additive explanations (SHAP) was applied to determine the importance of the selected variables and interpret the ML models. A total of 420 patients were entered into the study. Twenty four variables showed statistical differences among the groups. After synthetic minority oversampling technique (SMOTE) oversampling and RFE variables selection, the random forests (RF) model with 23 variables in MES and the extreme gradient boosting (XGBoost) model with 21 variables in USEIS, had the greatest discriminatory ability (AUC = 0.8192 in MES and 0.8006 in UCEIS in the test set). The results obtained from SHAP showed that albumin, rectal bleeding, and CRP/ALB contributed the most to the overall model. In addition, the above three variables had a more balanced contribution to each classification under the MES than the UCEIS according to the SHAP values. This proof-of-concept study demonstrated that the ML model could serve as an effective non-invasive approach to predicting endoscopic disease activity for patients with UC. RF and XGBoost, which were first introduced into data-based endoscopic disease activity prediction, are suitable for the present prediction modeling.

BackgroundThe growing demand for healthcare services challenges patient flow management in health systems. Alternative Level of Care (ALC) patients who no longer need acute care yet face discharge barriers contribute to prolonged stays and hospital overcrowding. Predicting these patients at admission allows for better resource planning, reducing bottlenecks, and improving flow. This study addresses three objectives: identifying likely ALC patients, key predictive features, and preparing guidelines for early ALC identification at admission.MethodsData from Nova Scotia Health (2015-2022) covering patient demographics, diagnoses, and clinical information was extracted. Data preparation involved managing outliers, feature engineering, handling missing values, transforming categorical variables, and standardizing. Data imbalance was addressed using class weights, random oversampling, and the Synthetic Minority Over-Sampling Technique (SMOTE). Three ML classifiers, Random Forest (RF), Artificial Neural Network (ANN), and eXtreme Gradient Boosting (XGB), were tested to classify patients as ALC or not. Also, to ensure accurate ALC prediction at admission, only features available at that time were used in a separate model iteration.ResultsModel performance was assessed using recall, F1-Score, and AUC metrics. The XGB model with SMOTE achieved the highest performance, with a recall of 0.95 and an AUC of 0.97, excelling in identifying ALC patients. The next best models were XGB with random oversampling and ANN with class weights. When limited to admission-only features, the XGB with SMOTE still performed well, achieving a recall of 0.91 and an AUC of 0.94, demonstrating its effectiveness in early ALC prediction. Additionally, the analysis identified diagnosis 1, patient age, and entry code as the top three predictors of ALC status.ConclusionsThe results demonstrate the potential of ML models to predict ALC status at admission. The findings support real-time decision-making to improve patient flow and reduce hospital overcrowding. The ALC guideline groups patients first by diagnosis, then by age, and finally by entry code, categorizing prediction outcomes into three probability ranges: below 30%, 30-70%, and above 70%. This framework assesses whether ALC status can be accurately predicted at admission or during the patient’s stay before discharge.

Most of the flight accident data have uneven distribution of categories. When the traditional classifier is applied to this data, it will pay less attention to the minority class data. Synthetic Minority Over-sampling Technique (SMOTE), and its improvements are well-known methods to address this imbalance problem at the data level. However, traditional algorithms still have the problems in blurring the boundary of positive and negative classes and changing the distribution of original data. In order to overcome these problems and accurately predict flight accidents, a new Clustered Biased Borderline SMOTE(CBB-SMOTE) is proposed for Quick Access Recorder (QAR) Go-Around data. It generates more obvious positive and negative class boundaries by using K-means for boundary minority class data and safety minority class data respectively, and maintains the original data distribution to the greatest extent through a biased oversampling method. Experiments were carried out on a group of QAR Go-Around data. The data set is balanced by CBB-SMOTE, SMOTE, Cluster-SMOTE algorithm respectively, and the random forest algorithm is used to predict the new data set. The experimental results show that CBB-SMOTE outperforms the SMOTE in terms of G-means value, Recall and AUC.

Unbalanced datasets are a common problem in supervised machine learning. It leads to a deeper understanding of the majority of classes in machine learning. Therefore, the machine learning model is more effective at recognizing the majority classes than the minority classes. Naturally, imbalanced data, such as disease data and data networking, has emerged in real life. DDOS is one of the network intrusions found to happen more often than R2L. There is an imbalance in the composition of network attacks in Intrusion Detection System (IDS) public datasets such as NSL-KDD and UNSW-NB15. Besides, researchers propose many techniques to transform it into balanced data by duplicating the minority class and producing synthetic data. Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic (ADASYN) algorithms duplicate the data and construct synthetic data for the minority classes. Meanwhile, machine learning algorithms can capture the labeled data's pattern by considering the input features. Unfortunately, not all the input features have an equal impact on the output (predicted class or value). Some features are interrelated and misleading. Therefore, the important features should be selected to produce a good model. In this research, we implement the recursive feature elimination (RFE) technique to select important features from the available dataset. According to the experiment, SMOTE provides a better synthetic dataset than ADASYN for the UNSW-B15 dataset with a high level of imbalance. RFE feature selection slightly reduces the model's accuracy but improves the training speed. Then, the Decision Tree classifier consistently achieves a better recognition rate than Random Forest and KNN.

Climate change is causing permafrost in the Qinghai–Tibet Plateau to degrade, triggering thermokarst hazards and impacting the environment. Despite their ecological importance, the distribution and risks of thermokarst lakes are not well understood due to complex influencing factors. In this study, we introduced a new interpretable ensemble learning method designed to improve the global and local interpretation of susceptibility assessments for thermokarst lakes. Our primary aim was to offer scientific support for precisely evaluating areas prone to thermokarst lake formation. In the thermokarst lake susceptibility assessment, we identified ten conditioning factors related to the formation and distribution of thermokarst lakes. In this highly accurate stacking model, the primary learning units were the random forest (RF), extremely randomized trees (EXTs), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost) algorithms. Meanwhile, gradient boosted decision trees (GBDTs) were employed as the secondary learning unit. Based on the stacking model, we assessed thermokarst lake susceptibility and validated accuracy through six evaluation indices. We examined the interpretability of the stacking model using three interpretation methods: accumulated local effects (ALE), local interpretable model-agnostic explanations (LIME), and Shapley additive explanations (SHAP). The results showed that the ensemble learning stacking model demonstrated superior performance and the highest prediction accuracy. Approximately 91.20% of the total thermokarst hazard points fell within the high and very high susceptible areas, encompassing 20.08% of the permafrost expanse in the QTP. The conclusive findings revealed that slope, elevation, the topographic wetness index (TWI), and precipitation were the primary factors influencing the assessment of thermokarst lake susceptibility. This comprehensive analysis extends to the broader impacts of thermokarst hazards, with the identified high and very high susceptibility zones affecting significant stretches of railway and highway infrastructure, substantial soil organic carbon reserves, and vast alpine grasslands. This interpretable ensemble learning model, which exhibits high accuracy, offers substantial practical significance for project route selection, construction, and operation in the QTP.