Comparing SMOTE Family Techniques in Predicting Insurance Premium Defaulting using Machine Learning Models

Compared with coronary artery bypass grafting (CABG) under cardiopulmonary bypass, off-pump coronary artery bypass (OPCAB) is minimally invasive and reduces the risk of intraoperative blood transfusion and acute kidney injury. Nonetheless, OPCAB-related complications still pose a threat. Machine learning technology can analyze a large number of clinical data, establish risk prediction models and help clinicians make early and correct clinical decisions. Risk prediction models are available for mortality and morbidity after cardiac surgery, but they are not specific to OPCAB. This study aimed to develop a predictive model of severe complications after OPCAB, based on machine learning. Anesthesia records of OPCAB from the General Hospital of the Northern Theater Command (Shenyang, China) collected between January 1, 2019, and June 15, 2020, were analyzed. The endpoint of the study was the occurrence of serious complications after OPCAB (postoperative unplanned intra-aortic balloon pump, secondary surgery and death). The features entered into the models were as follows: intraoperative ventricular fibrillation, number of saphenous vein grafts, nerve block (NeB), venous oxygen saturation (SvO2), skin incision-bypass time, and hypertension. A total of 8 machine learning algorithms were tested: logistic regression analysis (LRA), k-nearest neighbor (KNN), naïve Bayes (NB), support vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical features gradient boosting (CatBoost). Among the 506 patients found in the records, 27 met the endpoint. The highest area under the curve (AUC) value was achieved with the XGBoost model (AUC = 0.94), and the lowest with the SVM model (AUC = 0.75). The highest and lowest accuracy were observed with the XGBoost and NB models, respectively, while the highest and lowest precision were achieved using the SVM and NB models, respectively. Based on the receiver operating characteristic (ROC) curves, the XGBoost model was selected as the most useful in this study. This study suggests using the XGBoost model to predict the risk of complications after OPCAB.

An artificial intelligence-based decision support system for early and accurate diagnosis of Parkinson’s Disease

Imbalance data is common in real-world applications like text categorization, face recognition for gender classification, medical diagnosis, fraud detection, oil-spills detection of satellite images. Most of the algorithms in machine learning are focusing on classification of majority class while ignoring or misclassifying minority sample. The minority samples are those that rarely occur but very important. It is commonly agreed that standard classifiers such as neural networks, support vector machines, and C4.5 are heavily biased in recognizing mostly the majority class since they are built to achieve overall accuracy to which the minority class contributes very little. In this study, we demonstrate how the synthetic minority over-sampling technique (SMOTE) can significantly improve the imbalance problem in gender classification from the data-level perspective. Hu’s moment of the face images was generated as the numerical descriptors with different imbalance ratio and classified using a supervised decision tree (J48) algorithm. The results show that prior to preprocessing the data with SMOTE, the minority group was severely misclassified as the majority group. Our claims are confirmed through the application of SMOTE in reducing the imbalance effects before inducing the decision tree.

In the semiconductor manufacturing industry, Automatic Defect Classification (ADC) plays an important role in maintaining high wafer inspection quality and reducing yield loss. ADC performance has benefitted from using machine learning (ML) algorithms; however, performance is negatively affected by the data imbalance and limited amounts of training data. Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique to adjust the skewed class distribution of a dataset so that the bias of the majority class is reduced. This paper shows that applying SMOTE achieved higher accuracy and purity on two imbalanced datasets, consisting of scanning electron microscopy (SEM) images collected with ASML-HMI eP™ and eScan® series inspection tools. The ML models are also less sensitive to the selection of hyperparameters when SMOTE is applied. We also show that better classification results can be obtained with less training samples with SMOTE; we conducted an experiment where a ML model trained on only 25% of samples with SMOTE achieved a higher ADC accuracy and purity performance compared to the same ML model trained on all samples but without SMOTE. In another experiment using a highly imbalanced SEM dataset with very few counts of the defect-of- interest (DOI), the combination of SMOTE and random undersampling of the majority class improves the accuracy by up to 5x while maintaining the same level of purity.

A method for accurately classifying donkey breeds has been developed by integrating single nucleotide polymorphism (SNPs) data with machine learning algorithms. The approach includes preprocessing donkey genomic sequencing data, addressing data imbalance with the Synthetic Minority Over-sampling Technique (SMOTE), and utilizing an improved Leave-One-Out Cross-Validation (LOOCV) for dataset partitioning. Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Random Forest (RF) models were constructed and evaluated. The results demonstrated that different chromosomes significantly influence classifier performance. For instance, chromosome Chr2 showed the highest classification accuracy with KNN, while chromosome Chr19 performed best with SVM and RF models. After enhancing data quality and addressing imbalances, classification performance improved substantially, with accuracy, precision, recall, and F1 score showing increases of up to 15% in certain models, particularly on key chromosomes. This method offers an effective solution for donkey breed classification and provides technical support for the conservation and development of donkey genetic resources.

Background and aimsThe birth weight of a newborn is a crucial factor that affects their overall health and future well-being. Low birth weight (LBW) is a widespread global issue, which the World Health Organization defines as weighing less than 2,500 g. LBW can have severe negative consequences on an individual’s health, including neonatal mortality and various health concerns throughout their life. To address this problem, this study has been conducted using BDHS 2017–2018 data to uncover important aspects of LBW using a variety of machine learning (ML) approaches and to determine the best feature selection technique and best predictive ML model.MethodsTo pick out the key features, the Boruta algorithm and wrapper method were used. Logistic Regression (LR) used as traditional method and several machine learning classifiers were then used, including, DT (Decision Tree), SVM (Support Vector Machine), NB (Naïve Bayes), RF (Random Forest), XGBoost (eXtreme Gradient Boosting), and AdaBoost (Adaptive Boosting), to determine the best model for predicting LBW. The model’s performance was evaluated based on the specificity, sensitivity, accuracy, F1 score and AUC value.ResultsResult shows, Boruta algorithm identifies eleven significant features including respondent’s age, highest education level, educational attainment, wealth index, age at first birth, weight, height, BMI, age at first sexual intercourse, birth order number, and whether the child is a twin. Incorporating Boruta algorithm’s significant features, the performance of traditional LR and ML methods including DT, SVM, NB, RF, XGBoost, and AB were evaluated where LR, had a specificity, sensitivity, accuracy and F1 score of 0.85, 0.5, 85.15% and 0.915. While the ML methods DT, SVM, NB, RF, XGBoost, and AB model’s respective accuracy values were 85.35%, 85.15%, 84.54%, 81.18%, and 84.41%. Based on the specificity, sensitivity, accuracy, F1 score and AUC, RF (specificity = 0.99, sensitivity = 0.58, accuracy = 85.86%, F1 score = 0.9243, AUC = 0.549) outperformed the other methods. Both the classical (LR) and machine learning (ML) models’ performance has improved dramatically when important characteristics are extracted using the wrapper method. The LR method identified five significant features with a specificity, sensitivity, accuracy and F1 score of 0.87, 0.33, 87.12% and 0.9309. The region, whether the infant is a twin, and cesarean delivery were the three key features discovered by the DT and RF models, which were implemented using the wrapper technique. All three models had the identical F1 score of 0.9318. However, “child is twin” was recognized as a significant feature by the SVM, NB, and AB models, with an F1 score of 0.9315. Ultimately, with an F1 score of 0.9315, the XGBoost model recognized “child is twin” and “age at first sex” as relevant features. Random Forest again beat the other approaches in this instance.ConclusionsThe study reveals Wrapper method as the optimal feature selection technique. The ML method outperforms traditional methods, with Random Forest (RF) being the most effective predictive model for Low-Birth-Weight prediction. The study suggests that policymakers in Bangladesh can mitigate low birth weight newborns by considering identified risk factors.

Air pollution in Macau has become a serious problem following the Pearl River Delta’s (PRD) rapid industrialization that began in the 1990s. With this in mind, Macau needs an air quality forecast system that accurately predicts pollutant concentration during the occurrence of pollution episodes to warn the public ahead of time. Five different state-of-the-art machine learning (ML) algorithms were applied to create predictive models to forecast PM2.5, PM10, and CO concentrations for the next 24 and 48 h, which included artificial neural networks (ANN), random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and multiple linear regression (MLR), to determine the best ML algorithms for the respective pollutants and time scale. The diurnal measurements of air quality data in Macau from 2016 to 2021 were obtained for this work. The 2020 and 2021 datasets were used for model testing, while the four-year data before 2020 and 2021 were used to build and train the ML models. Results show that the ANN, RF, XGBoost, SVM, and MLR models were able to provide good performance in building up a 24-h forecast with a higher coefficient of determination (R2) and lower root mean square error (RMSE), mean absolute error (MAE), and biases (BIAS). Meanwhile, all the ML models in the 48-h forecasting performance were satisfactory enough to be accepted as a two-day continuous forecast even if the R2 value was lower than the 24-h forecast. The 48-h forecasting model could be further improved by proper feature selection based on the 24-h dataset, using the Shapley Additive Explanations (SHAP) value test and the adjusted R2 value of the 48-h forecasting model. In conclusion, the above five ML algorithms were able to successfully forecast the 24 and 48 h of pollutant concentration in Macau, with the RF and SVM models performing the best in the prediction of PM2.5 and PM10, and CO in both 24 and 48-h forecasts.

Machine learning based survival prediction of DLBCL: Using multimodal data.

Zero-day attacks pose a persistent threat to computing infrastructure by exploiting previously unknown software vulnerabilities that evade traditional signature-based network intrusion detection systems (NIDSs). To address this limitation, machine learning (ML) techniques offer a promising approach for enhancing anomaly detection in network traffic. This study evaluates several ML models on a labeled network traffic dataset, with a focus on zero-day attack detection. Ensemble learning methods, particularly eXtreme gradient boosting (XGBoost), achieved perfect classification, identifying all 6231 zero-day instances without false positives and maintaining efficient training and prediction times. While classical support vector machines (SVMs) performed modestly at 64% accuracy, their performance improved to 98% with the use of the borderline synthetic minority oversampling technique (SMOTE) and SMOTE + edited nearest neighbours (SMOTEENN). To explore quantum-enhanced alternatives, a quantum SVM (QSVM) is implemented using three-qubit and four-qubit quantum circuits simulated on the aer_simulator_statevector. The QSVM achieved high accuracy (99.89%) and strong F1-scores (98.95%), indicating that nonlinear quantum feature maps (QFMs) can increase sensitivity to zero-day exploit patterns. Unlike prior work that applies standard quantum kernels, this study introduces a parameterised quantum feature encoding scheme, where each classical feature is mapped using a nonlinear function tuned by a set of learnable parameters. Additionally, a sparse entanglement topology is derived from mutual information between features, ensuring a compact and data-adaptive quantum circuit that aligns with the resource constraints of noisy intermediate-scale quantum (NISQ) devices. Our contribution lies in formalising a quantum circuit design that enables scalable, expressive, and generalisable quantum architectures tailored for zero-day attack detection. This extends beyond conventional usage of QSVMs by offering a principled approach to quantum circuit construction for cybersecurity. While these findings are obtained via noiseless simulation, they provide a theoretical proof of concept for the viability of quantum ML (QML) in network security. Future work should target real quantum hardware execution and adaptive sampling techniques to assess robustness under decoherence, gate errors, and dynamic threat environments.

The rise of IoT technologies has increased the need for robust threat detection to address growing cyber-physical risks. Traditional machine learning (ML) models often struggle with imbalanced datasets, particularly in detecting rare threats. This study tackles this challenge using the "IoT_Modbus" dataset, a benchmark for multiclass cybersecurity classification, and explores advanced resampling techniques to enhance model performance. By combining methods like Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) with ensemble classifiers, Random Forest (RF), Gradient Boosting, XGBoost, LightGBM, and CatBoost, significant improvements in accuracy were achieved. RF outperformed others, reaching 99.06% accuracy with SMOTE and 99.00% with ADASYN. Among boosting-based models, XGBoost led with 86.21% (SMOTE) and 86.63% (ADASYN), while CatBoost followed closely. These results highlight the effectiveness of resampling in addressing class imbalance and improving predictive performance. Visual tools like heatmaps and accuracy charts further clarify these trends. This research provides cybersecurity professionals with a practical framework to enhance IoT threat detection and underscores the importance of resampling strategies for better classification outcomes. By guiding model selection for imbalanced datasets, it offers valuable insights for securing IoT systems in an interconnected world.

Machine learning (ML) algorithms can accurately extract quantitative patterns from datasets without requiring prior knowledge, playing an increasingly crucial role in tasks such as information extraction from remotely sensed data. This study employs several ML algorithms, including Back Propagation Network (BPN), Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting (XGBoost), to develop global oceanic chlorophyll-a (Chl-a) concentration inversion models for MODIS/Aqua (MODISA). The models utilize the SeaWiFS Bio-optical Archive and Storage System (SeaBASS) dataset, which contains in situ Chl-a concentration measurements paired with MODISA remote sensing reflectance (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R_{\mathrm {rs}}$ </tex-math></inline-formula>) for various water color bands from 2002 to 2017. This study includes the following processing steps. First, the SeaBASS dataset is randomly divided into training dataset and testing dataset. Second the Chl-a concentration inversion models are developed using the training dataset. Third, the testing dataset is utilized to evaluate the developed models. Finally, the inversion experiment is conducted using MODISA data. Experimental results indicate that all ML algorithms achieve higher inversion accuracy compared to the traditional ocean color index (OCI) algorithm. Particularly, the SVR model demonstrates the best performance. Additionally, to address the challenge of multidimensional parameter optimization in ML algorithms, this study introduces the Differential Evolution (DE) algorithm and develops a DE-based SVM model (DE-SVM). Compared to the standard SVM model, the DE-SVM model significantly reduces the mean deviation, mean absolute deviation, and root mean square error of the inverted chlorophyll-a concentration, while increasing the coefficient of determination (R2) from 0.87 to 0.926. These results indicate that the DE-SVM model further improves the inversion accuracy of Chl-a concentration, and holds substantial potential in satellite remote sensing inversion of Chl-a concentration.

to predict vestibular schwannoma (VS) response to radiosurgery by applying machine learning (ML) algorithms on radiomic features extracted from pre-treatment magnetic resonance (MR) images. patients with VS treated with radiosurgery in two Centers from 2004 to 2016 were retrospectively evaluated. Brain T1-weighted contrast-enhanced MR images were acquired before and at 24 and 36 months after treatment. Clinical and treatment data were collected contextually. Treatment responses were assessed considering the VS volume variation based on pre- and post-radiosurgery MR images at both time points. Tumors were semi-automatically segmented and radiomic features were extracted. Four ML algorithms (Random Forest, Support Vector Machine, Neural Network, and extreme Gradient Boosting) were trained and tested for treatment response (i.e., increased or non-increased tumor volume) using nested cross-validation. For training, feature selection was performed using the Least Absolute Shrinkage and Selection Operator, and the selected features were used as input to separately build the four ML classification algorithms. To overcome class imbalance during training, Synthetic Minority Oversampling Technique was used. Finally, trained models were tested on the corresponding held out set of patients to evaluate balanced accuracy, sensitivity, and specificity. 108 patients treated with Cyberknife® were retrieved; an increased tumor volume was observed at 24 months in 12 patients, and at 36 months in another group of 12 patients. The Neural Network was the best predictive algorithm for response at 24 (balanced accuracy 73% ± 18%, specificity 85% ± 12%, sensitivity 60% ± 42%) and 36 months (balanced accuracy 65% ± 12%, specificity 83% ± 9%, sensitivity 47% ± 27%). radiomics may predict VS response to radiosurgery avoiding long-term follow-up as well as unnecessary treatment.

This study investigates the use of machine learning (ML) models combined with a Synthetic Minority Over-sampling Technique (SMOTE) and its variants to predict perioperative pressure injuries (PIs) in an imbalanced dataset. PIs are a significant healthcare problem, often leading to prolonged hospitalization and increased medical costs. Conventional risk assessment scales are limited in their ability to predict PIs accurately, prompting the exploration of ML techniques to address this challenge.We utilized data from 7,292 patients admitted to a tertiary care hospital in Shanghai between May 2017 and July 2023, with a final dataset of 2,972 patients, including 158 with PIs. Seven ML algorithms-Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Extra Trees (ET), K-Nearest Neighbors (KNN), and Decision Trees (DT)-were used in conjunction with SMOTE, SMOTE+ENN, Borderline-SMOTE, ADASYN, and GAN to balance the dataset and improve model performance.Results revealed significant improvements in model performance when SMOTE and its variants were used. For instance, the XGBoost model hadan AUC of 0.996 with SMOTE, compared to 0.800 on raw data. SMOTE+ENN and Borderline-SMOTE further enhanced the models' ability to identify minority classes. External validation indicatedthat XGBoost, RF, and ET exhibited the highest stability and accuracy, with XGBoost having an AUC of 0.977. SHAP analysis revealed that factors such as anesthesia grade, age, and serum albumin levels significantly influenced model predictions.In conclusion, integrating SMOTE with ML algorithms effectively addressed a data imbalance and improved the prediction of perioperative PIs. Future work should focus on refining SMOTE techniques and exploring their application to larger, multi-center datasets to enhance the generalizability of these findings, and especially for diseaseswith a lowincidence.

Simple SummaryThe objective response rate of lenvatinib combined with transarterial chemoembolization for unresectable hepatocellular carcinoma is unsatisfactory. We aimed to develop predictive models using demographic characteristics, pre-treatment serum biomarkers and tumor characteristics of unresectable hepatocellular carcinoma patients by five machine learning algorithms to predict the response under combined treatments. We identified the 10 most important predictors, including K, low-density lipoprotein, D-dimer, red blood cell, alanine aminotransferase, albumin, monocyte, tumor size, triglyceride, and age. In addition, we applied the Shapley Additive exPlanation to explain the best-performing random forest predictive model to provide a reasonable explanation of the efficacy prediction at an individualized level. The combination of machine learning and Shapley Additive exPlanation can provide valuable suggestions for clinical decision making.Background: Lenvatinib and transarterial chemoembolization (TACE) are first-line treatments for unresectable hepatocellular carcinoma (HCC), but the objective response rate (ORR) is not satisfactory. We aimed to predict the response to lenvatinib combined with TACE before treatment for unresectable HCC using machine learning (ML) algorithms based on clinical data. Methods: Patients with unresectable HCC receiving the combination therapy of lenvatinib combined with TACE from two medical centers were retrospectively collected from January 2020 to December 2021. The response to the combination therapy was evaluated over the following 4–12 weeks. Five types of ML algorithms were applied to develop the predictive models, including classification and regression tree (CART), adaptive boosting (AdaBoost), extreme gradient boosting (XGBoost), random forest (RF), and support vector machine (SVM). The performance of the models was assessed by the receiver operating characteristic (ROC) curve and area under the receiver operating characteristic curve (AUC). The Shapley Additive exPlanation (SHAP) method was applied to explain the model. Results: A total of 125 unresectable HCC patients were included in the analysis after the inclusion and exclusion criteria, among which 42 (33.6%) patients showed progression disease (PD), 49 (39.2%) showed stable disease (SD), and 34 (27.2%) achieved partial response (PR). The nonresponse group (PD + SD) included 91 patients, while the response group (PR) included 34 patients. The top 40 most important features from all 64 clinical features were selected using the recursive feature elimination (RFE) algorithm to develop the predictive models. The predictive power was satisfactory, with AUCs of 0.74 to 0.91. The SVM model and RF model showed the highest accuracy (86.5%), and the RF model showed the largest AUC (0.91, 95% confidence interval (CI): 0.61–0.95). The SHAP summary plot and decision plot illustrated the impact of the top 40 features on the efficacy of the combination therapy, and the SHAP force plot successfully predicted the efficacy at the individualized level. Conclusions: A new predictive model based on clinical data was developed using ML algorithms, which showed favorable performance in predicting the response to lenvatinib combined with TACE for unresectable HCC. Combining ML with SHAP could provide an explicit explanation of the efficacy prediction.

Diabetes is a chronic disease that continues to be a primary and worldwide health concern since the health of the entire population has been affected by it. Over the years, many academics have attempted to develop a reliable diabetes prediction model using machine learning (ML) algorithms. However, these research investigations have had a minimal impact on clinical practice as the current studies focus mainly on improving the performance of complicated ML models while ignoring their explainability to clinical situations. Therefore, the physicians find it difficult to understand these models and rarely trust them for clinical use. In this study, a carefully constructed, efficient, and interpretable diabetes detection method using an explainable AI has been proposed. The Pima Indian diabetes dataset was used, containing a total of 768 instances where 268 are diabetic, and 500 cases are non-diabetic with several diabetic attributes. Here, six machine learning algorithms (artificial neural network (ANN), random forest (RF), support vector machine (SVM), logistic regression (LR), AdaBoost, XGBoost) have been used along with an ensemble classifier to diagnose the diabetes disease. For each machine learning model, global and local explanations have been produced using the Shapley additive explanations (SHAP), which are represented in different types of graphs to help physicians in understanding the model predictions. The balanced accuracy of the developed weighted ensemble model was 90% with a F1 score of 89% using a five-fold cross-validation (CV). The median values were used for the imputation of the missing values and the synthetic minority oversampling technique (SMOTETomek) was used to balance the classes of the dataset. The proposed approach can improve the clinical understanding of a diabetes diagnosis and help in taking necessary action at the very early stages of the disease.