Arabic Authorship Attribution Using Synthetic Minority Over-sampling Technique and Principal Components Analysis for Imbalanced Documents

In the semiconductor manufacturing industry, Automatic Defect Classification (ADC) plays an important role in maintaining high wafer inspection quality and reducing yield loss. ADC performance has benefitted from using machine learning (ML) algorithms; however, performance is negatively affected by the data imbalance and limited amounts of training data. Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique to adjust the skewed class distribution of a dataset so that the bias of the majority class is reduced. This paper shows that applying SMOTE achieved higher accuracy and purity on two imbalanced datasets, consisting of scanning electron microscopy (SEM) images collected with ASML-HMI eP™ and eScan® series inspection tools. The ML models are also less sensitive to the selection of hyperparameters when SMOTE is applied. We also show that better classification results can be obtained with less training samples with SMOTE; we conducted an experiment where a ML model trained on only 25% of samples with SMOTE achieved a higher ADC accuracy and purity performance compared to the same ML model trained on all samples but without SMOTE. In another experiment using a highly imbalanced SEM dataset with very few counts of the defect-of- interest (DOI), the combination of SMOTE and random undersampling of the majority class improves the accuracy by up to 5x while maintaining the same level of purity.

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

<p>Problem in imbalanced data is very common in classification or identification. The problem is raised when the number of instances of one class far exceeds the other. In the previous research, our DNA barcode-based Identification System of Tuna and Mackerel was developed in imbalanced dataset. The number of samples of Tuna and Mackerel were much more than those of other fish samples. Therefore, the accuracy of the classification model was probably still in bias. This research aimed at employing Synthetic Minority Oversampling Technique (SMOTE) to yield balanced dataset. We used kmers frequencies from DNA barcode sequences as features and Support Vector Machine (SVM) as classification method. In this research we used trinucleotide (3-mers) and tetranucleotide (4-mers). The training dataset was taken from Barcode of Life Database (BOLD). For evaluating the model, we compared the accuracy of model using SMOTE and without SMOTE in order to classify DNA barcode sequences which is taken from Department of Aquatic Product Technology, Bogor Agricultural University. The results showed that the accuracy of the model in the species level using SMOTE was 7% and 13% higher than those of non-SMOTE for trinucleotide (3-mers) and tetranucleotide (4-mers), respectively. It is expected that the use of SMOTE, as one of data balancing technique, could increase the accuracy of DNA barcode based fish classification system, particularly in the species level which is difficult to be identified.</p>

The imbalanced class problem is intrinsic to solar flare forecasting, as are other issues we find in data-driven forecasting problems that are often hidden within an imbalanced dataset. One method of dealing with imbalanced data is to balance the data by using synthetic oversampling to create synthetic examples of the minority class. Though synthetic oversampling techniques have been applied to problems in medicine, finance, security, and other areas, we have not seen these approaches used in solar flare forecasting. We investigate two methods of synthetic oversampling, Rapidly Converging Gibbs Sampler (RACOG) and Synthetic Minority Oversampling Technique (SMOTE). We devise three naive synthetic oversampling techniques for compar-ison. We rely on data provided by the Space Weather ANalytics for Solar Flares (SWAN- SF) benchmark dataset. Our results indicate that synthetic oversampling can be effective for machine learning based solar flare forecasting.

Oversampling is an efficient technique in dealing with class-imbalance problem. It addresses the problem by reduplicating or generating the minority class samples to balance the distribution between the samples of the majority and the minority class. Synthetic minority oversampling technique (SMOTE) is one of the typical representatives. During the past decade, researchers have proposed many variants of SMOTE. However, the existing oversampling methods may generate wrong minority class samples in some scenarios. Furthermore, how to effectively mine the inherent complex characteristics of imbalanced data remains a challenge. To this end, this paper proposes a parameter-free data cleaning method to improve SMOTE based on constructive covering algorithm. The dataset generated by SMOTE is first partitioned into a group of covers, then the hard-to-learn samples can be detected based on the characteristics of sample space distribution. Finally, a pair-wise deletion strategy is proposed to remove the hard-to-learn samples. The experimental results on 25 imbalanced datasets show that our proposed method is superior to the comparison methods in terms of various metrics, such as F-measure, G-mean, and Recall. Our method not only can reduce the complexity of the dataset but also can improve the performance of the classification model.

Imbalanced data is a challenge for classification models. It reduces the overall performance of traditional learning algorithms. Besides, the minority class of imbalanced datasets is misclassified with a high ratio even though this is a crucial object of the classification process. In this paper, a new model called the Lasso-Logistic ensemble is proposed to deal with imbalanced data by utilizing two popular techniques, random over-sampling and random under-sampling. The model was applied to two real imbalanced credit data sets. The results show that the Lasso-Logistic ensemble model offers better performance than the single traditional methods, such as random over-sampling, random under-sampling, Synthetic Minority Oversampling Technique (SMOTE), and cost-sensitive learning.This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium provided the original work is properly cited.

An Experimental Assessment of Random Forest Classification Performance Improvisation with Sampling and Stage Wise Success Rate Calculation

According to the Indonesian Health Profile in 2017, diabetes is one of the causes of death for almost 70% in the world. The high mortality rate induces the need for making the effort to reduce the number of people with diabetes by conducting studies that lead to making a diagnosis so that can detect a person with diabetes accurately. This study tries to compare the performance of the K-Nearest Neighbors algorithm using Synthetic Minority Over-sampling Technique and the K-Nearest Neighbors algorithm without Synthetic Minority Over-sampling Technique in diagnosing diabetes on imbalanced datasets. The parameters tested are the k value of the K-Nearest Neighbors and Synthetic Minority Over-sampling Technique. The testing is carried out using the K-Fold Cross Validation strategy. The data used in this study were 3876 data from Pertamina Central Hospital. Based on the results of tests conducted, it shows that the value of accuracy produced in diagnosing diabetes by using Synthetic Minority Over-sampling Technique is better than the accuracy produced without using Synthetic Minority Over-sampling Technique with the highest accuracy increase of 8.25%. The highest average accuracy is obtained when the value of k = 3 in the K-Nearest Neighbors, k = 5 in the Synthetic Minority Over-sampling Technique, and fold = 10, which reaches 78.06%.

This study investigates the use of machine learning (ML) models combined with a Synthetic Minority Over-sampling Technique (SMOTE) and its variants to predict perioperative pressure injuries (PIs) in an imbalanced dataset. PIs are a significant healthcare problem, often leading to prolonged hospitalization and increased medical costs. Conventional risk assessment scales are limited in their ability to predict PIs accurately, prompting the exploration of ML techniques to address this challenge.We utilized data from 7,292 patients admitted to a tertiary care hospital in Shanghai between May 2017 and July 2023, with a final dataset of 2,972 patients, including 158 with PIs. Seven ML algorithms-Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Extra Trees (ET), K-Nearest Neighbors (KNN), and Decision Trees (DT)-were used in conjunction with SMOTE, SMOTE+ENN, Borderline-SMOTE, ADASYN, and GAN to balance the dataset and improve model performance.Results revealed significant improvements in model performance when SMOTE and its variants were used. For instance, the XGBoost model hadan AUC of 0.996 with SMOTE, compared to 0.800 on raw data. SMOTE+ENN and Borderline-SMOTE further enhanced the models' ability to identify minority classes. External validation indicatedthat XGBoost, RF, and ET exhibited the highest stability and accuracy, with XGBoost having an AUC of 0.977. SHAP analysis revealed that factors such as anesthesia grade, age, and serum albumin levels significantly influenced model predictions.In conclusion, integrating SMOTE with ML algorithms effectively addressed a data imbalance and improved the prediction of perioperative PIs. Future work should focus on refining SMOTE techniques and exploring their application to larger, multi-center datasets to enhance the generalizability of these findings, and especially for diseaseswith a lowincidence.

The accuracy of the data class is very important in classification with a machine learning approach. The more accurate the existing data sets and classes, the better the output generated by machine learning. In fact, classification can experience imbalance class data in which each class does not have the same portion of the data set it has. The existence of data imbalance will affect the classification accuracy. One of the easiest ways to correct imbalanced data classes is to balance it. This study aims to explore the problem of data class imbalance in the medium case dataset and to address the imbalance of data classes as well. The Synthetic Minority Over-Sampling Technique (SMOTE) method is used to overcome the problem of class imbalance in obesity status in Indonesia 2013 Basic Health Research (RISKESDAS). The results show that the number of obese class (13.9%) and non-obese class (84.6%). This means that there is an imbalance in the data class with moderate criteria. Moreover, SMOTE with over-sampling 600% can improve the level of minor classes (obesity). As consequence, the classes of obesity status balanced. Therefore, SMOTE technique was better compared to without SMOTE in exploring the obesity status of Indonesia RISKESDAS 2013.

Abstract. Heart failure is caused by a disruption in the heart’s muscle wall, which results in the heart’s inability to pump blood in sufficient quantities to meet the body’s demand for blood. The increasing prevalence and mortality rates of heart failure can be reduced through early disease detection using data mining processes. Data mining is believed to aid in discovering and interpreting specific patterns in decision-making based on processed information. Data mining has also been applied in various fields, one of which is the healthcare sector. One of the data mining techniques used to predict a decision is the classification technique. Purpose: This research aims to apply SMOTE and PSO to the Random Forest classification algorithm in predicting the survival of heart failure patients and to determine its accuracy results. Methods/Study design/approach: To predict the survival of heart failure patients, we utilize the Random Forest classification algorithm and incorporate data imbalance handling with SMOTE and feature selection techniques with PSO on the Heart Failure Clinical Records Dataset. The data mining process consists of three distinct phases. Result/Findings: The application of SMOTE and PSO on the Heart Failure Clinical Records Dataset in the Random Forest classification process resulted in an accuracy rate of 93.9%. In contrast, the Random Forest classification process without SMOTE and PSO resulted in an accuracy rate of only 88.33%. This indicates that the proposed method combination can optimize the performance of the classification algorithm, achieving a higher accuracy compared to previous research. Novelty/Originality/Value: Data imbalance and irrelevant features in the Heart Failure Clinical Records Dataset significantly impact the classification process. Therefore, this research utilizes SMOTE as a data balancing method and PSO as a feature selection technique in the Heart Failure Clinical Records Dataset before the classification process of the Random Forest algorithm.

Machine learning methods for imbalanced data set for prediction of faecal contamination in beach waters

The Synthetic Minority Over-Sampling Technique (SMOTE) is one of the most well-known methods to solve the unequal class distribution problem in imbalanced datasets. However, it has three shortcomings: <xref ref-type="disp-formula" rid="deqn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(1)</xref> it may cause the over-generalization problem due to oversampling of noisy samples, <xref ref-type="disp-formula" rid="deqn2" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(2)</xref> over-sampling of uninformative samples, and <xref ref-type="disp-formula" rid="deqn3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(3)</xref> increasing the overlaps between different classes around the class boundaries. Different approaches SMOTE based have been proposed to handle these problems, but most of them implement hyperparameters and tend to generate noise because the synthetic sample is generated, randomly, in the area delimited by current random minority data. In this research, an improved SMOTE-based method, namely Fuzzy-ADAptative-SMOTE-Based-Methods (FADA-SOMTE-Ms), which targets all three problems at the same time, is introduced. In this regard, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-SMOTE is chosen in such a way that the synthetic data is as far as possible from the two closest majority data. More precisely, this method processes into six steps: (a) clustering minority class into <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> groups (b) selecting a safe region (c) selecting random two minority data, (d) finding the M closest majority data to these minority data using original membership functions based on Fuzzy mean and flirting results, (e) finding the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-SMOTE producing a synthetic data as close as possible to the minority class and as far as possible from the M majority data by solving a very simple multi-objective mathematical optimization model, and (f) using SMOTE to generate synthetic samples using optimal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-SMOTE. FADA-SOMTE-Ms is evaluated using 5 classifiers, 21 unbalanced datasets, and it’s compared to 8 oversampling methods using three performance measures. FADA-SOMTE-Ms consistently outperforms other popular oversampling methods.

Synthetic Minority Oversampling Technique (SMOTE) is a preferable method used to solve the imbalanced data classification issues. However, its efficiency for resolving the issues of minority sample classification still need to be improved. In order to balance its value and shortcome, we designed a perfected algorithm called “C-K-SMOTE’', which is a mixture clustering algorithm of the Canopy and K-means. For the final purpose of obtaining an approximately balanced data, first we use Canopy to achieve the approximate clustering, then use the K-means to obtain the accurate clustering, and after that we apply the SMOTE to increase the number of minority samples. The referential imbalanced data sets used in the article are selected from KEEL (Knowledge Extraction on Evolutionary Learning). By adopting random forest disaggregated model to carry experiments, SMOTE's efficiency of balancing the imbalanced databases is verified.

A new sampling method for classifying imbalanced data based on support vector machine ensemble