CTVAE: Contrastive Tabular Variational Autoencoder for imbalance data

Learning from imbalanced data sets, where the number of examples of one (majority) class is much higher than the others, presents an important challenge to the machine learning community. Traditional machine learning algorithms may be biased towards the majority class, thus producing poor predictive accuracy over the minority class. In this paper, we describe a new approach that combines boosting, an ensemble-based learning algorithm, with data generation to improve the predictive power of classifiers against imbalanced data sets consisting of two classes. In the DataBoost-IM method, hard examples from both the majority and minority classes are identified during execution of the boosting algorithm. Subsequently, the hard examples are used to separately generate synthetic examples for the majority and minority classes. The synthetic data are then added to the original training set, and the class distribution and the total weights of the different classes in the new training set are rebalanced. The DataBoost-IM method was evaluated, in terms of the F-measures, G-mean and overall accuracy , against seventeen highly and moderately imbalanced data sets using decision trees as base classifiers. Our results are promising and show that the DataBoost-IM method compares well in comparison with a base classifier, a standard benchmarking boosting algorithm and three advanced boosting-based algorithms for imbalanced data set. Results indicate that our approach does not sacrifice one class in favor of the other, but produces high predictions against both minority and majority classes.

In previous work, imbalanced datasets composed of more benign samples (the majority class) than the malicious one (the minority class) have been widely adopted in Android malware detection. These imbalanced datasets bias learning toward the majority class, so that the minority class examples are more likely to be misclassified. To solve the problem, we propose a new oversampling method called fuzzy–synthetic minority oversampling technique, which is based on fuzzy set theory and the synthetic minority oversampling technique method. As the sample size of the majority class increases relative to that of the minority class, fuzzy–synthetic minority oversampling technique generates more synthetic examples for each minority class examples in the fuzzy region, where the minority examples have a low degree of membership to the minority class and are more likely to be misclassified. Using the new synthetic examples, the classifiers build larger decision regions that contain more minority examples, and they are no longer biased to the majority class. Compared with synthetic minority oversampling technique and Borderline–synthetic minority oversampling technique methods, fuzzy–synthetic minority oversampling technique achieves higher accuracy on both the minority class and the entire datasets.

The class imbalance problem is a well-known classification challenge in machine learning that has vexed researchers for over a decade. Under-representation of one or more of the target classes (minority class(es)) as compared to others (majority class(es)) can restrict the application of conventional classifiers directly on the data. In addition, emerging challenges such as overlapping classes, make class imbalance even harder to solve. Class overlap is caused due to ambiguous regions in the data where the prior probability of two or more classes are approximately equal. We are motivated to address the challenge of class overlap in the presence of imbalanced classes by a problem in pervasive computing. Specifically, we are designing smart environments that perform health monitoring and assistance. Our solution, ClusBUS, is a clustering-based under sampling technique that identifies data regions where minority class samples are embedded deep inside majority class. By removing majority class samples from these regions, ClusBUS preprocesses the data in order to give more importance to the minority class during classification. Experiments show that ClusBUS achieves improved performance over an existing method for handling class imbalance.

<title>Abstract</title> Class imbalance remains a critical challenge in machine learning, particularly in high stakes domains such as fraud detection and medical diagnosis, where minority class instances carry greater significance. Traditional classifiers often exhibit biased performance toward the majority class, leading to suboptimal detection of rare but important events. To address this challenge, this paper proposes an Enhanced Genetic Algorithm based Data Balancer (EGADB) that optimally resamples imbalanced datasets prior to classification. The proposed technique leverages genetic optimization principles to generate a balanced training set while preserving the intrinsic data distribution. The effectiveness of EGADB is evaluated using multiple base learners including Logistic Regression, Naive Bayes, and Bagging, as well as a developed voter based ensemble model. Experiments are conducted on the Credit Card Fraud Detection dataset and a breast cancer dataset, comparing classifier performance on unbalanced data and data balanced using EGADB. Performance is assessed using accuracy, precision, recall, F1 score, optimized precision, Matthews correlation coefficient, and balanced accuracy score. Experimental results demonstrate that EGADB significantly enhances minority class recognition and overall classification robustness across all evaluated models. The developed EGADB voter ensemble consistently achieves superior performance, attaining near perfect detection metrics on balanced datasets while maintaining stability across evaluation measures. These findings confirm that EGADB provides an effective and scalable solution for mitigating class imbalance and improving ensemble based predictive performance in imbalanced learning environments.

Anomaly detection in medical imaging is pivotal for early diagnosis and treatment planning. However, the inherent class imbalance in medical datasets poses significant challenges, often leading to biased models that underperform on minority classes. This study investigates the integration of the Synthetic Minority Over-sampling Technique (SMOTE) with various machine learning and deep learning models to enhance anomaly detection in medical images. By applying SMOTE to balance datasets and evaluating its impact across multiple models, we demonstrate improved detection accuracy, sensitivity, and specificity. The findings underscore the efficacy of SMOTE in addressing class imbalance, thereby enhancing the reliability of anomaly detection systems in medical imaging.

An overlapping minimization-based over-sampling algorithm for binary imbalanced classification

Cyber-security experts often require the help of an automated process that filters and classifies network attacks. To apply specific preventive measures for securing networks, the classification of the attack type is the key. Many Machine Learning (ML) models have been proposed as a base for Network Intrusion Detection (NID) systems. However, their performance varies based on multiple factors. For instance, an ML model fitted on a highly imbalanced dataset can be biased toward over-represented attack types. On the other hand, paying attention only to the ML model’s performance in the minority classes can negatively affect its performance in the majority classes. This paper proposes an NID system that addresses the issue of imbalanced datasets and uses Convolutional Neural Networks (CNN) to classify the different attack types. We compare the performance of our proposed system to other systems that use: Random Over-Sampling (ROS), Synthetic Minority Oversampling TEchnique (SMOTE), Adaptive Synthetic Sampling (ADASYN), and Generative Adversarial Networks (GAN). Using the NSL-KDD and the BoT-IoT datasets for benchmarking, we show that our proposed system performs well in the minority classes: recall scores of 70.50% and 72.08% on the User to Root (U2R) and Remote to Local (R2L) attack classes of the NSL-KDD dataset, respectively, while maintaining an overall False Alarm Rate (FAR) of 6.50% and a recall of 90.46% on the binary classification task. Our proposed system scores a weighted average F1-Score of 99.45% on the multi-class classification task using the BoT-IoT dataset.

Background: Class imbalance is often discussed as a strenuous task in the realm of sentiment analysis. In an imbalanced classification, few minority class instances are unable to provide sufficient information, therefore direct learning from an unbalanced dataset can produce unsatisfactory results. This work aims to address the problem of class imbalance. Methods: At primary level this study uses a novel Synthetic Minority Oversampling Technique (SMOTE) for balancing the dataset and then proposes an ensemble model, named Ensemble Bagging Support Vector Machine (EBSVM) for opinion mining. To measure the performance of the particular approach, numerous analyses are conducted on both imbalanced and balanced datasets. Then the work compares the effectiveness of the suggested model with three base classifiers (Nave Bayes (NB), Decision Tree (DT), and Support Vector Machine (SVM)). The customer reviews for restaurants is chose as the dataset for this work. Accuracy, precision, recall and F-measure are used as metrics for evaluation. Findings: According to the results, the suggested EBSVM model excels all other individual classifiers with the imbalanced and SMOTE balanced dataset. The balanced EBSVM classifier improves the imbalanced EBSVM Classifier in terms of accuracy. Precision, recall and F-measure of the minority class in the imbalanced classifiers have been improved in balanced Classifiers. Novelty: The performance of opinion mining classifiers for imbalanced and balanced datasets is evaluated in this paper. The work examines not only general opinions, but also specific aspects such as food, service, ambiance, quality, and price. Comparing the suggested model with existing classification algorithms in the literature, it has found that it outperformed the other models. Keywords: Bagging; Accuracy; Ensemble; Precision; Recall; Fmeasure

CIRA: Class imbalance resilient adaptive Gaussian process classifier

In the semiconductor manufacturing industry, Automatic Defect Classification (ADC) plays an important role in maintaining high wafer inspection quality and reducing yield loss. ADC performance has benefitted from using machine learning (ML) algorithms; however, performance is negatively affected by the data imbalance and limited amounts of training data. Synthetic Minority Oversampling Technique (SMOTE) is an oversampling technique to adjust the skewed class distribution of a dataset so that the bias of the majority class is reduced. This paper shows that applying SMOTE achieved higher accuracy and purity on two imbalanced datasets, consisting of scanning electron microscopy (SEM) images collected with ASML-HMI eP™ and eScan® series inspection tools. The ML models are also less sensitive to the selection of hyperparameters when SMOTE is applied. We also show that better classification results can be obtained with less training samples with SMOTE; we conducted an experiment where a ML model trained on only 25% of samples with SMOTE achieved a higher ADC accuracy and purity performance compared to the same ML model trained on all samples but without SMOTE. In another experiment using a highly imbalanced SEM dataset with very few counts of the defect-of- interest (DOI), the combination of SMOTE and random undersampling of the majority class improves the accuracy by up to 5x while maintaining the same level of purity.

High-dimensional imbalanced datasets pose significant challenges in pattern recognition, often leading to overfitting and classifier bias toward majority classes. While numerous feature selection algorithms exist, most struggle to effectively address both high dimensionality and class imbalance simultaneously. This paper introduces Harmony Search Kullback–Leibler (HKL), a novel feature selection algorithm that integrates Kullback–Leibler divergence with the Harmony Search metaheuristic to specifically address these dual challenges. HKL establishes an information-theoretic foundation by employing KL divergence as a statistical framework to evaluate feature subsets based on their ability to separate minority and majority classes. Unlike existing Harmony Search variants that operate as class-blind optimizers treating feature selection as a generic optimization problem, HKL fundamentally shifts the paradigm by incorporating direct class distribution awareness into the optimization process. The algorithm implements a dual optimization approach that simultaneously balances classification performance metrics with class distribution divergence measures. This design specifically enhances minority class discrimination by prioritizing features that maximize the divergence between class distributions, ensuring that selected features provide discriminative power for underrepresented classes rather than simply favoring the majority class. Experimental validation across multiple high-dimensional biomedical datasets demonstrates that HKL consistently outperforms existing state-of-the-art methods in terms of AUC and G-mean metrics, with particular improvements for minority class classification. The algorithm achieves optimal performance while using substantially reduced feature subsets, often requiring only a quarter to half of the original features to maintain or exceed baseline classification accuracy. Statistical significance testing confirms that these performance improvements represent genuine algorithmic advantages rather than random variation. The proposed approach offers an effective solution to both dimensionality reduction and class imbalance challenges, providing a valuable tool for complex classification tasks across various domains.

A synthetic informative minority over-sampling (SIMO) algorithm leveraging support vector machine to enhance learning from imbalanced datasets

The problem of classification for imbalanced datasets is frequently encountered in practical applications. The data to be classified in this problem are skewed, i.e., the samples of one class (the minority class) are much less than those of other classes (the majority class). When dealing with imbalanced datasets, most classifiers encounter a common limitation, that is, they often obtain better classification performances on the majority classes than those on the minority class. To alleviate the limitation, in this study, a fuzzy rule-based modeling approach using information granules is proposed. Information granules, as some entities derived and abstracted from data, can be used to describe and capture the characteristics (distribution and structure) of data from both majority and minority classes. Since the geometric characteristics of information granules depend on the distance measures used in the granulation process, the main idea of this study is to construct information granules on each class of imbalanced data using Minkowski distance measures and then to establish the classification models by using “If-Then” rules. The experimental results involving synthetic and publicly available datasets reflect that the proposed Minkowski distance-based method can produce information granules with a series of geometric shapes and construct granular models with satisfying classification performance for imbalanced datasets.

Imbalanced datasets for classification is a recurring problem in machine learning, as most real-life datasets present classes that are not evenly distributed. This causes many problems for classification algorithms trained on such datasets, as they are often biases towards the majority class. Moreover, the minority class often yields more interest for data scientist, when at the same time it is also the hardest to predict. Many different approaches have been proposed to tackle the problem of imbalanced datasets: they often rely on the sampling of the majority class, or the creation of synthetic examples for the minority one. In this paper, we take a completely different perspective on this problem: we propose to use the notion of distance between databases, to sample from the majority class, so that the minority and majority class are as distant as possible. The chosen distance is based on functional dependencies, with the intuition of capturing inherent constraints of the database. We propose algorithms to generate distant synthetic datasets, as well as experimentations to verify our conjecture on the classification on distant instances. Despite the mitigated results obtained so far, we believe this is a promising research direction, at the intersection of machine learning and databases, and it deserves more investigations.

Skewed class proportions in real-world datasets present a challenge for machine learning algorithms, as they have a tendency to correctly categorize the majority class while incorrectly classifying the minority class. Such classification disparities hold significant implications, particularly in predictive scenarios involving minority groups, where misclassifying minority instances could lead to adverse outcomes. To tackle this, class imbalance learning has gained attention, with the Synthetic Minority Oversampling Technique (SMOTE) being a notable approach that addresses class imbalance by generating synthetic instances for the minority class based on their feature space neighbors. Despite its effectiveness and simplicity, SMOTE is known to suffer from a noise propagation issue where noisy and uninformative samples are introduced. While various SMOTE variants, including hybrids with undersampling, have been developed to tackle this problem, identifying noisy samples in complex real-world datasets remains a challenge. To address this, our study introduces a new SMOTE-based hybrid approach called SMOTE-centroid displacement-based k-NN (SMOTE-CDNN). SMOTE-CDNN employs centroid displacement for class prediction, which is more robust against noisy data. After SMOTE is applied, noise instances are detected and removed for clearer decision boundaries if their labels predicted by our centroid displacement-based k-NN algorithm are different from the real ones. While our experiments on 24 imbalance datasets demonstrate the resilience and efficiency of our proposed algorithm, which outperforms state-of-art resampling algorithms with various classification models, we acknowledge the need for further investigation into specific dataset characteristics and classification scenarios to determine the generalizability of our approach.