WTEN: An Advanced Coupled Tensor Factorization Strategy for Learning from Imbalanced Data

Imbalanced data classification is one of the challenging problems in data mining and machine learning research. The traditional classification algorithms are often biased towards the majority class when learning from imbalanced data. Much work have been proposed to address this problem, including data re-sampling, algorithm modification, and cost-sensitive learning. However, most of them focus on one of these techniques. This paper proposes to utilize both algorithm modification and cost-sensitive learning based on decision-theoretic rough set (DTRS) model. In particular, we use naive Bayes classifier as the base classifier and modify it for imbalanced learning. For cost-sensitive learning, we adopt the systematic method from DTRS to derive required thresholds that have the minimum decision cost. Our experimental results on three well-known text classification databases show that unified DTRS provides similar performance on balanced class distribution, outperforms naive Bayes classifier on imbalanced datasets, and is competitive with other imbalanced learning classifier.

Learning effective and efficient classifiers for imbalanced data is one of ten challenge problems in data mining research. Studying classifiers for imbalanced data is a popular area in machine learning and data mining, which also has great significance in many areas, such as cancer diagnose, credit card fraud detection and intrusion detection. The study for imbalanced data classification can be divided into three major parts: resampling data, change internal algorithm and cost-sensitive learning. Weighted feature methods can also enhance the accuracy of classification. In this paper, we intend to use a novel method combining weighted feature and cost-sensitive learning to deal with imbalanced data. We add weights to features, which causes that the position of each instance in space changes. Our goal is to increase the separation between classes by enlarging the space around the separating boundary surface through weighted features. Since the margin is enlarged, the chance that the instances in minority class are classified into majority class by mistake will be lower. Weighted Features cost-sensitive SVM (WF-CSSVM) performs well in both accuracy and cost. UCI datasets are utilized in experiment part and most of them can be classified perfectly. Accuracy, G-mean and ROC are employed as evaluation metrics.

Integrating Machine Learning (ML) in medicine has unlocked many opportunities to harness complex medical data, enhancing patient outcomes and advancing the field. However, the inherent imbalanced distribution of medical data poses a significant challenge, resulting in biased ML models that perform poorly on minority classes. Mitigating the impact of class imbalance has prompted researchers to explore various strategies, wherein Cost-Sensitive Learning (CSL) arises as a promising approach to improve the accuracy and reliability of ML models. This paper presents the first review of CSL for imbalanced medical data. A comprehensive exploration of the existing literature encompassed papers published from January 2010 to December 2022 and sourced from five major digital libraries. A total of 173 papers were selected, analysed, and classified based on key criteria, including publication years, channels and sources, research types, empirical types, medical sub-fields, medical tasks, CSL approaches, strengths and weaknesses of CSL, frequently used datasets and data types, evaluation metrics, and development tools. The results indicate a noteworthy publication rise, particularly since 2020, and a strong preference for CSL direct approaches. Data type analysis unveiled diverse modalities, with medical images prevailing. The underutilisation of cost-related metrics and the prevalence of Python as the primary programming tool are highlighted. The strengths and weaknesses analysis covered three aspects: CSL strategy, CSL approaches, and relevant works. This study serves as a valuable resource for researchers seeking to explore the current state of research, identify strengths and gaps in the existing literature and advance CSL’s application for imbalanced medical data.

Bank telemarketing campaigns often experience low subscription rates due to customer heterogeneity and severe class imbalance, which pose challenges for reliable predictive modeling. This study investigates a data-driven approach that integrates synthetic minority oversampling and cost-sensitive learning to improve the prediction of telemarketing outcomes. Experiments are conducted using the Portuguese Bank Marketing dataset, comprising 41,188 instances with a positive response rate of 11.3%. Eight machine learning models are evaluated under a unified preprocessing pipeline and five-fold stratified cross-validation, including Logistic Regression, Decision Tree, Random Forest, and Ensemble methods. The results show that Ensemble models, particularly CatBoost, XGBoost, and LightGBM, achieve improved performance compared with traditional baselines, with notable gains in minority-class recall and overall discrimination ability. The best-performing model attains an F1-score of 0.540, a recall of 0.812 for the positive class, and a ROC–AUC of 0.908. To enhance interpretability, SHAP-based analysis is applied to quantify feature contributions, identifying campaign duration, previous contact outcomes, and selected macroeconomic indicators as key predictors. These findings indicate that combining resampling strategies with cost-sensitive optimization provides a robust and transparent approach for learning from imbalanced telemarketing data, thereby supporting reproducible and data-driven financial decision-making by explicitly addressing difficulty in minority-class identification under imbalance and class imbalance under cross-entropy training in imbalanced banking data.

Area Under the ROC Curve (AUC) is an objective indicator of evaluating classification performance for imbalanced data. In order to deal with large-scale imbalanced streaming data, especially high-dimensional sparse data, this paper proposes a Sparse Stochastic Online AUC Optimization (SSOAO) method. Specifically, we first turn the standard online AUC optimization problem into a stochastic saddle point problem, then optimizing AUC by solving stochastic saddle point problem through AdaGrad optimizer. A sparse regularization term is also added for learning sparse data with high dimension. Comprehensive evaluation has been carried out on the recent benchmark. The experimental results show that the proposed SSOAO has the comparable performance on low-dimensional data, and outperforms other popular AUC optimization methods on high-dimensional sparse imbalanced streaming data. Both time and space complexity for model updating are reduced from \(O(d^2)\) to O(d), which equal to the data dimension.

An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics

In the field of data mining applications, the most challenging research work is to classify imbalanced data in supervised learning. From the beginning of the machine learning era, a huge number of researches has done by inventing a lot of successful data balancing methods, however, these are not yet fully compatible to learn from imbalanced data. The key reason is data is producing in every moment and randomly by various sources, like humans, machines, sensors, robots, and so on. To date, the existing machine learning algorithms mostly biased to the majority class instances and neglecting the minority class instances, therefore they have a crucial effect on the prediction capacity and exactness on the final results. To deal with imbalanced data the most popular classifiers are under-sampling and over-sampling, cost-sensitive learning and ensemble learning. In this paper, we have presented a novel data balancing method by employing clustering technique with SVM especially decision boundary instances for creating balanced data. The proposed approach selects the most informative majority class and minority class support vectors nearest to the decision boundary or hyperplane, and finally, combines them to establishes a novel method. We have experimented with the performance of our proposed technique with some popular single classifiers such as C4.5 Decision Tree (DT) and Naive Bayes (NB) classifier and ensemble classifiers such as Random Forest and AdaBoost on thirteen standard real-life imbalanced datasets. The experimental result shows apparently, that the proposed idea generates improved accuracy when classifying both the minority and majority class instances compared to other existing methods.

Online streaming feature selection has received extensive attention in the past few years. Existing approaches have a common assumption that the feature space of the fixed data instances increases dynamically without any missing entry. This assumption, however, does not always hold in many real-world applications. For example, in a credit evaluation system, we cannot collect the complete dynamic features for each person and/or enterprise. Motivated by this observation, this paper aims at conducting online feature selection from capricious streaming features, where features flow in one by one with some random missing entries while the number of data instances remains fixed. To do so, we propose a general framework named GF-CSF. The main idea of GF-CSF is to adopt latent factor analysis to preprocess capricious streaming features for completing their missing entries before conducting feature selection. Both theoretical and experimental analyses indicate that GF-CSF can efficiently improve any existing model of online streaming features selection to achieve online capricious streaming features selection.

Data sparsity and data imbalance are practical and challenging issues in cross-domain recommender systems (RSs). This paper addresses those problems by leveraging the concepts which derive from representation learning, adversarial learning, and transfer learning (particularly, domain adaptation). Although various transfer learning methods have shown promising performance in this context, our proposed novel method RecSys-DAN focuses on alleviating the cross-domain and within-domain data sparsity and data imbalance and learns transferable latent representations for users, items, and their interactions. Different from the existing approaches, the proposed method transfers the latent representations from a source domain to a target domain in an adversarial way. The mapping functions in the target domain are learned by playing a min-max game with an adversarial loss, aiming to generate domain indistinguishable representations for a discriminator. Four neural architectural instances of ResSys-DAN are proposed and explored. Empirical results on real-world Amazon data show that, even without using labeled data (i.e., ratings) in the target domain, RecSys-DAN achieves competitive performance as compared to the state-of-the-art supervised methods. More importantly, RecSys-DAN is highly flexible to both unimodal and multimodal scenarios, and thus it is more robust to the cold-start recommendation which is difficult for the previous methods.

REMEDIAL-HwR: Tackling multilabel imbalance through label decoupling and data resampling hybridization

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.

Truncated loss functions are robust to class noise and outliers. A robust twin bounded support vector machine is proposed in this paper that truncates the growth of its loss functions at a pre-specified point, thus, flattens the function that pre-specified score afterwards. Moreover, to make the proposed method capable of handling datasets with different imbalance ratio, cost-sensitive learning is implemented by scaling the total error of the classes based on the number of samples of each class. However, the adopted loss functions take a non-convex structure which does not always assure global optimum. To handle this issue, we suggest concave-convex procedure (CCCP) to ensure global convergence by decomposing the cost functions into additions of one convex and one concave part. The behaviour of the proposed method for varied imbalance ratio is analysed experimentally and depicted graphically. Classification performance of the proposed method in terms of AUC, F-Measure and G-Mean is compared with other related methods, viz. Hinge loss support vector machine (SVM), Ramp loss SVM (RSVM), twin SVM (TWSVM), twin bounded SVM (TBSVM), Pinball loss SVM (pin-SVM), entropy-based fuzzy SVM (EFSVM), non-parallel hyperplane Universum SVM (U-NHSVM), stochastic gradient twin support vector machine (SGTSVM), k-nearest neighbor (KNN)-based maximum margin and minimum volume hyper-sphere machine (KNN-M3VHM) and affinity and class probability-based fuzzy SVM (ACFSVM) on several real-world datasets with imbalance ratio raging from low to high. Further, to establish the significance of the proposed method in pattern classification, pair-wise statistical comparison of the methods is performed based on their average ranks on AUC. Experimental results are convincing.

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.

High dimensionality and class imbalance have been largely recognized as important issues in machine learning. A vast amount of literature has indeed investigated suitable approaches to address the multiple challenges that arise when dealing with high-dimensional feature spaces (where each problem instance is described by a large number of features). As well, several learning strategies have been devised to cope with the adverse effects of imbalanced class distributions, which may severely impact on the generalization ability of the induced models. Nevertheless, although both the issues have been largely studied for several years, they have mostly been addressed separately, and their combined effects are yet to be fully understood. Indeed, little research has been so far conducted to investigate which approaches might be best suited to deal with datasets that are, at the same time, high-dimensional and class-imbalanced. To make a contribution in this direction, our work presents a comparative study among different learning strategies that leverage both feature selection, to cope with high dimensionality, as well as cost-sensitive learning methods, to cope with class imbalance. Specifically, different ways of incorporating misclassification costs into the learning process have been explored. Also different feature selection heuristics have been considered, both univariate and multivariate, to comparatively evaluate their effectiveness on imbalanced data. The experiments have been conducted on three challenging benchmarks from the genomic domain, gaining interesting insight into the beneficial impact of combining feature selection and cost-sensitive learning, especially in the presence of highly skewed data distributions.