Cost-Guided Class Noise Handling for Effective Cost-Sensitive Learning

Cost-sensitive matrixized classification learning with information entropy

Noise correction to improve data and model quality for crowdsourcing

Cost-sensitive classification is an attractive topic in data mining. Although genetic programming (GP) technique has been applied to general classification, to our knowledge, it has not been exploited to address cost-sensitive classification in the literature, where the costs of misclassification errors are non-uniform. To investigate the applicability of GP to cost-sensitive classification, this paper first reviews the existing methods of cost-sensitive classification in data mining. We then apply GP to address cost-sensitive classification by means of two methods through: a) manipulating training data, and b) modifying the learning algorithm. In particular, a constrained genetic programming (CGP), a GP-based cost-sensitive classifier, has been introduced in this study. CGP is capable of building decision trees to minimize not only the expected number of errors, but also the expected misclassification costs through a novel constraint fitness function. CGP has been tested on the heart disease dataset and the German credit dataset from the UCI repository. Its efficacy with respect to cost has been demonstrated by comparisons with non-cost-sensitive learning methods and cost-sensitive learning methods in terms of the costs.

Imbalanced data classification is one of the challenging problems in data mining and machine learning research. The traditional classification algorithms are often biased towards the majority class when learning from imbalanced data. Much work have been proposed to address this problem, including data re-sampling, algorithm modification, and cost-sensitive learning. However, most of them focus on one of these techniques. This paper proposes to utilize both algorithm modification and cost-sensitive learning based on decision-theoretic rough set (DTRS) model. In particular, we use naive Bayes classifier as the base classifier and modify it for imbalanced learning. For cost-sensitive learning, we adopt the systematic method from DTRS to derive required thresholds that have the minimum decision cost. Our experimental results on three well-known text classification databases show that unified DTRS provides similar performance on balanced class distribution, outperforms naive Bayes classifier on imbalanced datasets, and is competitive with other imbalanced learning classifier.

Real-time transient stability status prediction (RTSSP) is very important to maintain the safety and stability of electrical power systems, where any unstable contingency will be likely to cause large-scale blackout. Most of machine learning methods used for RTSSP attempt to attain a low classification error, which implies that the misclassification costs of different categories are the same. However, misclassifying an unstable case as stable one usually leads to much higher costs than misclassifying a stable case as unstable one. In this paper, a new RTSSP method based on cost-sensitive extreme learning machine (CELM) is proposed, which recognizes the RTSSP as a cost-sensitive classification problem. The CELM is constructed pursuing the minimum misclassification costs, and its detailed implementation procedures for RSSTP are also researched in this work. The proposed method is implemented on the New England 39-bus electrical power system. Compared with three cost-blind methods (ELM, SVM and DT) and two cost-sensitive methods (cost-sensitive DT, cost-sensitive SVM), the simulation results have proved that the lower total misclassification costs and false dismissal rate with low computational complexity can be achieved by the proposed method, which meets the demands for the computation speed and the reliability of RTSSP.

Learning effective and efficient classifiers for imbalanced data is one of ten challenge problems in data mining research. Studying classifiers for imbalanced data is a popular area in machine learning and data mining, which also has great significance in many areas, such as cancer diagnose, credit card fraud detection and intrusion detection. The study for imbalanced data classification can be divided into three major parts: resampling data, change internal algorithm and cost-sensitive learning. Weighted feature methods can also enhance the accuracy of classification. In this paper, we intend to use a novel method combining weighted feature and cost-sensitive learning to deal with imbalanced data. We add weights to features, which causes that the position of each instance in space changes. Our goal is to increase the separation between classes by enlarging the space around the separating boundary surface through weighted features. Since the margin is enlarged, the chance that the instances in minority class are classified into majority class by mistake will be lower. Weighted Features cost-sensitive SVM (WF-CSSVM) performs well in both accuracy and cost. UCI datasets are utilized in experiment part and most of them can be classified perfectly. Accuracy, G-mean and ROC are employed as evaluation metrics.

Abstract: Decision tree induction has been widely studied and applied. In safety applications, such as determining whether a chemical process is safe or whether a person has a medical condition, the cost of misclassification in one of the classes is significantly higher than in the other class. Several authors have tackled this problem by developing cost-sensitive decision tree learning algorithms or have suggested ways of changing the distribution of training examples to bias the decision tree learning process so as to take account of costs. A prerequisite for applying such algorithms is the availability of costs of misclassification. Although this may be possible for some applications, obtaining reasonable estimates of costs of misclassification is not easy in the area of safety. This paper presents a new algorithm for applications where the cost of misclassifications cannot be quantified, although the cost of misclassification in one class is known to be significantly higher than in another class. The algorithm utilizes linear discriminant analysis to identify oblique relationships between continuous attributes and then carries out an appropriate modification to ensure that the resulting tree errs on the side of safety. The algorithm is evaluated with respect to one of the best known cost-sensitive algorithms (ICET), a well-known oblique decision tree algorithm (OC1) and an algorithm that utilizes robust linear programming.

It is often expensive to acquire data in real-world data mining applications. Most previous data mining and machine learning research, however, assumes that a fixed set of training examples is given. In this paper, we propose an online cost-sensitive framework that allows a learner to dynamically acquire examples as it learns, and to decide the ideal number of examples needed to minimize the total cost. We also propose a new strategy for Partial Example Acquisition (PAS), in which the learner can acquire examples with a subset of attribute values to reduce the data acquisition cost. Experiments on UCI datasets show that the new PAS strategy is an effective method in reducing the total cost for data acquisition.

Bayesian learning provides a simple but efficient method for classification by combining the sample information with the prior knowledge and the dependencies with probability estimates. However, Bayesian network classifiers that minimize the number of misclassification errors ignore different misclassification costs. For example, in rock burst prediction, the cost of misclassifying a rock which happens to burst as a rock which doesn’t burst is much higher than the opposite type of error. This paper studies the cost-sensitive learning and then applies it to different Bayesian Network classifiers, and the resulted algorithms are called cost-sensitive Bayesian Network classifiers. The experimental results on 36 UCI datasets validate their effectiveness in terms of the total misclassification costs. Finally, we apply the cost-sensitive Bayesian Network classifiers to some real-world rock burst prediction examples and achieve good results.

Cost-sensitive active learning with a label uniform distribution model

The main purpose of traditional classification algorithms on bioinformatics application is to acquire better classification accuracy. However, these algorithms cannot meet the requirement that minimises the average misclassification cost. In this paper, a new algorithm of cost-sensitive regularised extreme learning machine (CS-RELM) was proposed by using probability estimation and misclassification cost to reconstruct the classification results. By improving the classification accuracy of a group of small sample which higher misclassification cost, the new CS-RELM can minimise the classification cost. The 'rejection cost' was integrated into CS-RELM algorithm to further reduce the average misclassification cost. By using Colon Tumour dataset and SRBCT (Small Round Blue Cells Tumour) dataset, CS-RELM was compared with other cost-sensitive algorithms such as extreme learning machine (ELM), cost-sensitive extreme learning machine, regularised extreme learning machine, cost-sensitive support vector machine (SVM). The results of experiments show that CS-RELM with embedded rejection cost could reduce the average cost of misclassification and made more credible classification decision than others.

Nowadays in many real-world scenarios, high speed data streams are usually with non-uniform misclassification costs and thus call for cost-sensitive classification algorithms of data streams. However, only little literature focuses on this issue. On the other hand, the existing algorithms for cost-sensitive classification can achieve excellent performance in the metric of total misclassification costs, but always lead to obvious reduction of accuracy, which restrains the practical application greatly. In this paper, we present an improved folk theorem. Based on the new theorem, the existing accuracy-based classification algorithm can be converted into soft cost-sensitive one immediately, which allows us to take both accuracy and cost into account. Following the idea of this theorem, the soft-CsGDT algorithm is proposed to process the data streams with non-uniform misclassification costs, which is an expansion of GDT. With both synthetic and real-world datasets, the experimental results show that compared with the cost-sensitive algorithm, the accuracy in our soft-CsGDT is significantly improved, while the total misclassification costs are approximately the same.

Sentiment analysis has been rapidly employed for business decision support. New data mining researchers are yet to have an adequate understanding of the various applications of sentiment analysis while utilising social media data. As a result, it is critical to define the data mining and text analytics research trend holistically using existing literature. The study explores sentiment analysis research for its application in transforming social media data and identifies relevant research aspects through a comprehensive bibliometric review of 523 research articles published in the Scopus database (between 2018 and 2022) to discern the content and thematic analysis. Findings suggested that key purposes of the sentiment analysis are mainly related to innovation, transparency, and efficiency. Our review also highlights the distinctiveness of sentiment analysis for synthesising social media information to investigate various features, including the knowledge-domain map that detects author collaboration networks in the past.

Data have been accumulated to wait for being analyzed in real world. But the imperfection of data complicates the analysis process. According to “garbage in, garbage out”, model built on such data will mislead the following study. Multiple empirical studies have showed that noise in dataset dramatically decrease the classification accuracy and increase the complexity of classification. Therefore, the problem of noise in classification is always the focus in machine learning and data mining. At the same time, noise is uncertain, so the problem is also a difficult and open problem. For systematically studying the problem, we summarize and analyze the main researches from the aspects of noise model, method of handling noise and algorithms of handling noise. Based on the past and current work, we discuss some new directions in solving the problem.

In many binary classification applications, the costs of false positives and negatives are imbalanced. Furthermore, there is often uncertainty about the exact costs of these errors. A natural measure-of-interest to be minimised in such scenarios is the expected misclassification cost. We identify many situations where this measure has analytic gradients, and thus it can be used as a training loss and optimised directly using empirical risk minimisation. In particular, we derive such losses from the Beta, Gamma and Gaussian distributions to model different kinds of cost uncertainty. The Beta family includes commonly used losses such as cross-entropy, squared error and 0–1 loss as special cases. The question then arises as to when it is appropriate to directly optimize the measure-of-interest, versus using a standard surrogate like cross-entropy or focal loss during training. After revisiting the theory of surrogate losses, proper losses and cost-sensitive learning to obtain good candidate surrogates out of derived families, we conduct an empirical comparison of derived training losses that, to our knowledge, were never tried on deep neural networks before, with the aim to minimise cost-sensitive measures-of-interest. The findings show that using Beta losses in training leads to improved performance compared to traditional training objectives like cross-entropy, label smoothing, and focal loss. This improvement is seen not only in terms of misclassification cost metrics, but (perhaps surprisingly) also in conventional metrics such as accuracy, mean squared error, and the area under the ROC curve.