A Benefit-Cost Based Method for Cost-Sensitive Decision Trees

Decision tree learning is one of the most widely used and practical methods for inductive inference. A fundamental issue in decision tree inductive learning is the attribute selection measure at each non-terminal node of the tree. However, existing literatures have not taken both classification ability and cost-sensitive into account well. In this paper, we present a new strategy for attributes selection, which is a trade-off method between attributes' information and cost-sensitive learning including misclassification costs and test costs with different units, for selecting splitting attributes in cost-sensitive decision trees induction. The experimental results show our method outperform than the existing methods, such as, information gain method, total costs methods, in terms of the decrease of misclassification costs with different missing rate and various costs in UCI datasets.

Abstract: Decision tree induction has been widely studied and applied. In safety applications, such as determining whether a chemical process is safe or whether a person has a medical condition, the cost of misclassification in one of the classes is significantly higher than in the other class. Several authors have tackled this problem by developing cost-sensitive decision tree learning algorithms or have suggested ways of changing the distribution of training examples to bias the decision tree learning process so as to take account of costs. A prerequisite for applying such algorithms is the availability of costs of misclassification. Although this may be possible for some applications, obtaining reasonable estimates of costs of misclassification is not easy in the area of safety. This paper presents a new algorithm for applications where the cost of misclassifications cannot be quantified, although the cost of misclassification in one class is known to be significantly higher than in another class. The algorithm utilizes linear discriminant analysis to identify oblique relationships between continuous attributes and then carries out an appropriate modification to ensure that the resulting tree errs on the side of safety. The algorithm is evaluated with respect to one of the best known cost-sensitive algorithms (ICET), a well-known oblique decision tree algorithm (OC1) and an algorithm that utilizes robust linear programming.

Aiming at the advantages in commonly used method of generating decision tree, a method of generating cost-sensitive decision tree (CSDT) based on the correlation degree of neural network attributes is proposed through quoting the correlation degree of neural network attributes and cost-sensitive learning. This method reduces the condition attributes by using rough set theory, and it takes the correlation degree and cost performance of attributes as the bases of split node to build the CSDT by using modified information gain method during the process of building decision tree. It is shown in test result that such method is superior to commonly used method of generating decision tree in classification accuracy and the number of nodes generated.

Aiming at the advantages in commonly used method of generating decision tree, a method of generating cost-sensitive decision tree (CSDT) based on the correlation degree of neural network attributes is proposed through quoting the correlation degree of neural network attributes and cost-sensitive learning. This method reduces the condition attributes by using rough set theory, and it takes the correlation degree and cost performance of attributes as the bases of split node to build the CSDT by using modified information gain method during the process of building decision tree. It is shown in test result that such method is superior to commonly used method of generating decision tree in classification accuracy and the number of nodes generated.

Delays in the completion of pvd production can be caused by several factors. Including the actual experience in the production of the difficulty of each process and color type, even the difficulty of the product type can also be affected. In this study, the prediction of the delay in the completion of pvd production was carried out using the C4.5 decision tree and Multilayer Perceptron data mining method approach using Production Results data at PT. Surya Toto Indonesia, whose results are expected to provide information and input for the company in making production plans in the future. The data testing method was carried out with 5 (five) testing times with different amounts of data to determine the level of consistency of accuracy obtained. C4.5 gives the results of a decision tree where the root is the color type and as the leaf is the product category, type type and order period. The average value of accuracy generated in the C4.5 decision tree method is 87.15%. While the Multilayer Perceptron obtained an average accuracy of 87.98%, which is greater than the decision tree method with a difference of 0.83%.

Cost-sensitive matrixized classification learning with information entropy

Since conventional decision tree (DT) induction methods cannot efficiently take advantage of geospatial knowledge in the classification of remotely sensed imagery, a co-location decision tree (CL-DT) method, combining the co-location technique with the conventional DT method, has been proposed by several researchers. However, the CL-DT method only considers the Euclidean distance of neighborhood events, which cannot truly reflect the co-location relationship between instances for which there is a nonlinear distribution in a high-dimensional space. For this reason, this paper develops the theory and method for a maximum variance unfolding (MVU)-based CL-DT method (known as MVU-based CL-DT). The presented method has been validated by classifying remote sensing image and is compared with CL-DT method and random forest (RF) method. The experimental results show that the proposed method can better construct decision tree and reach a high classification accuracy.

Co-location decision tree for enhancing decision-making of pavement maintenance and rehabilitation

Sparse representation has been extensively studied for image super-resolution (SR), and it achieved great improvement. Deep-learning-based SR methods have also emerged in the literature to pursue better SR results. In this paper, we propose to use a set of decision tree strategies for fast and high-quality image SR. Our proposed SR using decision tree (SRDT) method takes the divide-and-conquer strategy, which performs a few simple binary tests to classify an input low-resolution (LR) patch into one of the leaf nodes and directly multiplies this LR patch with the regression model at that leaf node for regression. Both the classification process and the regression process take an extremely small amount of computation. To further boost the SR results, we introduce a SR using hierarchical decision trees (SRHDT) method, which cascades multiple layers of decision trees for SR and progressively refines the estimated high-resolution image. Inspired by the random forests approach, which combines regression models from an ensemble of decision trees, we propose to fuse regression models from relevant leaf nodes within the same decision tree to form a more robust approach. The SRHDT method with fused regression model (SRHDT_f) improves further the SRHDT method by 0.1-dB in PNSR. Our experimental results show that our initial approach, the SRDT method, achieves SR results comparable to those of the sparse-representation-based method and the deep-learning-based method, but our method is much faster. Furthermore, our enhanced version, the SRHDT_f method, achieves more than 0.3-dB higher PSNR than that of the A+ method, which is the state-of-the-art method in SR.

Post-pruning in decision tree induction using multiple performance measures

The decision tree-based classification is a popular approach for pattern recognition and data mining. Most decision tree induction methods assume training data being present at one central location. Given the growth in distributed databases at geographically dispersed locations, the methods for decision tree induction in distributed settings are gaining importance. This paper extends two well-known decision tree methods for centralized data to distributed data settings. The first method is an extension of CHAID algorithm and generates single feature based multi-way split decision trees. The second method is based on Fisher's linear discriminant (FLO) function and generates multifeature binary trees. Both methods aim to generate compact trees and are able to handle multiple classes. The suggested extensions for distributed environment are compared to their centralized counterparts and also to each other. Theoretical analysis and experimental tests demonstrate the effectiveness of the extensions. In addition, the side-by-side comparison highlights the advantages and deficiencies of these methods under different settings of the distribution environments.

C4.5 algorithm is used to simplify decision tree from decision table by generating decision tree from an existing decision tree. With this algorithm, knowledge base in the decision table can be simplified. This research will build a consultation program using C4.5 algorithm which is called decision tree generator. Decision tree generator provides inference facility and a user interface for consultation. The user is required to build a knowledge base first, and the application will generate user interface automatically. There are two steps in decision tree generator: firstly, the application will build the decision tree, and after that the application will build the user interface for consultation session. The results of this research show that the decision tree generator can get goal and advice from tree exploration in consultation session.

Decision tree induction is one of the widely used classification approaches. It constructs a tree in which an internal node is split based on values of a selected attribute. Depending on the attribute selected at each level of the tree, a training dataset could lead to many different trees. Consequently, it is possible that an unseen case is classified into different and conflict classes using different trees. On the other band, recently developed association role based classifications are able to generate more interesting and useful rules than decision trees. However, the large number of rules without appropriate data structure brings the issue of efficiency. In this paper, we propose to unify decision tree classification with association-based classification using generalized decision trees (GDT). GDT generalizes the concept of decision tree to encode all interesting classification rules discovered based on association in a tree form. it inherits the merits of both approaches and removes their drawbacks. The data structure and related algorithms are discussed. Results of an experimental study are presented to indicate the advantages of GDT.

Induction of decision trees as classification models through metaheuristics

In this study, a method is proposed to estimate energy consumption and to plan maintenance works on energy lines according to energy consumption. In our study, energy consumption can be estimated at any time of the day by analysing parameters such as temperature, pressure, and wind using decision tree methods. For this aim, "Fine Tree", "Medium Tree" and "Coarse Tree" which are different implementations of decision trees are used. The success is 76.77% with the Fine Tree method, 76.42% with the Medium Tree method, and 73.92% with the Coarse Tree method. As a result, Energy consumption is estimated by using environmental conditions, time, day, month, and season information within the scope of the smart city. Thus, energy distribution companies can make energy planning by estimating the energy consumption of their customers. In addition, performing maintenance and repair operations on energy distribution lines in a convenient time period according to these results will increase customer satisfaction.