Algorithms for Decision Tree Construction

This chapter is devoted to the study of 16 types of greedy algorithms for decision tree construction. The dynamic programming approach is used for construction of optimal decision trees. Optimization is performed relative to minimal values of average depth, depth, number of nodes, number of terminal nodes, and number of nonterminal nodes of decision trees. We compare average depth, depth, number of nodes, number of terminal nodes and number of nonterminal nodes of constructed trees with minimum values of the considered parameters obtained based on a dynamic programming approach. We report experiments performed on data sets from UCI ML Repository and randomly generated binary decision tables. As a result, for depth, average depth, and number of nodes we propose a number of good heuristics.

The purpose of this work is to investigate some of the algorithms used for representing switching functions by space efficient decision diagrams. This representation is an optimization problem. Adopting the terminology from combinatorial optimization problems, we have:Problem Instance: A multivalued function f(x) where x is a vector, and a storing cost, s(xi), associated with each variable xi in x.Feasible Solution: A decision diagram D (acyclic, directed graph with indegrees of the internal nodes possibly exeeding one) representing f(x).Objective Function: The total storage cost,S(D) = Σ s(xi) for each feasible solution D.xie DOptimal Solution: A decision diagram D representing the function with minimum total storage cost S(D), i.e. a feasible solution which minimizes the objective function.To date, only two algorithms for the construction of optimal decision diagrams with respect to the total storage cost have been developed; Reinwald & Soland's branch & bound algorithm and Thayse's, based on the extension of Shannon's expansion formula. The second algorithm is analyzed and proven to be highly exponential in the number of variables. This comes as no surprise since, in most cases, the construction of optimal decision trees is an NP-hard problem. The construction of decision diagrams is generally more complicated than that of trees. One cannot seriously doubt that the constructions of optimal diagrams is also NP-hard for most costs including the total storage cost. Therefore, a heuristic approach to this problem is needed. Two heuristics, based on the greedy algorithmic technique, are presented. These heuristics with the analysis of Thayse's algorithm constitute the major achievements of this work.

The construction of decision trees is an efficient tool for inductive learning, and fuzzy decision trees are particularly interesting because they enable the user to take into account imprecise descriptions of the cases, or heterogeneous values (symbolic, numerical, or fuzzy). However, since the method to construct a fuzzy decision tree is not unique, in this paper, a comparative study is presented to point out differences between three methods. This study focus on differences between methods when ranking attributes during the construction of a fuzzy decision tree. The aim is to enable the reader to understand what kind of fuzzy decision tree is obtained by each method.

Optimal Decision Trees for Feature Based Parameter Tuning: Integer Programming Model and VNS Heuristic

Decision trees are one of the most popular data-mining techniques for knowledge discovery. Many approaches for induction of decision trees often deal with the continuous data and missing values in information systems. However, they do not perform well in real situations. This paper presents a new algorithm, decision tree construction based on the Cloud transform and Rough set theory under the characteristic relation (CR), for mining classification knowledge from a given data set. The continuous data is transformed into discrete qualitative concepts via the cloud transformation and then the attribute with the smallest weighted mean roughness under the characteristic relation is selected as the current splitting node. Experimental evaluation shows the decision trees constructed by the CR algorithm tend to have a simpler structure, much higher classification accuracy and more understandable rules than those by C5.0 in most cases.

Uniform distribution is the simplest statistical distribution. Although there is hardly any hydrologic variable that follows a uniform probability distribution, it is invoked in a variety of applications. For example, in Bayesian statistical modeling in hydrology it is frequently used as a prior distribution. In systems hydrology, uniform distribution is the pulse function obtained by subtracting two step functions lagged by the length of the uniform distribution. The pulse function is a key to deriving the unit hydrograph theory. The instantaneous unit hydrograph of the rational method, used in urban hydrology, is a uniform distribution (Singh, 1988). Of all the statistical distributions, uniform distribution has the highest entropy. In river morphology, when a river approaches equilibrium or dynamic equilibrium, its characteristics tend to follow a uniform distribution. Under equilibrium, rivers follow the minimum rate of energy dissipation. Furthermore, a river constantly adjusts its cross-sectional geometry and longitudinal profile to accommodate the influx of water and sediment coming from its drainage basin, and this adjustment is in accordance with the principle of maximum entropy. Thus, there is a close link between equilibrium and uniform distribution and then between maximum entropy (uniform distribution) and minimum rate of energy dissipation. This link plays a fundamental role in river engineering and training works, river morphology, evolution of deltas, etc.

The growing availability of large amounts of multimedia contents in science and industry have made data mining applications such as data classification highly demanding. The contribution of this paper is two-fold. First, we propose an approach for constructing a decision tree based classification model for multimedia contents. Second, in order to speed up the performance of the proposed model, we propose a hybrid CPU-GPU approach for construction of decision tree on Graphic Processing Unit (GPU). Our approach not only accelerates the construction of decision tree via GPU computing, but also does so by considering the power and energy consumption of the GPU. Through the experiments, we demonstrate that the proposed hybrid CPU-GPU approach outperforms CPU-based sequential implementation by several times.

Decision trees with minimum average depth for sorting eight elements

Slope failure prediction using a decision tree: A case of engineered slopes in South Korea

With the rapid development of China's economy, credit business has become one of the main business of financial institutions such as banks, and the evaluation of personal credit is one of the important components of loan business of financial institutions such as banks. However, how to accurately and efficiently predict the credit of borrowers has become an urgent problem for practitioners in the financial field. More and more attention has been paid to using machine learning algorithm to predict people's credit rating in the era of artificial intelligence. Using previous data, this paper uses five models of support vector machine, K-nearest neighbor, naive bayes, convolutional neural network and decision tree in machine learning algorithms to predict people's credit rating in four countries of China, Japan, Australia and Germany by modeling, using accuracy and F1score as evaluation criteria, the accuracy of five models for credit prediction is obtained, and the definition and structure of decision tree model, principle, and decision tree construction and algorithm are emphatically introduced. The results show that the accuracy of decision tree algorithm models for credit prediction is relatively high and the most stable.

Learning from structured data is becoming increasingly important. Besides the well-known approaches which deal directly with complex data representations (inductive logic programming and multi-relational data mining), new techniques have been recently proposed by upgrading propositional learning algorithms. Focusing on distance-based methods, these techniques are extended by incorporating similarity functions defined over structured domains, for instance a k-NN algorithm solving a graph classification problem. Since a measure between objects is the essential component for this kind of methods, this paper starts with a description of some of the recent similarity functions defined over common structured data (lists, sets, terms, etc.). However, many of the most common classification techniques, such as decision tree learning, are not distance-based methods or cannot be directly adapted to be so (as kernel methods and neural networks have been adapted). In this work, we extend decision trees to use any kind of similarity function. The method is inspired by centre splitting, which constructs decision trees by defining splits based on the distance to two or more centroids. We include an experimental analysis with both propositional data and complex data. Apart from the advantages of the new proposed method, it can be used as an example of how other partition-based methods can be adapted to deal with distances and, hence, with structured data.

"ENGINEER", established in 1973 by The Institution of Engineers, is a Journal for dissemination of Engineering knowledge, published quarterly.If you are interested in contributing an Original Technical Paper based on research by the author(s) (intended for Section I) or an Original Article of Professional or Technical interest related to Engineering. (Section II) to this journal please go through the publication checklist which can be downloaded from here.Cover DescriptionCover images show the ancient Rambakan Oya sluice canal which is the longest sluice canal in Sri Lanka and the newly constructed Rambakan Oya reservoir.

10559 Background: Gastric cancer is one of the most common malignancy in the world and one of the leading causes of cancer related death in Korea. Most treatments for advanced gastric cancer have limited efficacy. So early detection of gastric cancer could have profound impact on the successful treatment. Application of multiple biomarkers may improve the diagnostic prediction to distinguish cancer from non-cancer. ProteinChip Surface-Enhanced Laser Desorption/Ionization Time-of-flight Mass Spectrometry (SELDI-TOF-MS) system is one of the currently used techniques to identify biomarkers for cancers. In this study, we have explored whether the serum proteomic patterns by ProteinChip SELDI system can differentiate gastric cancers from non-cancer cohorts. Methods: We have screened protein profiles of 100 serum samples obtained from 60 gastric cancer patients and 40 healthy individuals. Protein expression profiles were expressed on ProteinChip Array and analyzed by PreoteinChip Reader. Peak intensities were normalized by total ion currency and analyzed by the Biomarker Wizard Software to identify the peaks showing significantly different intensities between normal and cancer groups. Classification analysis and construction of decision trees were done with the Biomarker Pattern Software. Results: SELDI -TOF-MS by averaging 50 laser spots collected at a laser intensity setting of 160, a detector sensitivity of 6, and mean mass range of 30 kDa. Seventeen protein peaks shown significant differences between two groups were chosen to make a protein biomarker pattern. The decision tree which gives the highest discrimination included four peaks at 5,919, 8,583, 10,286, and 13,758 as splitters. The sensitivity and the specificity for classification of with the decision tree giving the highest discrimination were 96.7% (58/60) and 97.5% (39/40), respectively. When the protein biomarker pattern was tested with the blinded test set including 30 gastric cancer patients and 20 healthy individuals and, it yielded a sensitivity of 93.3% (28/30) and a specificity of 90% (18/20). Conclusions: These results suggest that serum-protein profiling pattern by SELDI system may distinguish gastric cancer patients from normal counterparts with relatively high sensitivity and specificity. No significant financial relationships to disclose.

In this paper, we present an analytical model to find the probability density function (PDF) of link life time in vehicular ad hoc networks (VANETs). Assuming free flow traffic state and the vehicle speed to be a random variable with uniform distribution, we analyse the impact of vehicle mobility, vehicle density and transmission range on the link duration PDF and mean link duration in VANETs. Our analytical and simulation results suggest that, in the free flow uncongested traffic state in which vehicle speed has uniform probability distribution, the link duration PDF can be approximated as lognormal with appropriate parametrization. We present the Kolmogorov-Smirnov goodness-of-fit test results to justify this claim.

In most pattern recognition (PR) system, selecting the best feature vectors is an important task. Feature vectors serve as a reduced representation of the original data/signal/input that helps avoid the curse of dimensionality in a PR task. In this work, we consider further effort in selecting the best feature vectors for the PR task. A framework to determine the best eigenvectors of two main human postures based on the rules of thumb of Principal Component Analysis (PCA) has been developed, which includes the KG-rule, cumulative variance and the scree test. Accordingly, two rules of thumb suggest in retaining only the first six eigenvectors or also known as 'eigenposture', as inputs for classification. Using decision tree (DT) as our classifier, two distinct processes need to be implemented namely, building the tree and then performing classification using the top-down approach in DT construction. At each node, decision to what is likely to be the best split is determined using the predicted value. Each branch in the tree is labeled with its decision rule whilst each terminal node is labeled with the predicted value of that node. Cross validation ensures selection of an optimum tree size and helps avoid the problem of over fitting. Consequently, the framework has enabled us to select the best and most optimized eigenpostures for classification purpose.