Tree retraining in the decision tree learning algorithm

Decision tree learning algorithms are known to be unstable, such that small changes in the training data can result in highly different output models. Instability is an important issue in the context of machine learning which is usually overlooked. In this paper, we illustrate and discuss the problem of instability of decision tree induction algorithms and propose a framework to induce more stable decision trees. In the proposed framework, the split test encompasses two advantageous properties: First, it is able to contribute multiple attributes. Second, it has a polylithic structure. The first property alleviates the race between the competing attributes to be installed at an internal node, which is the major cause of instability. The second property has the potential of improving the stability by providing the locality of the effect of the instances on the split test. We illustrate the effectiveness of the proposed framework by providing a complying decision tree learning algorithm and conducting several experiments. We have evaluated the structural stability of the algorithms by employing three measures. The experimental results reveal that the decision trees induced by the proposed framework exhibit great stability and competitive accuracy in comparison with several well-known decision tree learning algorithms.

The Decision Tree Learning Algorithms (DTLAs) are getting keen attention from the natural language processing research community, and there have been a series of attempts to apply them to verbal case frame acquisition. However, a DTLA cannot handle structured attributes like nouns, which are classified under a thesaurus. In this paper, we present a new DTLA that can rationally handle the structured attributes. In the process of tree generation, the algorithm generalizes each attribute optimally using a given thesaurus. We apply this algorithm to a bilingual corpus and show that it successfully learned a generalized decision tree for classifying the verb take and that the tree was smaller with more prediction power on the open data than the tree learned by the conventional DTLA.

Decision tree learning algorithms have a long history, but they are still popular due to their efficiency. Tree construction starts at the root and continues to each leaf node, creating a “near-optimal” tree. One of the key steps in creating a decision tree is the selection of a feature to split at each root node, which affects the classification accuracy. This process can be quite labor intensive. The article proposes a new approach to constructing decision trees based on the use of differential evolution for two-level tree optimization. Differential evolution works in two stages: at the first level, a feature is selected for separation, and at the second level, the threshold value for this feature is optimized. The results of the work were tested on several examples of classification problems.

We apply multiparty computation (MPC) techniques to show, given a database that is secret-shared among multiple mutually distrustful parties, how the parties may obliviously construct a decision tree based on the secret data. We consider data with continuous attributes (i.e., coming from a large domain), and develop a secure version of a learning algorithm similar to the C4.5 or CART algorithms. Previous MPC-based work only focused on decision tree learning with discrete attributes (De Hoogh et al. 2014). Our starting point is to apply an existing generic MPC protocol to a standard decision tree learning algorithm, which we then optimize in several ways. We exploit the fact that even if we allow the data to have continuous values, which a priori might require fixed or floating point representations, the output of the tree learning algorithm only depends on the relative ordering of the data. By obliviously sorting the data we reduce the number of comparisons needed per node to O(N log2 N) from the naive O(N 2), where N is the number of training records in the dataset, thus making the algorithm feasible for larger datasets. This does however introduce a problem when duplicate values occur in the dataset, but we manage to overcome this problem with a relatively cheap subprotocol. We show a procedure to convert a sorting network into a permutation network of smaller complexity, resulting in a round complexity of O(log N) per layer in the tree. We implement our algorithm in the MP-SPDZ framework and benchmark our implementation for both passive and active three-party computation using arithmetic modulo 264. We apply our implementation to a large scale medical dataset of ≈ 290 000 rows using random forests, and thus demonstrate practical feasibility of using MPC for privacy-preserving machine learning based on decision trees for large datasets.

Damage prediction for regular reinforced concrete buildings using the decision tree algorithm

Dengue fever is widespread disease in the tropical areas caused by bite of female Eddie mosquito. Pakistan has been victim of this rapidly growing disease since last few years. The world health organization identified the four types of dengue fever. The experts are facing the problem of misdiagnosis of dengue fever. The Tests needed for empirical Classification of Dengue Fever takes a lot of time and money especiallyin epidemic situation in a country with limited resources. Therefore, we have used data mining techniques for the efficient classification of the dengue fever Type. The decision tree learning algorithm has been used as a classification Model. We performed two experiments using decision tree. The first general experiment shows the accuracy of 99.44%. It prunes the attributes which classify the dengue fever on the basis of the values in the dataset. The Second experiment classifies the dengue fever on the basis of expert weighted attributes, which are used in the classification on the basis of Minimum Cost and resource availability. The accuracy of this Model is still high 98.62%. We compared the performance in term of Type II error. We found that the Type II error is very low in second experiment

본 논문에서는 결정트리 학습 알고리즘을 활용한 축구 게임 수비 NPC 제어 방법을 제안한다. 제안하는 방법은 실제 게임 사용자들의 이동 방향 패턴과 행동 패턴을 추출하여 결정트리학습 알고리즘에 적용한다. 그리고 학습된 결정트리를 바탕으로 NPC의 이동방향과 행동을 결정한다. 실험결과 제안하는 방법은 결정트리 학습에 시간이 다소 걸리지만, 학습된 결정트리를 바탕으로 이동방향이나 행동을 결정하는 시간은 약 0.001-0.003 ms(밀리초)가 소요되어 실시간으로 NPC를 제어할 수 있었다. 또한, 제안하는 방법은 현재 상태 정보 뿐만 아니라 이를 분석한 관계정보, 이전 상태 정보도 함께 활용하므로, 기존방법인 (Letia98)에 비해 이동방향 결정시 높은 정확도를 나타냈다.

Rough Set theory is a mathematical theory for classification based on structural analysis of relational data. It can be used to find the minimal reduct. Minimal reduct is the minimal knowledge representation for the relational data. The theory has been successfully applied to various domains in data mining. However, a major limitation in Rough Set theory is that finding the minimal reduct is an NP-hard problem. C4.5 is a very popular decision tree-learning algorithm. It is very efficient at generating a decision tree. This project uses the decision tree generated by C4.5 to find the optimal reduct for a relational table. This method does not guarantee finding a minimal reduct, but test results show that the optimal reduct generated by this approach is equivalent or very close to the minimal reduct.

Code smells are generally not considered as bugs; instead, they point out certain shortcomings in the software design or code. Identification of code smell is a necessary step for improving the software quality and reducing the maintenance effort. In this study, we introduce a bad smell prediction technique based on object-oriented software metrics that use Decision Tree (DT) and Random Forest (RF) machine learning algorithm. An open-source project, namely JHOTDRAW, was used as our dataset, for which values of object-oriented software metrics were calculated. Two feature selection methods-Random Forest Importance (RFI) and Information Gain (IG) were applied to extract the most relevant attributes for the prediction of code smells, namely, Feature envy, Dispersed coupling, refused parent bequest, and God class. The random-search algorithm was used to tune the parameters of Random Forest and Decision Tree. Results show that the best classification accuracy for Decision Tree was obtained at 99.13% for refused parent bequest code smells. Results also show that after using the Random Forest classifier, refused parent bequest smell was predicted with the best accuracy of 99.14%. Finally, in this research study, a set of code smell prediction rules were extracted using Decision Tree.

This paper proposed a diagnostic supporting tool - i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> DiaKAW (intelligent and interactive knowledge acquisition workbench), which automatically extracts useful knowledge from massive medical data by applying various data mining techniques for supporting real medical diagnosis. To fulfill this effort, i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> DiaKAW has been developed to provide a novel dynamic on-line decision tree learning scheme - i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> Learning (intelligent and incremental learning) methodology that is able to incrementally incorporate the new incoming data with an existing decision tree without relearning the entire tree from scratch. i <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">+</sup> Learning strategy makes up for the traditional incremental decision tree learning algorithms by concerning the new available features in addition to the new coming instances. Such theory is a new attempt and is designed specifically for bridging the current gaps. The empirical results demonstrate a comparison of performance of various classification algorithms on several real-life datasets from UCI repository.

In this paper is presented an experimental study with Data Mining (DM) tools, such as, CMSR Data Miner, KnowledgeSTUDIO and RapidMiner. It is goal to discover at that problems (classification, regression, clustering, association, among others) may be used, including, to verify if can treat Big Data, these tools were applied to three databases: Breast Cancer, Iris and Wine (obtained from the Machine Learning repository from University of California). In the experiments with the CMSR Data Miner tool, the Decision Tree and SOM (Self Organizing Maps) Neural Network algorithms of Data Mining were used. After that, with the KnowledgeSTUDIO tool, Logistic Regression, Deep Learning and K-Means algorithms were used. And finally, with the RapidMiner tool, Decision Tree, Naive Bayes and Deep Learning algorithms of Data Mining were applied. Additionally, it is also with this research aim to decision making by organizational managers, helping them to choose tools from DM more appropriate to their daily work.

In this paper we propose a new image classification technique. According to this note that most research focuses on extraction of features in the frequency domain, location, and reduction of feature dimensions, in this research we focused on learning step in image classification. The main aim is to use the heuristic methods to increase the function of the estimator of the learning algorithm and continue to achieve the desired state, as well as categorization without user interference and automatically performed by the model produced from the above steps. So, in this paper, a new learning approach based on the Salp Swarm Algorithm was proposed that was implemented and evaluated on learning algorithm Decision Tree, K-Nearest Neighbors and Naïve Bayes. The results demonstrate the improvement of the performance of learning algorithms in all the achieved criteria by using the SSA algorithm in comparison with traditional learning algorithms. In the accuracy, sensitivity, classification error and F1 criterion, the best performance of the proposed model is using the Decision Tree learning method with values of 99.17%, 100%, 0.83% and 95.65% respectively. In the specificity and precision criterion, the best performance of the proposed model is based on K-Nearest Neighbors learning method with values of 100%.

The decision-tree learning algorithms, e.g., C5, are good at dataset classification. But those algorithms usually work with only one attribute at a time. The dependencies among attributes are not considered in those algorithms. Unfortunately, in the real world, most datasets contain attributes, which are dependent. Generally, these dependencies are classified into two types: categorical-type and numerical-type dependencies. Thus, it is very important to construct a model to discover the dependencies among attributes, and to improve the accuracy of the decision-tree learning algorithms. Neural network model is a good choice to concern with these two types of dependencies. In this paper, we propose a Neural Decision Tree (NDT) model to deal with the problems described above. NDT model combines the neural network technologies and the traditional decision-tree learning capabilities to handle the complicated and real cases. The experimental results show that the NDT model can significantly improve the accuracy of C5.

An assessment of machine learning algorithms for healthcare analysis based on improved MapReduce

This paper presents a posture recognition system aimed at detecting sitting postures of a wheelchair user. The main goals of the proposed system are to identify and inform irregular and improper posture to prevent sitting-related health issues such as pressure ulcers, with the potential that it could also be used for individuals without mobility issues. In the proposed monitoring system, an array of 16 screen printed pressure sensor units was employed to obtain pressure data, which are sampled and processed in real-time using read-out electronics. The posture recognition was performed for four sitting positions: right-, left-, forward- and backward leaning based on k-nearest neighbors (k-NN), support vector machines (SVM), random forest (RF), decision tree (DT) and LightGBM machine learning algorithms. As a result, a posture classification accuracy of up to 99.03 percent can be achieved. Experimental studies illustrate that the system can provide real-time pressure distribution value in the form of a pressure map on a standard PC and also on a raspberry pi system equipped with a touchscreen monitor. The stored pressure distribution data can later be shared with healthcare professionals so that abnormalities in sitting patterns can be identified by employing a post-processing unit. The proposed system could be used for risk assessments related to pressure ulcers. It may be served as a benchmark by recording and identifying individuals’ sitting patterns and the possibility of being realized as a lightweight portable health monitoring device.