K + Decision Trees

Decision trees are considered as one of the efficient classification techniques in data mining fields. But the standard decision trees are unfit to cope with data pervaded with uncertainty both at the construction and classification phase. Dempster-Shafer theory offers an alternative to traditional probabilistic theory for the mathematical representation of uncertainty. This paper added new aggregation combination operator and uncertainty measure operator into general framework for data mining based on evidence theory. Combining these two operators with decision tree and multidimensional cube, decision tree technique can be extended to uncertain environment. In the phase of node splitting, this algorithm can pre-prune the decision tree and generate a decision tree with fewer branches. Simulations have shown the effectiveness of this method.

Classification of sensory data is a major research problem in wireless sensor networks and it can be widely used in reducing the data transmission in wireless sensor networks effectively and also in process monitoring. In order to examine the huge size of data set in stream model generated by sensor network, it will be analyzed different sensor's output signal, topology of sensors network, number of sensor parameters and number of acquisition data. In our wind energy monitoring, sensor node monitors six attributes: speed, direction, temperature, pressure, humidity, and battery voltage. Every attribute value is set as four measures: average, instantaneous, minimum, and maximum. This paper presents several data mining techniques applied on the wireless sensor network's data considered: Naïve Bayes, k-nearest neighbor, decision trees, IF-THEN rules, and neural networks. Before classification, the data was clustered in order to be labeled. A similarity based algorithm, k-means, was selected in the clustering process for its simplicity and efficiency. A conclusion that decision trees are a suitable method to classify the large amount of data considered is made finally according to the mining result and its reasonable explanation.

This paper presents a novel hybrid method combining genetic programming and decision tree learning. The method starts by estimating a benchmark level of reasonable accuracy, based on decision tree performance on bootstrap samples of the training set. Next, a normal GP evolution is started with the aim of producing an accurate GP. At even intervals, the best GP in the population is evaluated against the accuracy benchmark. If the GP has higher accuracy than the benchmark, the evolution continues normally until the maximum number of generations is reached. If the accuracy is lower than the benchmark, two things happen. First, the fitness function is modified to allow larger GPs, able to represent more complex models. Secondly, a decision tree with increased size and trained on a bootstrap of the training data is injected into the population. The experiments show that the hybrid solution of injecting decision trees into a GP population gives synergetic effects producing results that are better than using either technique separately. The results, from 18 UCI data sets, show that the proposed method clearly outperforms normal GP, and is significantly better than the standard decision tree algorithm.

Decision Trees have been applied widely for classification in many fields such as finance, marketing, engineering, and medicine. The increased field of application, made the requirement for understanding various aspects of decision trees in deep. In addition, it is crucial to understand the different type of costs associated with the classification task in a decision tree classifier and their relationship with the classifier’s accuracy, as balancing the two is a major concern these days in many fields such as medical diagnosis. This paper introduces the concept of decision trees, presents their various areas of application in data mining, summarizes the standard decision tree algorithms, and identifies their main advantages and disadvantages. It mainly aims to clarify relationship between the classification accuracy and classification cost in decision trees.

In uses of pre-trained machine learning models, it is a known issue that the target population in which the model is being deployed may not have been reflected in the source population with which the model was trained. This can result in a biased model when deployed, leading to a reduction in model performance. One risk is that, as the population changes, certain demographic groups will be under-served or otherwise disadvantaged by the model, even as they become more represented in the target population. The field of domain adaptation proposes techniques for a situation where label data for the target population does not exist, but some information about the target distribution does exist. In this paper we contribute to the domain adaptation literature by introducing domain-adaptive decision trees (DADT). We focus on decision trees given their growing popularity due to their interpretability and performance relative to other more complex models. With DADT we aim to improve the accuracy of models trained in a source domain (or training data) that differs from the target domain (or test data). We propose an in-processing step that adjusts the information gain split criterion with outside information corresponding to the distribution of the target population. We demonstrate DADT on real data and find that it improves accuracy over a standard decision tree when testing in a shifted target population. We also study the change in fairness under demographic parity and equal opportunity. Results show an improvement in fairness with the use of DADT.

Decision trees as possibilistic classifiers

The objective of this work is to exploit the potential of latest pattern recognition techniques in power quality applications. This paper presents a novel hybrid pattern recognizer for classification of power quality disturbances. The hybrid learning methodology integrates a multiobjective genetic algorithm (GA) and decision trees (CART) in order to evolve optimal subsets of discriminatory features for robust pattern classification. In the training phase the multiobjective GA based on the wrapper approach is used to find a subset of relevant attributes that minimizes both classification error rate and size of the tree discovered by the classification algorithm, namely CART, using the Pareto dominance approach. For a given feature subset, CART is invoked to produce a decision tree; the classification error and the complexity of the decision tree are used as the fitness functions by the GA to evolve better subsets. Experimental results reveal that the proposed multiobjective GA-CART combined approach yields improved classification performance and reduced classification time as compared to standard CART decision trees.

ODT: Optimal deadline-based trajectory for mobile sinks in WSN: A decision tree and dynamic programming approach

Augmentation in performance and security of WSNs for IoT applications using feature selection and classification techniques

Discovery of interesting rules describing the behavioural patterns of smokers' quitting intentions is an important task in the determination of an effective tobacco control strategy. In this paper, we investigate a compact and simplified rule discovery process for predicting smokers' quitting behaviour that can provide feedback to build an scientific evidence-based adaptive tobacco control policy. Standard decision tree (SDT) based rule discovery depends on decision boundaries in the feature space which are orthogonal to the axis of the feature of a particular decision node. This may limit the ability of SDT to learn intermediate concepts for high dimensional large datasets such as tobacco control. In this paper, we propose a cluster based rule discovery model (CRDM) for generation of more compact and simplified rules for the enhancement of tobacco control policy. The cluster-based approach builds conceptual groups from which a set of decision trees (a decision forest) are constructed. Experimental results on the tobacco control data set show that decision rules from the decision forest constructed by CRDM are simpler and can predict smokers' quitting intention more accurately than a single decision tree.

This dissertation is concerned with the application of notions and methods from the field of information theory to the field of communication complexity. It consists of two main parts. In the first part of the dissertation, we prove lower bounds on the randomized two-party communication complexity of functions that arise from read-once boolean formulae. A read-once boolean formula is a formula in propositional logic with the property that every variable appears exactly once. Such a formula can be represented by a tree, where the leaves correspond to variables, and the internal nodes are labeled by binary connectives. Under certain assumptions, this representation is unique. Thus, one can define the depth of a formula as the depth of the tree that represents it. The complexity of the evaluation of general read-once formulae has attracted interest mainly in the decision tree model. In the communication complexity model many interesting results deal with specific read-once formulae, such as disjointness and tribes. In this dissertation we use information theory methods to prove lower bounds that hold for any read-once formula. Our lower bounds are of the form n(f )/c^d(f ) , where n(f ) is the number of variables and d(f ) is the depth of the formula, and they are optimal up to the constant in the base of the denominator. In the second part of the dissertation, we explore the applicability of the information-theoretic method in the number-on-the-forehead model. The work of Bar-Yossef, Jayram, Kumar & Sivakumar [BYJKS04] revealed a beautiful connection between Hellinger distance and two-party randomized communication protocols. Inspired by their work and motivated by the open questions in the number-on-the-forehead model, we introduce the notion of Hellinger volume. We show that it lower bounds the information cost of multi-party protocols. We provide a small toolbox that allows one to manipulate several Hellinger volume terms and also to lower bound a Hellinger volume when the distributions involved satisfy certain conditions. In doing so, we prove a new upper bound on the difference between the arithmetic mean and the geometric mean in terms of relative entropy. Finally, we show how to apply the new tools to obtain a lower bound on the informational complexity of the ANDk function.

We investigate the problem of learning Boolean functions with a short DNF representation using decision trees as a concept description language. Unfortunately, Boolean concepts with a short description may not have a small decision tree representation when the tests at the nodes are limited to the primitive attributes. This representational shortcoming may be overcome by using Boolean features at the decision nodes. We present two new methods that adaptively introduce relevant features while learning a decision tree from examples. We show empirically that these methods outperform a standard decision tree algorithm for learning small random DNF functions when the examples are drawn at random from the uniform distribution.

We apply multiparty computation (MPC) techniques to show, given a database that is secret-shared among multiple mutually distrustful parties, how the parties may obliviously construct a decision tree based on the secret data. We consider data with continuous attributes (i.e., coming from a large domain), and develop a secure version of a learning algorithm similar to the C4.5 or CART algorithms. Previous MPC-based work only focused on decision tree learning with discrete attributes (De Hoogh et al. 2014). Our starting point is to apply an existing generic MPC protocol to a standard decision tree learning algorithm, which we then optimize in several ways. We exploit the fact that even if we allow the data to have continuous values, which a priori might require fixed or floating point representations, the output of the tree learning algorithm only depends on the relative ordering of the data. By obliviously sorting the data we reduce the number of comparisons needed per node to O(N log2 N) from the naive O(N 2), where N is the number of training records in the dataset, thus making the algorithm feasible for larger datasets. This does however introduce a problem when duplicate values occur in the dataset, but we manage to overcome this problem with a relatively cheap subprotocol. We show a procedure to convert a sorting network into a permutation network of smaller complexity, resulting in a round complexity of O(log N) per layer in the tree. We implement our algorithm in the MP-SPDZ framework and benchmark our implementation for both passive and active three-party computation using arithmetic modulo 264. We apply our implementation to a large scale medical dataset of ≈ 290 000 rows using random forests, and thus demonstrate practical feasibility of using MPC for privacy-preserving machine learning based on decision trees for large datasets.

Decision trees belong to the most effective classification methods. The main advantage of decision trees is a simple and user-friendly interpretation of the results obtained. But despite its well-known advantages the method has some disadvantages as well. One of them is that decision tree learning algorithm build an “almost optimal” tree. The paper considers the way to improve the efficiency of decision trees. The paper proposes a modification of decision tree learning algorithms by retraining the part of tree at every node training. The classification problems were solved to compare standard decision tree learning algorithms with the modified ones. Algorithm efficiency refers to the percentage of correctly classified test sample objects. Statistical analysis based on Student’s t-test was carried out to compare the efficiency of the algorithms.

Due to the major operating restrictions, ensuring security is the fundamental problem of Wireless Sensor Networks (WSNs). Because of their inadequate security mechanisms, WSNs are indeed a simple point for malware (worms, viruses, malicious code, etc.). According to the epidemic nature of worm propagation, it is critical to develop a worm defense mechanism in the network. This concept aims to establish novel malware detection in WSN that consists of several phases: “(i) Preprocessing, (ii) feature extraction, as well as (iii) detection”. At first, the input data is subjected for preprocessing phase. Then, the feature extraction takes place, in which principal component analysis (PCA), improved linear discriminant analysis (LDA), and autoencoder-based characteristics are retrieved. Moreover, the retrieved characteristics are subjected to the detection phase. The detection is performed employing combined shallow learning and DL. Further, the shallow learning includes decision tree (DT), logistic regression (LR), and Naive Bayes (NB); the deep learning (DL) includes deep neural network (DNN), convolutional neural network (CNN), and recurrent neural network (RNN). Here, the DT output is given to the DNN, LR output is subjected to CNN, and the NB output is given to the RNN, respectively. Eventually, the DNN, CNN, and RNN outputs are averaged to generate a successful outcome. The combination can be thought of as an Ensemble classifier. The weight of the RNN is optimally tuned through the Self Improved Shark Smell Optimization with Opposition Learning (SISSOOL) model to improve detection precision and accuracy. Lastly, the outcomes of the suggested approach are computed in terms of different measures.