Genetic Programming based Evolution of Classification Trees for Decision Support in Banking Sector

Credit decision-making is a vital process in the banking sector as it helps to reduce losses by identifying non-creditable individuals. Classification algorithms in data mining provide accurate results in the aforementioned area. But, such real-world lending environments require classification results to be easy to interpret. The lack of explicability of several existing classifiers makes banks reluctant in using them. An ideal classifier needs to be accurate with interpretability encapsulated within it. Decision trees are accurate, but for large datasets, the tree becomes very large and may not be comprehensible. Genetic programming (GP) is widely applied for solving classification problems since it can produce smaller trees by using tree-size, as fitness measure or by depth-limiting the trees. Hence, we propose an algorithm named GPeCT that merges decision tree and GP to produce a near-optimal decision tree classifier. We demonstrate the performance of GPeCT through experiments on large datasets from banking and other domains.

Classification is a machine learning procedure that tags data instances into predefined class labels which are used to predict the data according to those Classes. Many classification algorithms in data mining have been stated, such as: C4.5, Apriori, Genetic algorithm, and Fuzzy set approaches which mainly uses heuristic or greedy search to get frequent sets in data for classification, resulting in high error ratio. Recently, a new method for classification has been proposed, called the Classification Using Association also known as associative classification. The main purpose for this is to mine class-association rules. Associative Classification has more advantages than the heuristic and greedy method, as it easily removes noise and higher accuracy is obtained. It additionally generates a rule set, that are more complete than traditional classification methods. This paper presents a meticulous survey on various Associative classification techniques. Moreover, a comparative analysis of accuracy and efficiency of those methods is presented.

Researchers in higher education are beginning to explore the potential of data mining in analyzing data for the purpose of giving quality service and needs of their graduates. Thus, educational data mining emerges as one tools to study academic data to identify patterns and help for decision making affecting the education. This paper predicts the employability of IT graduates using nine variables. First, different classification algorithms in data mining were tested making logistic regression with accuracy of 78.4 is implemented. Based on logistic regression analysis, three academic variables directly affect; IT_Core, IT_Professional and Gender identified as significant predictors for employability. The data were collected based on the five year profiles of 515 students randomly selected at the placement office tracer study.

Recent advances in computing, communications, digital storage technologies, and high-throughput data-acquisition technologies, make it possible to gather and store incredible volumes of data. It creates unprecedented opportunities for large-scale knowledge discovery from huge database. Data mining (DM) technology has emerged as a means of performing this discovery. There are countless researchers working on designing efficient data mining techniques, methods, and algorithms. Many data mining methods and algorithms have been developed and applied in a lot of application fields [1]. Unfortunately, most data mining researchers pay much attention to technique problems for developing data mining models and methods, while little to basic issues of data mining [2]. In this talk, some basic issues of data mining are addressed. What is data mining? What is the product of a data mining process? What are we doing in a data mining process? What is the rule we should obey in a data mining process? Through analyzing existing data mining methods, and domain-driven (or user-driven) data mining models [3-5], we find that we should take a data mining process as a process of knowledge transformation. Based on this understanding of data mining, a conceptual data mining model of domain-oriented data-driven data mining (3DM) is proposed [2]. The relationship between traditional domain-driven (or user-driven) data mining models and our proposed 3DM model is also analyzed. Some domain-oriented data-driven data mining algorithms for mining such knowledge as default rule [6], decision tree [7], and concept lattice [8] from database are proposed. The experiment results for these algorithms are also shown to illustrate the efficiency and performance of the knowledge acquired by our 3DM data mining algorithms.

In recent years, the use of addictive drugs and substances has turned out to be a challenging social problem worldwide. The illicit use of these types of drugs and substances appears to be increasing among elementary and high school students. After becoming addicted to drugs, life becomes unbearable and gets even worse for their users. Scientific studies show that it becomes extremely difficult for an individual to break this habit after being a user. Hence, preventing teenagers from addiction becomes an important issue. This study focuses on an urgent precaution system that helps families and educators prevent teenagers from developing this type of addiction. The aim of this study is to detect a teenager's probability of being a drug abuser using classification algorithms in machine learning and data mining. The objective is not to test the classifiers theoretically on the benchmark datasets, but rather to use this study as a basis for advanced and detailed studies in this field in the future. This paper not only uses a special dataset but also focuses on psychometrics and statistics. The findings of this study show that if there is a computed high risk for a teenager, some precautions, if necessary, may be taken by educators and parents to keep the teenager away from those substances.

Abstract: The service dramatically affects the number of guests staying at the hotel. Bali is the most visited tourist area by foreign tourists. Therefore, improved service is crucial for determining the rating level of a hotel. This research aims to combine two data mining algorithms: clustering and classification. This research is expected to contribute to hospitality in improving the best services for tourists, especially in the City of Bali. Clustering algorithms are used to group the best number of hotels based on the four clusters selected from the k-means clustering algorithm. The classification algorithm using C4.5 determines the factors most dominant in determining the hotel rating level based on the gain ratio. The data used in this study results from observations on the website agoda.com in Bali of 51 data. The results of this study explained that cluster_0 is the highest-rated cluster, with a total number of 19 hotels found in claster_0. Data cluster0 is used for classification analysis using a decision tree, and the most dominant factor is the service factor, with an accuracy of 80%. Keywords: data mining; kmeans; decision tree; hotel; bali; Abstrak: Pelayanan sangat mempengaruhi jumlah pengunjung yang menginap dihotel. Bali merupakan daerah wisata paling banyak dikunjungi oleh wisatawan mancanegara. Oleh karena itu, peningkatan pelayanan sangat penting untuk penentuan level rating dari hotel. Tujuan dari penelitian ini untuk menggabungkan dua algoritma data mining yaitu clustering dan klasifikasi. Dengan penelitian ini diharapkan dapat memberikan kontribusi bagi perhotelan dalam meningkatkan pelayanan yang terbaik bagi wisatawan khususnya di Kota Bali. Algoritma Clustering digunakan untuk mengelompokkan dari jumlah hotel yang terbaik berdasarkan empat cluster yang dipilih dari algoritma clustering berupa k-means. Algoritma klasifikasi menggunakan C4.5 digunakan untuk mengetahui faktor apa yang paling dominan dalam menentukan level rating hotel berdasarkan gain ratio. Data yang digunakan dalam penelitian ini hasil observasi di website agoda.com di bali sebanyak 51 data. Hasil dari penelitian ini menjelaskan dataset cluster_0 merupakan cluster rating tertinggi dengan jumlah 19 hotel yang terdapat di cluster_0. Data cluster_0 digunakan untuk analisis klasifikasi menggunakan decesion tree, didapat faktor yang paling dominan adalah faktor layanan dengan nilai akurasi sebesar 80%. Kata kunci: data mining; kmeans; decision tree; hotel; bali;

ObjectivesThe interpretation of clinical data for the diagnosis of coronary heart disease can be done using algorithms in data mining. Most clinical data interpretation systems for diagnosis developed using data mining algorithms with a black-box approach cannot recognize examination attribute relationships with the incidence of coronary heart disease.MethodsThis study proposes a system to interpretation clinical examination results for the diagnosis of coronary heart disease based the decision tree algorithm. This system comprises several stages. First, oversampling is carried out by a combination of the synthetic minority oversampling technique (SMOTE), feature selection, and the C4.5 classification algorithm. System testing is done using k-fold cross-validation. The performance parameters are sensitivity, specificity, positive prediction value (PPV), negative prediction value (NPV) and the area under the curve (AUC).ResultsThe results showed that the performance of the system has a sensitivity of 74.7%, a specificity of 93.7%, a PPV of 74.2%, an NPV of 93.7%, and an AUC of 84.2%.ConclusionsThis study demonstrated that, by using C4.5 algorithms, data can be interpreted in the form of a decision tree, to aid the understanding of the clinician. In addition, the proposed system can provide better performance by category.

Induction of decision trees as classification models through metaheuristics

Scaling up the data-mining classification algorithms to very large datasets has been attracting growing interest now a days. Many techniques have been employed to improve those algorithms but efficient data-mining classification algorithms that have a minimal decrease in accuracy with little increase in time complexity remain very important. The C4.5 algorithm is one of the data mining classification algorithms that have been used for uncovering hidden patterns and gleaning useful and novel knowledge in such large datasets. This work proposes a new C4.5 data mining algorithm with a lesser time complexity for large dataset compared with traditional C.45 algorithm, but however for smaller dataset traditional C.45 algorithm has lesser time complexity. The new algorithm was improved using Principle of Equivalent of Infinitesimal and Arithmetic Mean Best Selection Attribute.

Feature construction using genetic programming is carried out to study the effect on the performance of a range of classification algorithms with the inclusion of the evolved attributes. Two different fitness functions are used in the genetic program, one based on information gain and the other based on the gini index. The classification algorithms used are three classification tree algorithms, namely C5, CART, CHAID and an MLP neural network. The intention of the research is to ascertain if the decision tree classification algorithms benefit more using features constructed using a genetic programme whose fitness function incorporates the same fundamental learning mechanism as the splitting criteria of the associated decision tree.

Evolving dynamic fitness measures for genetic programming

Object based image analysis techniques give accurate results when a good knowledge base is extracted from remote sensing imagery. Data mining algorithms and especially evolutionary process can extract useful knowledge that can be used in different fields. In this paper, object-oriented classification was used, more particularly object-based image analysis approach (OOIA) to classify a large feature space composed of a very high spatial resolution satellite image (VHR). Genetic programming (GP) concept was applied to extract classification rules with an induction form. Comparison of the performance of three GP algorithms (Bojarczuc_GP, Falco_GP and Tan_GP) was mad using JCLEC Framework. Results showed two main conclusions. 1) testing and evaluation of the generated rules allow us to discover that GP algorithms can classify and extract useful knowledge from VHR satellite data. 2) evaluation of the performance of the three Genetic programming models demonstrates that the Bojarczuk model is efficient on accuracy classification than the Falco and Tan models.

A fuzzy logic based resource manager (RM) has been developed that automatically allocates electronic attack resources in real-time over many dissimilar platforms. Two different data mining algorithms have been developed to determine rules, strategies, and fuzzy decision tree structure. The first data mining algorithm uses a genetic algorithm as a data mining function and is called from an electronic game. The game allows a human expert to play against the resource manager in a simulated battlespace with each of the defending platforms being exclusively directed by the fuzzy resource manager and the attacking platforms being controlled by the human expert or operating autonomously under their own logic. This approach automates the data mining problem. The game automatically creates a database reflecting the domain expert's knowledge. It calls a data mining function, a genetic algorithm, for data mining of the database as required and allows easy evaluation of the information mined in the second step. The criterion for re- optimization is discussed as well as experimental results. Then a second data mining algorithm that uses a genetic program as a data mining function is introduced to automatically discover fuzzy decision tree structures. Finally, a fuzzy decision tree generated through this process is discussed.

As the most widely known data mining algorithm, classification algorithms have attracted wide attention. K-Nearest Neighbor (KNN) algorithm and decision tree algorithm are the two widely known algorithms in classification algorithms. Sometimes, people not sure how to choose the suitable to solve the classification problems. In this paper, we establish KNN algorithm model and decision tree ID3 algorithm model to analyze the accuracy of the two algorithms in the same data set with different number of features. Through the learning curve and cross validation, we find ID3 algorithm is better than KNN algorithm, and when the number of feature increased the accuracy of KNN is increasing while ID3 is decreased.

In this paper, genetic programming (GP) combined with decision trees is used to evolve the structure and weights for artificial neural network (ANN). The learning rule of the decision tree technique is defined as a function of global information by employing a divide-and-conquer strategy. Learning rules with lower fitness values are replaced by new ones generated using GP techniques. The reciprocal connection between decision tree and GP emerges from the coordination of learning rules. Since there is no constraint on initial network structure, a more suitable network can be found for a given task. Further, the fitness values are improved by using a hybrid GP technique which is a combined technique of GP and back propagation. The proposed method is applied to a medical diagnostic system and experimental results demonstrate that effective learning rule evolves.