Evolutionary Feature Construction Using Information Gain and Gini Index

Potholes are defects on the surface of roads, streets, or pavements brought on by depressions or holes, which are hazardous for vehicles and pedestrians, whether small divots or huge craters. Various methods have been explored to improve the accuracy of potholes detection. Existing approaches have advantages and disadvantages. This paper presents the proposed method of pothole detection utilising Doppler Radar signal's Power Spectrum Density (PSD) together with the Decision Tree classification algorithm. While continuous waveform (CW) radar is able to identify moving targets, it cannot localise the exact depth of the reflector, which is the prominent characteristic of potholes. In addition, the target's reflected signal is likely to be masked by nearby harmonics. Since the radar is moving despite the target of interest, mounting it on a moving vehicle offers a different perspective. This paper explores the potential of Doppler radar's signal for pothole detection while comparing two Machine Learning (ML) techniques. A commercially over-the-shelf (COTS) K-LC2 Doppler radar was employed to acquire pothole and non-pothole raw datasets. Doppler signal was hardly distinguished between pothole and non-pothole, either in the time or frequency domain. Hence, Doppler signals were converted to power spectral density (PSD), and PSD's features were extracted. Extracted features were applied with the coarse Decision Tree (DT) and K-Nearest Neighbours (KNN) classification algorithms. The result exhibits a better accuracy of 91.2% for 80:20 distribution by using the Decision Tree.

Aim: The aim of the work is to evaluate the accuracy and precision in predicting heart disease using Support Vector Machine (SVM), Random forest (RF), Logistic Regression (LR), Decision Tree (DT) Classification algorithms. Materials and Methods: Classification algorithm is appealed on a heart dataset which consists of 180 records. A framework for heart disease prediction in the medical sector comparing Random forest, Logistic Regression, Decision Tree and SVM classifiers has been proposed and developed. The sample size was calculated as 55 in each group using G power 80%. Sample size was calculated using clincalc analysis, with alpha and beta values 0.05 and 0.5, 95% confidence, pretest power 80% and enrolment ratio 1. Results: The Novel Random Forest Algorithm (92.13%), Support Vector Machine (62.51%), Logistic Regression (84.89%), Decision Tree (86.25%) classifiers produce respectively. SVM, RF exists a statistically significant difference between the two groups (p=0.001,p=.004;p<0.05).LR, RF exists a statistically insignificant difference between the two groups (p=.103, P=.080;p>0.05) both with confidence interval 95%. Hence Random forest is better than SVM, RF, DT classifiers. Conclusion: The results show that the performance of RF is better when compared with SVM, LR and DT in terms of both precision and accuracy.

Feature selection and construction are important pre-processing techniques in data mining. They allow not only dimensionality reduction but also classification accuracy and efficiency improvement. While feature selection consists in selecting a subset of relevant features from the original feature set, feature construction corresponds to the generation of new high-level features, called constructed features, where each one of them is a combination of a subset of original features. However, different features can have different abilities to distinguish different classes. Therefore, it may be more difficult to construct a better discriminating feature when combining features that are relevant to different classes. Based on these definitions, feature construction could be seen as a BLOP (Bi-Level optimization Problem) where the feature subset should be defined in the upper level and the feature construction is applied in the lower level by performing mutliple followers, each of which generates a set class dependent constructed features. In this paper, we propose a new bi-level evolutionary approach for feature construction called BCDFC that constructs multiple features which focuses on distinguishing one class from other classes using Genetic Programming (GP). A detailed experimental study has been conducted on six high-dimensional datasets. The statistical analysis of the obtained results shows the competitiveness and the outperformance of our bi-level feature construction approach with respect to many state-of-art algorithms.

A good and healthy life is the basic right of every human being. To achieve that, it is needed to make available better and cheaper medical facilities to everyone. There are a lot of disease a human life suffers with. Our focus in this paper is to study and analyze various classifiers algorithms to predict various liver diseases. We have compared the performance of Random Forest, Logistic Regression, Support Vector Machine, Decision Tree, Reduction Rule, Naive Bayes, and K-Nearest Neighbors classification algorithms. We have utilized an Irvine database of the University of California, namely the Indian Liver Patient Dataset (ILPD). The various characteristics of a patient such as gender, age, total bilirubins, direct bilirubins, albumins, globulin ratios, alkphos, sgot and sgpt were evaluated. The implementation of classification algorithms is done on the rapid miner tool along with Liver Patient dataset and comparison is done on the parameters such as training time, scoring time, accuracy, precision. The result of other parameters like classification error, recall is also compared on liver patient diseases dataset. We also proposed the use of a radial kernel function as a support vector machine, and it has observed that true prediction of class1 is about 81.30% having class recall is 96.15% and class2 is about 82.22% having class recall is 44.31%.

Feature construction and feature selection are common pre-processing techniques to obtain smaller but better discriminating feature sets than the original ones. These two techniques are essential in high-dimensional data with thousands or tens of thousands of features where there may exist many irrelevant and redundant features. Genetic programming (GP) is a powerful technique that has shown promising results in feature construction and feature selection. However, constructing multiple features for high-dimensional data is still challenging due to its large search space. In this paper, we propose a GP-based method that simultaneously performs multiple feature construction and feature selection to automatically transform high-dimensional datasets into much smaller ones. Experiment results on six datasets show that the size of the generated feature set is less than 4% of the original feature set size and it significantly improves the performance of K-Nearest Neighbour, Naive Bayes and Decision Tree algorithms on 15 out of 18 comparisons. Compared with the single feature construction method using GP, the proposed method has better performance on half cases and similar on the other half. Comparisons between the constructed features, the selected features and the combination of both constructed and selected features by the propose method reveal different preferences of the three learning algorithms on these feature sets.

Credit decision-making is a vital process in the banking sector as it helps to reduce losses by identifying non-creditable individuals. Classification algorithms in data mining provide accurate results in the aforementioned area. But, such real-world lending environments require classification results to be easy to interpret. The lack of explicability of several existing classifiers makes banks reluctant in using them. An ideal classifier needs to be accurate with interpretability encapsulated within it. Decision trees are accurate, but for large datasets, the tree becomes very large and may not be comprehensible. Genetic programming (GP) is widely applied for solving classification problems since it can produce smaller trees by using tree-size, as fitness measure or by depth-limiting the trees. Hence, we propose an algorithm named GPeCT that merges decision tree and GP to produce a near-optimal decision tree classifier. We demonstrate the performance of GPeCT through experiments on large datasets from banking and other domains.

This paper describes an approach to the use of genetic programming (GP) for the automatic detection of rhythmic stress in spoken New Zealand English. A linear-structured GP system uses speaker independent prosodic features and vowel quality features as terminals to classify each vowel segment as stressed or unstressed. Error rate is used as the fitness function. In addition to the standard four arithmetic operators, this approach also uses several other arithmetic, trigonometric, and conditional functions in the function set. The approach is evaluated on 60 female adult utterances with 703 vowels and a maximum accuracy of 92.61% is achieved. The approach is compared with decision trees (DT) and support vector machines (SVM). The results suggest that, on our data set, GP outperforms DT and SVM for stress detection, and GP has stronger automatic feature selection capability than DT and SVM.KeywordsSupport Vector MachineProsodic FeatureLexical StressVowel QualityGenetic Programming SystemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This study presents a passive model to detect islanding conditions of synchronous distributed generation resources in a distribution network or a microgrid. The proposed approach uses the classification and regression tree algorithm for distinguishing between islanding and non‐islanding situations. It utilises the rate of change of frequency (ROCOF) and harmonic content of the equivalent reactance seen at the location of distributed generation as input features for decision tree construction. Indeed the thresholds of the proposed input features are extracted by the decision tree algorithm. The output if‐then rules of the decision tree algorithm are then utilised to make a new relay logic for islanding detection. The proposed scheme successfully classifies the input data with approximately zero non‐detection zone and prevent false operation of traditional scheme. The performance of the proposed protective algorithm is compared with the traditional ROCOF relay, artificial neural network and support vector machine techniques. The proposed method is implemented in a typical microgrid, which is connected to the transmission network of the dynamic WSCC three‐machine, 9bus test system with considering electric faults at upstream transmission and sub‐transmission networks.

Classification is an important task in the field of machine learning. Decision tree algorithms are a popular choice for handling classification tasks due to their high accuracy, simple algorithmic process, and good interpretability. Traditional decision tree algorithms, such as ID3, C4.5, and CART, differ primarily in their criteria for splitting trees. Shannon entropy, Gini index, and mean squared error are all examples of measures that can be used as splitting criteria. However, their performance varies on different datasets, making it difficult to determine the optimal splitting criterion. As a result, the algorithms lack flexibility. In this paper, we introduce the concept of generalized entropy from information theory, which unifies many splitting criteria under one free parameter, as the split criterion for decision trees. We propose a new decision tree algorithm called RSE (RS-Entropy decision tree). Additionally, we improve upon a two-term information measure method by incorporating penalty terms and coefficients into the split criterion, leading to a new decision tree algorithm called RSEIM (RS-Entropy Information Method). In theory, the improved algorithms RSE and RSEIM are more flexible due to the presence of multiple free parameters. In experiments conducted on several datasets, using genetic algorithms to optimize the parameters, our proposed RSE and RSEIM methods significantly outperform traditional decision tree methods in terms of classification accuracy without increasing the complexity of the resulting trees.

Evidence for variable selection bias in classification tree algorithms based on the Gini Index is reviewed from the literature and embedded into a broader explanatory scheme: Variable selection bias in classification tree algorithms based on the Gini Index can be caused not only by the statistical effect of multiple comparisons, but also by an increasing estimation bias and variance of the splitting criterion when plug-in estimates of entropy measures like the Gini Index are employed. The relevance of these sources of variable selection bias in the different simulation study designs is examined. Variable selection bias due to the explored sources applies to all classification tree algorithms based on empirical entropy measures like the Gini Index, Deviance and Information Gain, and to both binary and multiway splitting algorithms.

Many real-world problems have an uneven distribution of data over different classes. The imbalanced nature of data impacts the performance of classifiers. The higher counts of majority class samples influence the learning abilities of well-known classifiers. Genetic programming (GP) algorithm based on natural evolution also impacts if the data’s nature is imbalanced. The fitness function plays a pivotal role and impacts almost each building block of the GP framework. GP with the standard fitness function produces under-fitted and biased classifiers. Therefore, this paper has proposed a new fitness function in GP to classify the imbalanced data. The proposed method is used to classify nine imbalanced problems: ABL-18, ABL-9-18, BAL, YEAST2, YEAST1, ABL-9, ION, WDBC, and SPECT. The imbalanced factor of benchmark problems varies from 99:1 to 59:41. The proposed method’s performance is compared with K-Nearest-Neighbourhood (KNN) and the standard fitness function-based GP methods. The GP with newly proposed fitness function gives average AUC values for ABL-18(99:1), ABL-9-18(94:6), BAL(92:8), YEAST2(89:11), YEAST1(84:16), ABL-9(83:17), ION(64:36), WDBC(63:37), and SPECT(59:41) as 0.714, 0.812, 0.975, 0.916, 0.768, 0.654, 0.872, 0.939, and 0.704, respectively, which are higher than KNN and the standard fitness function-based GP methods. The result outcomes prove the superiority of the proposed method.

The aim of this study was to design an expert system to predict the Non-Dipping or Dipping pattern by using several basic clinical and laboratory data through an artificial intelligence algorithm. Data Mining is a technique which extracts information from data sets by using a combination of both statistical analysis methods and artificial intelligence algorithms. Also in this study, the decision tree and naivebayes classification algorithms of this technique were used. Firstly, sixty-five patients (mean age 51±7 years, 40 females,) were included in the study. Systolic and diastolic dipping were found in 13 and 15 % of the patients, respectively. In the advancing process of the experiment, the number of instances were reduced, because of some missing data of the patients. The data sets were tested using the J48 decision tree algorithm. This classification algorithm was implemented on 56 instances, and also the number of attributes was reduced from 35 to 23. 66 % of the instances (37) were reserved for training and 44 % of the instances (19) were reserved for testing. When the algorithm was run, the Non-Dipper/Dipper pattern of the instances were correctly predicted in a rate of 73.6842 %. Model was built in 0.02 seconds. This pilot study shows that a machine learning algorithm can help in the prediction of diurnal blood pressure pattern relying on some basic demographic, clinical and laboratory data, with a reasonable accuracy.

Documents classification is from most important fields for Natural language processing and text mining. There are many algorithms can be used for this task. In this paper, focuses on improving Text Classification by feature selection. This means determine some of the original features without affecting the accuracy of the work, where our work is a new feature selection method was suggested which can be a general formulation and mathematical model of Recursive Feature Elimination (RFE). The used method was compared with other two well-known feature selection methods: Chi-square and threshold. The results proved that the new method is comparable with the other methods, The best results were 83% when 60% of features used, 82% when 40% of features used, and 82% when 20% of features used. The tests were done with the Naïve Bayes (NB) and decision tree (DT) classification algorithms , where the used dataset is a well-known English data set “20 newsgroups text” consists of approximately 18846 files. The results showed that our suggested feature selection method is comparable with standard Like Chi-square.

<p>Symbolic regression (SR) is a function identification process, the task of which is to identify and express the relationship between the input and output variables in mathematical models. SR is named to emphasise its ability to find the structure and coefficients of the model simultaneously. Genetic Programming (GP) is an attractive and powerful technique for SR, since it does not require any predefined model and has a flexible representation. However, GP based SR generally has a poor generalisation ability which degrades its reliability and hampers its applications to science and real-world modeling. Therefore, this thesis aims to develop new GP approaches to SR that evolve/learn models exhibiting good generalisation ability. This thesis develops a novel feature selection method in GP for high-dimensional SR. Feature selection can potentially contribute not only to improving the efficiency of learning algorithms but also to enhancing the generalisation ability. However, feature selection is seldom considered in GP for high-dimensional SR. The proposed new feature selection method utilises GP’s built-in feature selection ability and relies on permutation to detect the truly relevant features and discard irrelevant/noisy features. The results confirm the superiority of the proposed method over the other examined feature selection methods including random forests and decision trees on identifying the truly relevant features. Further analysis indicates that the models evolved by GP with the proposed feature selection method are more likely to contain only the truly relevant features and have better interpretability. To address the overfitting issue of GP when learning from a relatively small number of instances, this thesis proposes a new GP approach by incorporating structural risk minimisation (SRM), which is a framework to estimate the generalisation performance of models, into GP. The effectiveness of SRM highly depends on the accuracy of the Vapnik-Chervonenkis (VC) dimension measuring model complexity. This thesis significantly extends an experimental method (instead of theoretical estimation) to measure the VC-dimension of a mixture of linear and nonlinear regression models in GP for the first time. The experimental method has been conducted using uniform and non-uniform settings and provides reliable VC-dimension values. The results show that our methods have an impressively better generalisation gain and evolve more compact model, which have a much smaller behavioural difference from the target models than standard GP and GP with bootstrap, The proposed method using the optimised non-uniform setting further improves the one using the uniform setting. This thesis employs geometric semantic GP (GSGP) to tackle the unsatisfied generalisation performance of GP for SR when no overfitting occurs. It proposes three new angle-awareness driven geometric semantic operators (GSO) including selection, crossover and mutation to further explore the geometry of the semantic space to gain a greater generalisation improvement in GP for SR. The angle-awareness brings new geometric properties to these geometric operators, which are expected to provide a greater leverage for approximating the target semantics in each operation, and more importantly, to be resistant to overfitting. The results show that compared with two kinds of state-of-the-art GSOs, the proposed new GSOs not only drive the evolutionary process fitting the target semantics more efficiently but also significantly improve the generalisation performance. A further comparison on the evolved models shows that the new method generally produces simpler models with a much smaller size and containing important building blocks of the target models.</p>

The figure of merit (FOM) is popular for testing an edge detector's performance, but there are very few reports using FOM as an evaluation method in Genetic Programming (GP). In this study, FOM is investigated as a fitness function in GP for edge detection. Since FOM has some drawbacks from type II errors, new fitness functions are developed based on FOM in order to address these weaknesses. Experimental results show that FOM can be used to evolve GP edge detectors that perform better than the Sobel detector, and the new fitness functions clearly improve the ability of GP edge detectors to find edge points and give a single response on edges, compared with the fitness function using FOM.