A new feature selection using dynamic interaction

Feature selection is referred to the process of obtaining a subset from an original feature set according to certain feature selection criterion, which selects the relevant features of the dataset. It plays a role in compressing the data processing scale, where the redundant and irrelevant features are removed. Feature selection techniques show that more information is not always good in machine learning applications. Apply different algorithms for the data at hand and with baseline classification performance values we can select a final feature selection algorithm. In this paper, we propose a hybrid classification model, which has correlation based filter feature selection algorithm and Machine learning as classifiers. The objective of this study is to select relevant features and analyze the outperform machine learning algorithms in order to train our model, predict and compare their classification performance. In this method, features are ordered according to their Absolute correlation value with respect to the class attribute. Then top K Features are selected from ordered list of features to form a reduced dataset. This proposed classifier model is applied to our smart meter datasets. To measure the performance of these selected features; seven benchmark classifier are used; Random Forest (RF), Logistic Regression (LR), k-Nearest Neighbor (kNN), Naive Bayes (NB), Decision Tree (DT), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). This paper then analyzes the performance of all classifiers with feature selection in term of accuracy, sensitivity, F-Measure, Specificity, Precision, and MCC. From our experiment, we found that Random Forest classifier performed higher than other used classifiers.

Irrelevant feature in heart disease dataset affects the performance of binary classification model. Consequently, eliminating irrelevant and redundant feature (s) from training set with feature selection algorithm significantly improves the performance of classification model on heart disease detection. Sequential feature selection (SFS) is successful algorithm to improve the performance of classification model on heart disease detection and reduces the computational time complexity. In this study, sequential feature selection (SFS) algorithm is implemented for improving the classifier performance on heart disease detection by removing irrelevant features and training a model on optimal features. Furthermore, exhaustive and permutation based feature selection algorithm are implemented and compared with SFS algorithm. The implemented and existing feature selection algorithms are evaluated using real world Pima Indian heart disease dataset and result appears to prove that the SFS algorithm outperforms as compared to exhaustive and permutation based feature selection algorithm. Overall, the result looks promising and more effective heart disease detection model is developed with accuracy of 99.3%.

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

A hybrid ensemble-filter wrapper feature selection approach for medical data classification

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

This paper initiates perceptions and algorithms of feature selection, survey of existing feature selection algorithms and assesses diverse algorithms with a classifying frame based on search approaches, valuation criteria, and provides strategy in selecting feature selection algorithms. A unifying platform is projected to continue our efforts headed for building an incorporated system for intelligent feature selection. Feature selection intends to reduce the dimensionality of patterns for classification by choosing the most informative instead of irrelevant and/or redundant features. In this proposed work Intuitionistic fuzzy based feature clustering is proposed for grouping features based on the degree of membership and degree of indeterminacy among the attributes and clusters. In this proposed work a novel approach which uses an Intuitionistic fuzzy version of k-means has been introduced for grouping interdependent features. Genetic algorithm reinstating which is the variation of traditional genetic algorithm is then applied to appraise whether the measured feature is independent of class labels; hence, it leads to eliminate unrelated clusters to classification process and progress the selection of features. The proposed method achieves improvement on classification accuracy and perhaps to select less number of features which show the way to simplification of learning task to a big extent. The Experiment results have been demonstrated by the good performance and also find good enough subset features of this method on using UCI benchmark datasets that are for data mining methods such as Breast Cancer, Sensor and Iris Records.

Software effort estimation is a vital process in the software industry for successfully administering 5Ds of the software development life cycle (SDLC). The 5Ds stand for demand, development, direction, deployment, and designated cost of the software. Software development effort estimation (SDEE) is an effort prediction mechanism to calculate the effort for the development of the software product in order to minimize the challenges in the software field. Academics and practitioners are striving to identify which machine learning estimation technique yields more accurate results based on evaluation metrics, datasets, and other pertinent aspects. The feature selection techniques impact accuracy by selecting the main and relevant features in the dataset and eliminating the redundant and irrelevant features in the dataset. To achieve accurate estimations, the paper utilizes feature selection algorithms, along with various machine learning techniques, which predict the desired effort and the performance of the model has been measured in terms of prediction accuracy, value, relative error, and mean absolute error. The datasets China and Maxwell are trained with the relevant features by applying feature selection algorithms, and estimation techniques are applied to predict the effort. The performance is compared with the regression models and feature selection techniques utilized by many authors previously. The result of the proposed methodology significantly gives the best performance with the combination of feature selection and estimation models than all regression models when applied alone, to both datasets. From the results, it is perceptible that random forest is performing well with the feature selection techniques and obtains the highest prediction accuracy of 99.33% with the China and 89.47% with the Maxwell datasets.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

A data-driven method approach for prediction of coal seam gas content combining feature selection and machine learning

The intrusion detection system has been widely studied and deployed by researchers for providing better security to computer networks. The increasing volume of attacks, com-bined with the rapid improvement of machine learning (ML) has made the collaboration of intrusion detection techniques with machine learning and deep learnings are a popular subject and a feasible approach for cyber threat protection. Machine learning usually involves the training process using huge sample data. Since the huge input data may cause a negative effect on the training and detection performance of the machine learning model, feature selection becomes a crucial technique to rule out the irrelevant and redundant features from the dataset. This study applied a feature selection approach for intrusion detection that incorporated state-of-the-art feature selection algorithms with attack characteristic feature to produce an optimized set of features for the machine learning algorithms, which was then used to train the machine learning model. CSECIC- IDS2018 dataset, the most recent benchmark dataset with a wide attack diversity and features have been used to create the efficient feature subset. The result of the experiment was produced using machine learning models with a decision tree classifier and analyzed with respect to the accuracy, precision, recall, and f1 score.

Correlation based feature selection with clustering for high dimensional data

Feature selection plays an important role in big data mining and pattern recognition, which includes selecting a subset of the most informative features that produces compatible results as the original entire set of features. A new similarity measure is proposed in terms of information theory, in particular joint mutual information between features with respect to class labels. Based on the similarity measure, an adaptive spectral clustering that could determine cluster number adaptively is proposed. Furthermore, we propose a new feature selection algorithm called Adaptive Spectral Clustering based on Joint Mutual Information (ASC-JMI) which can effectively and efficiently deal with both irrelevant and redundant features, and select a high-quality feature subset. The experimental results on five high-dimensional benchmark datasets demonstrate that the ASC-JMI is effective in selecting the informative features and obtains competitive classification performance.

Feature selection is an important pre-processing step in many fields, such as data mining, machine learning and pattern recognition. This paper focuses on dynamically updating a subset of features with new samples arriving and provides a hypergraph model to deal with dynamic feature selection problem. Firstly, we discuss the relationship between feature selection of information system and minimum vertex cover of hypergraph, and feature selection is converted to a minimum vertex cover problem based on this relationship. Then, an algorithm for generating induced hypergraph from information system is presented, the induced hypergraph can be divided into two part: the original induced hypergraph and the added hypergraph with new samples arriving. Finally, a novel dynamic feature selection algorithm based on minimum vertex cover of hypergraph is proposed, and this algorithm only needs a small amount of computation. Experiments show that the proposed method is feasible and highly effective.

An enhanced and efficient approach for feature selection for chronic human disease prediction: A breast cancer study

Many feature selection algorithms have been proposed in the past focusing on improving classification accuracy. In this work, we point out the importance of stable feature selection for knowledge discovery from high-dimensional data, and identify two causes of instability of feature selection algorithms: selection of a minimum subset without redundant features and small sample size. We propose a general framework for stable feature selection which emphasizes both good generalization and stability of feature selection results. The framework identifies dense feature groups based on kernel density estimation and treats features in each dense group as a coherent entity for feature selection. An efficient algorithm DRAGS (Dense Relevant Attribute Group Selector) is developed under this framework. We also introduce a general measure for assessing the stability of feature selection algorithms. Our empirical study based on microarray data verifies that dense feature groups remain stable under random sample hold out, and the DRAGS algorithm is effective in identifying a set of feature groups which exhibit both high classification accuracy and stability.