Feature selection based on multiple correlation measures for medical examination dataset

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

<p>Classification problems often have a large number of features, but not all of them are useful for classification. Irrelevant and redundant features may even reduce the classification accuracy. Feature selection is a process of selecting a subset of relevant features, which can decrease the dimensionality, shorten the running time, and/or improve the classification accuracy. There are two types of feature selection approaches, i.e. wrapper and filter approaches. Their main difference is that wrappers use a classification algorithm to evaluate the goodness of the features during the feature selection process while filters are independent of any classification algorithm. Feature selection is a difficult task because of feature interactions and the large search space. Existing feature selection methods suffer from different problems, such as stagnation in local optima and high computational cost. Evolutionary computation (EC) techniques are well-known global search algorithms. Particle swarm optimisation (PSO) is an EC technique that is computationally less expensive and can converge faster than other methods. PSO has been successfully applied to many areas, but its potential for feature selection has not been fully investigated. The overall goal of this thesis is to investigate and improve the capability of PSO for feature selection to select a smaller number of features and achieve similar or better classification performance than using all features. This thesis investigates the use of PSO for both wrapper and filter, and for both single objective and multi-objective feature selection, and also investigates the differences between wrappers and filters. This thesis proposes a new PSO based wrapper, single objective feature selection approach by developing new initialisation and updating mechanisms. The results show that by considering the number of features in the initialisation and updating procedures, the new algorithm can improve the classification performance, reduce the number of features and decrease computational time. This thesis develops the first PSO based wrapper multi-objective feature selection approach, which aims to maximise the classification accuracy and simultaneously minimise the number of features. The results show that the proposed multi-objective algorithm can obtain more and better feature subsets than single objective algorithms, and outperform other well-known EC based multi-objective feature selection algorithms. This thesis develops a filter, single objective feature selection approach based on PSO and information theory. Two measures are proposed to evaluate the relevance of the selected features based on each pair of features and a group of features, respectively. The results show that PSO and information based algorithms can successfully address feature selection tasks. The group based method achieves higher classification accuracies, but the pair based method is faster and selects smaller feature subsets. This thesis proposes the first PSO based multi-objective filter feature selection approach using information based measures. This work is also the first work using other two well-known multi-objective EC algorithms in filter feature selection, which are also used to compare the performance of the PSO based approach. The results show that the PSO based multiobjective filter approach can successfully address feature selection problems, outperform single objective filter algorithms and achieve better classification performance than other multi-objective algorithms. This thesis investigates the difference between wrapper and filter approaches in terms of the classification performance and computational time, and also examines the generality of wrappers. The results show that wrappers generally achieve better or similar classification performance than filters, but do not always need longer computational time than filters. The results also show that wrappers built with simple classification algorithms can be general to other classification algorithms.</p>

A simple and efficient filter feature selection method via document-term matrix unitization

Predictive modeling and data analysis are severely hampered by the enormous dimensionality and complexity of medical datasets. Enhancing model performance, interpretability, and computing efficiency all depend on careful feature selection. In order to extract the most pertinent characteristics from the Medical Information Mart for Intensive Care III (MIMIC-III) dataset, this paper offers a novel hybrid feature selection method that combines the Harvest Algorithm, Convolutional Neural Networks (CNN), and Correlation-Based Feature Selection (CFS). De-identified medical records, including diagnostic codes, vital signs, prescriptions, and other clinical observations, are all included in the MIMIC-III dataset. Our suggested approach makes use of CNN for deep feature extraction, the Harvest Algorithm for the first feature subset creation, and CFS for the final feature selection based on correlation measures. Test findings show that when compared to conventional feature selection techniques Random Forest with Information Gain Method (RF-IG) and SVM with Recursive Feature Elimination (SVM-RFE), our hybrid strategy greatly increases the accuracy and efficiency of prediction models. The chosen characteristics demonstrate the potential of our approach in clinical decision support and medical data analysis by offering significant insights into important variables influencing patient outcomes.

In many big data applications, online streaming feature selection plays a critical role in processing feature stream and dealing with high-dimensional problems. However, traditional online streaming feature selection methods focus on relevant features, irrelevant and/or redundant features, ignore the interaction between features. i.e., individual feature and label are irrelevant or weakly correlated, but when it is combined with another irrelevant or weakly feature, they show strongly correlated with label. In this paper, we propose a novel feature selection algorithm that considers feature interaction based on neighborhood rough set. This algorithm select features based on the following principles: the discrimination capability of the selected feature subset should be greater than or equal to the original feature space, and the number of features subset should be as small as possible by using feature interaction. Under this framework, we propose an online significance analysis criterion to select significance features relative to the currently selected features, and design an online redundancy analysis criterion to retain highly interactive features and filter out redundant features. Experimental results on a series of benchmark datasets show that the proposed algorithm significantly outperforms other state-of-the-art online streaming feature selection methods.

A hybrid ensemble-filter wrapper feature selection approach for medical data classification

Target spectrum based feature selection (TSFS): A new method based on chain coding for target detection problems

The feature selection (FS) plays a important role in identification of the significant genes in bioinformatics and related fields. Additionally, it is frequently necessary step to avoid over-fitting and to reduce complexity and computational time. Wang et al [1] proposed new two stage feature selection method achieving excellent classification performance while selecting only few relevant genes. We present new feature selection methods, based on the idea of the Wang's paper, and analyze how the particular filter FS method, used in first stage, influence overall performance. The performance is analyzed by means of the FS stability and influence on the prediction performance. Our results indicate that the stability of FS is significantly affected by the type of FS used in the first stage, but the prediction performance is not so sensitive to the choice of FS in the first stage.

On the benefit of feature selection and ensemble feature selection for fuzzy k-nearest neighbor classification

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

DNA microarrays can simultaneously measure the expression level of thousands of gene within a particular mRNA sample that provide information about the state of cells and tissues. Though these expressive values are useful in cancer classification, and understand the mechanisms involved in the genesis of disease processes, however, only a few genes out of these thousands of genes contribute towards disease classification. On this basis, usage of feature selection algorithm is favourable, as the main goal of feature selection algorithm is to identify the relevant features (here genes) efficiently. In this paper, we have applied four filter Feature Selection (FS) methods, namely, Mutual Information (MI), Pearson Correlation Coefficient (PCC), Chi <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , ReliefF along with three well-known classifiers, namely, Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbour (KNN) on six microarray datasets (both binary and multi-class) namely, Leukemia, Lung, Lymphoma and Leukemia, Gastric, SRBCT and Childhood Tumor and recorded the accuracies.

Advancements in medical technology have created numerous large datasets including many features. Usually, all captured features are not necessary, and there are redundant and irrelevant features, which reduce the performance of algorithms. To tackle this challenge, many metaheuristic algorithms are used to select effective features. However, most of them are not effective and scalable enough to select effective features from large medical datasets as well as small ones. Therefore, in this paper, a binary moth-flame optimization (B-MFO) is proposed to select effective features from small and large medical datasets. Three categories of B-MFO were developed using S-shaped, V-shaped, and U-shaped transfer functions to convert the canonical MFO from continuous to binary. These categories of B-MFO were evaluated on seven medical datasets and the results were compared with four well-known binary metaheuristic optimization algorithms: BPSO, bGWO, BDA, and BSSA. In addition, the convergence behavior of the B-MFO and comparative algorithms were assessed, and the results were statistically analyzed using the Friedman test. The experimental results demonstrate a superior performance of B-MFO in solving the feature selection problem for different medical datasets compared to other comparative algorithms.

A Comprehensive Survey on Metaheuristic Algorithm for Feature Selection Techniques

Nowadays, mobile robot navigation is a crucial topic in the Industrial Internet of Things, and a number of sensors are arranged around robots to avoid obstacles on navigation paths. Obtaining optimal sensor readings that can be used to optimize the path planning of mobile robots is essential. Thus, an effective feature selection technique is explored in this study to select the optimal subset of sensor readings for mobile robot navigation. Feature selection can reasonably remove irrelevant and redundant features, reduce the dimensionality of data, and improve the learning accuracy and comprehensibility. A novel feature selection method based on 3D mutual information is studied. First, the proposed method defines symmetrical uncertainty with 3D mutual information to measure the correlation between candidate features and select features. Afterward, a merit function of feature sets is defined based on the symmetrical uncertainty to search for the optimal feature subset. Lastly, the proposed feature selection method is applied to a wall‐following robot navigation data set. Results show that the proposed method can obtain few sensor readings but with enhanced prediction accuracy of the robot's movements.

A feature selection method via relevant-redundant weight