Feature Selection for Cross-Scene Hyperspectral Image Classification via Improved Ant Colony Optimization Algorithm

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

Recently, information extraction from hyperspectral images (HI) has become an attractive research area for many practical applications in earth observation due to the fact that HI provides valuable information with a huge number of spectral bands. In order to process such a huge amount of data in an effective way, traditional methods may not fully provide a satisfactory performance because they do not mostly consider high dimensionality of the data which causes curse of dimensionality also known as Hughes phenomena. In case of supervised classification, a poor generalization performance is achieved as a consequence resulting in availability of limited training samples. Therefore, advance methods accounting for the high dimensionality need to be developed in order to get a good generalization capability. In this work, a method of High Dimensional Model Representation (HDMR) was utilized for dimensionality reduction, and a novel feature selection method was introduced based on global sensitivity analysis. Several implementations were conducted with hyperspectral images in comparison to state-of-art feature selection algorithms in terms of classification accuracy, and the results showed that the proposed method outperforms the other feature selection methods even with all considered classifiers, that are support vector machines, Bayes, and decision tree j48.

Sentiment classification or sentiment analysis has been acknowledged as an open research domain. In recent years, an enormous research work is being performed in these fields by applying various numbers of methodologies. Feature generation and selection are consequent for text mining as the high-dimensional feature set can affect the performance of sentiment analysis. This paper investigates the inability or incompetency of the widely used feature selection methods (IG, Chi-square, and Gini Index) with unigram and bigram feature set on four machine learning classification algorithms (MNB, SVM, KNN, and ME). The proposed methods are evaluated on the basis of three standard datasets, namely, IMDb movie review and electronics and kitchen product review dataset. Initially, unigram and bigram features are extracted by applying n-gram method. In addition, we generate a composite features vector CompUniBi (unigram + bigram), which is sent to the feature selection methods Information Gain (IG), Gini Index (GI), and Chi-square (CHI) to get an optimal feature subset by assigning a score to each of the features. These methods offer a ranking to the features depending on their score; thus a prominent feature vector (CompIG, CompGI, and CompCHI) can be generated easily for classification. Finally, the machine learning classifiers SVM, MNB, KNN, and ME used prominent feature vector for classifying the review document into either positive or negative. The performance of the algorithm is measured by evaluation methods such as precision, recall, and F-measure. Experimental results show that the composite feature vector achieved a better performance than unigram feature, which is encouraging as well as comparable to the related research. The best results were obtained from the combination of Information Gain with SVM in terms of highest accuracy.

A new hybrid filter/wrapper algorithm for feature selection in classification

Feature selection is the process of choosing, or removing, features to obtain the most informative feature subset of minimal size. Such subsets are used to improve performance of machine learning algorithms and enable human understanding of the results. Approaches to feature selection in literature exploit several optimization algorithms. Multi-objective methods also have been proposed, minimizing at the same time the number of features and the error. While most approaches assess error resorting to the average of a stochastic K-fold cross-validation, comparing averages might be misleading. In this paper, we show how feature subsets with different average error might in fact be non-separable when compared using a statistical test. Following this idea, clusters of non-separable optimal feature subsets are identified. The performance in feature selection can thus be evaluated by verifying how many of these optimal feature subsets an algorithm is able to identify. We thus propose a multi-objective optimization approach to feature selection, EvoFS, with the objectives to i. minimize feature subset size, ii. minimize test error on a 10-fold cross-validation using a specific classifier, iii. maximize the analysis of variance value of the lowest-performing feature in the set. Experiments on classification datasets whose feature subsets can be exhaustively evaluated show that our approach is able to always find the best feature subsets. Further experiments on a high-dimensional classification dataset, that cannot be exhaustively analyzed, show that our approach is able to find more optimal feature subsets than state-of-the-art feature selection algorithms.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

In order to improve the search result and low evolution speed, and avoid the tendency towards stagnation and falling into the local optimum of ant colony optimization(ACO) in solving the complex function, the traditional ant colony optimization algorithm is analyzed in detail, an improved ant colony optimization(IWSMACO) algorithm based on information weight factor and supervisory mechanism is proposed in this paper. In the proposed IWSMACO algorithm, the information weight factor is added to the path selection and pheromone adjustment mechanisms in order to dynamically adjust path selection probability and randomly select the behavior rules for further intelligentializing the ant colony. The supervisory mechanism added the dynamic convergence criterion of supervisory distance and used the optimal pheromone update strategy to self-adaptively select the excellent ants for updating the pheromone trails, and improve the solution qualities of each iteration, better guide the later ants for learning. Finally, the proposed IWSMACO algorithm is carried out by 12 TSP instances. The simulation experiment results show that the proposed IWSMACO algorithm can not only avoid falling into the local optimum, but also enhance the convergence speed. And it takes on remarkable optimized ability and higher search accuracy.

Mutual Information (MI) is a good selector of relevance between input and output feature and have been used as a measure for ranking features in several feature selection methods. Theses methods cannot estimate optimal feature subsets by themselves, but depend on user defined performance. In this paper, we propose estimation of optimal feature subsets by using rough sets to determine candidate feature subset which receives from MI feature selector. The experiment shows that we can correct nonlinear problems and problems in situation of two or more combined features are dominant features, maintain an improve classification accuracy.KeywordsMutual InformationRough setsNonlinear ProblemsMLPC4.5

A Gaussian process embedded feature selection method based on automatic relevance determination

The paper proposes a hybrid approach to reduce dimension in text classification problems, to overcome the issue of Curse of Dimensionality. This hybrid approach is a combination of Feature Selection (FS) and Feature Extraction (FE) methods, considering different aspects of feature relevance, to effectively reduce the dimension in large text datasets. It prevents feature selection biased in favor of a particular FS method. Many FS methods like Term Variance, Document Frequency, Information Gain, Shannons Entropy measure, Mean-Median and Mean Absolute Difference, were implemented and a comparative study was made on their performance when implemented in a hybrid system. The features selected by the individual FS methods are merged using three approaches, namely, Union, Intersection and Modified Union. The sub lists of features further undergo Feature Extraction by PCA, and the reduced feature sub list is clustered with k-means. Finally, the sentiment-score of the individual clusters are calculated using SentiWordNet database which gives the polarity of the data. The experiments were conducted on the benchmark datasets namely Reuters-21,578 and Classic4. The performance evaluation of the system made using the measures like precision, recall, f-score and accuracy shows that the proposed method has improved performance compared to its competitive methods.

An innovative methodology is proposed to identify potential risk factors and possible accident escalation consequences, and to determine the evolution of an accident from cause to consequence, thereby to identify the most probable path and discover key risk factors along the path rapidly. Based on the principle of a directed weighted complex network (DWCN), the bow-tie (BT) model, risk entropy and the improved ant colony optimization (IACO) algorithm are integrated into this methodology. First, the qualitative analysis of risk evolution based on the BT model is carried out. The evolution development based on accident suppression can be divided into two stages: accident precursor stage and accident evolution stage. Then, a new method for mapping BT into DWCN is proposed. Lastly, the shortest path analysis of risk evolution based on the IACO algorithm is carried out, fuzzy set theory (FST) is introduced to calculate the failure probability of risk factors, and risk entropy is used to represent the uncertainty of risk propagation. Thus, the IACO algorithm can be used to calculate the shortest path of risk evolution. The proposed method is applied to oil and gas leakages in the FPSO oil and gas processing system. The results show that it is an effective method to identify the shortest evolution path and the most vulnerable risk factors.

In order to solve the problem of face recognition, the method of feature extraction and feature selection is presented in this paper. First using Gabor filters and face image as the convolution Operator to extract the Gabor feature vector of the image and also to uniform sampling; then using the PCA + LDA method to reduce the dimension for high-dimensional Gabor feature vector; Finally, using the nearest neighbor classifier to discriminate and determine the identity of a face image. The result I get is that the sampled Gabor feature in high-dimensional space can be projected onto low-dimensional space though the method of feature selection and compression. The new and original in this paper is that the method of PCA + LDA overcomes the problem of the spread matrix singular in the class and matrix too large which is brought by directly use the LDA.

To achieve efficient search performance for the multi-robot system which carries out the goal search task with consideration of timeliness, a multi-robot collaborative path planning system is designed to guide the robots during the search process. In the system, a planning method based on an Improved Ant Colony Optimization (IACO) algorithm is proposed. In the solution procedure, the path cost and goal timeliness are taken as two optimization goals. Compared to the traditional optimization algorithm, the IACO algorithm has global superiority which brings about better solution. Experiment results show that all robots accomplish the search task safely and efficiently by collaborative work using the proposed approach.

In the field of fault diagnosis, machine learning is highly valued for its broad applicability and efficiency. Feature extraction and feature selection are key steps in the application of machine learning, and the performance of fault diagnosis methods relies heavily on the effective execution of these two steps. For this reason, this paper aims to enhance the performance of fault diagnosis methods by improving these two aspects. Firstly, to address the non-linearity and non-stationarity of rotating machinery vibration signals under variable operation conditions, this paper proposes an improved rapid refined composite multiscale sample entropy (IR2CMSE) feature extraction method. In addition, this paper decomposes the vibration signals with improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) and extracts the sensitive intrinsic modal functions’ (IMFs') IR2CMSE values (SI-IR2CMSE) as the initial feature vector, which more accurately reveals the intrinsic time-scale characteristics of the vibration signals. Secondly, to address the problem of over-reliance on sample labels in most feature selection methods, this paper proposes a semi-supervised Gaussian mixing model with sparse regularization (SSGMM-SR) feature selection model. The model does not require complete fault labels and can automatically identify important features. Finally, validation with two rotating machinery fault datasets shows that the method proposed in this study exhibits high diagnostic accuracy and stability across multiple classifiers.

Wireless sensor networks with node calculation ability is weak, node power (battery provides) Limited, routing algorithm of wireless sensor networks should be the energy saving and simple algorithm. Ant colony algorithm shows good search results in the discrete solution space. This paper improves the ant colony optimization algorithm. The paper presents analysis of routing protocols for wireless sensor networks based on improved ant colony optimization algorithm, and the simulation results verify the effectiveness of the improved algorithm.