Local sparse discriminative feature selection

LSFSR: Local label correlation-based sparse multilabel feature selection with feature redundancy

Image is an important class of matrix structured data. In practical application scenarios, local features of data often contain more discriminative information. Considering that the existing sparse feature selection algorithms rarely consider the local structure information contained in image data, this paper proposes a sparse feature selection algorithm for image data based on local feature based sparse linear discriminant analysis (LFSLDA). Firstly, the scale-invariant feature transform (SIFT) local feature descriptor matrix is used to represent the original image matrix data, and then row and column transformation are carried out respectively. In order to better select the features with more discriminant information and no correlation between each other, the algorithm introduces the generalized uncorrelation constraint and applies it to the total divergence matrix, which not only avoids the singularity problem, but also preserves the structure of the data. Experimental results show that this algorithm can make better use of the local feature information of matrix data and have better performance than other related algorithms on six image data sets.

Efficient nonconvex sparse group feature selection via continuous and discrete optimization

This paper addresses the problem of feature selection for the classification of ground penetrating radar signals. We propose a new classification approach based on time-frequency analysis and sparse kernel feature selection. In the proposed approach, a time-frequency or a time-scale transform is first applied to the one-dimensional radar trace. Sparse kernel feature selection is then employed to extract an optimum set of features for classification. The sparse kernel method is formulated as an underdetermined linear system in a high-dimensional space, and the category labels of the training samples are used as measurements to select the most informative features. The proposed approach is evaluated through an industrial application of assessing railway ballast fouling conditions. Experimental results show that the proposed combination of sparse kernel feature selection and support vector machine classification yields very high classification rates using only a small number of features.

A robust graph-based semi-supervised sparse feature selection method

Typical feature selection methods choose an optimal global feature subset that is applied over all regions of the sample space. In contrast, in this paper we propose a novel localized feature selection (LFS) approach whereby each region of the sample space is associated with its own distinct optimized feature set, which may vary both in membership and size across the sample space. This allows the feature set to optimally adapt to local variations in the sample space. An associated method for measuring the similarities of a query datum to each of the respective classes is also proposed. The proposed method makes no assumptions about the underlying structure of the samples; hence the method is insensitive to the distribution of the data over the sample space. The method is efficiently formulated as a linear programming optimization problem. Furthermore, we demonstrate the method is robust against the over-fitting problem. Experimental results on eleven synthetic and real-world data sets demonstrate the viability of the formulation and the effectiveness of the proposed algorithm. In addition we show several examples where localized feature selection produces better results than a global feature selection method.

SummarySoftware testing is the process of improving software quality by classifying and removing defects in the software development. Previously, several methods were used for software defect prediction, but any one method did not provide sufficient accuracy. To overcome this issue, an improved may fly optimization with deep stacked sparse auto encoder feature selection scorched gradient descent driven dropout extreme learning machine framework (SDP‐IMFOFS‐GDDDXLMC) is proposed in this article for software defect prediction. Here, an IMFO method is considered for feature selection techniques. In feature selection technique, features are selected, such as PC1, PC4, and MC1 for selecting the probable minimal attribute. Then, GDDDXLMC approach is used to classify buggy and clean software detection. The proposed approach is implemented in MATLAB platform. The performance metrics, such as accuracy, precision, F‐measure, sensitivity, specificity, Mathew correlation coefficient (MCC) is examined to examine the performance of the proposed method. The simulation performance of the proposed SDP‐IMFOFS‐GDDDXLMC method attains higher accuracy 99.75%, 97.85%, 95.13%, 14.89%, 16.34%, 17.89%, and 98.79, higher sensitivity 96.34%, 91.23%, 89.12%, 12.67%, 17.56%, 18.90%, and 87.25% higher specificity 14.89%, 16.89%, 20.67%, 93.67%, 92.37%, 98.47%, and 94.78% compared with the existing methods, like SDP‐MLP‐PSO, SDP‐BPNN‐RBFNN, SDP‐CNN‐RNN‐LSTM, SDP‐KBN‐LIME, SDP‐SLDeep‐LSTM, SDP‐K‐PCA‐ELM, and SDP‐CNN‐PHI forest, respectively.

Feature selection has achieved unprecedented success in obtaining sparse discriminative features. However, the existing methods almost use the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,p}$</tex-math></inline-formula> -norm constraint on transformation matrix to obtain sparse features, which introduces extra parameters and cannot obtain the features directly. In addition, existing algorithms only focused on the global structure and ignored the local structure, leading to poor performance when solving data with non-Gaussian distributions which a single center point cannot describe precisely. Based on above considerations, we propose a supervised feature selection via multi-center and local structure learning. We further introduce trace ratio criterion into our model in favor of improving the discriminant of features selected. In order to address the overlap problem, we use multiple center points to match the distribution of data and construct a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -Nearest Neighbor graph to explore the local structure of the data. In addition, we also propose an efficient method to optimize the transformation matrix with the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math></inline-formula> -norm constraint and can directly obtain the sparse features. We evaluate our method on Toy datasets and several real-world datasets, show improvement over state-of-the-art feature selection methods, and demonstrate the effectiveness of our model in dealing with non-Gaussian distributed data problems.

This paper solves the multi-class classification problem for Parkinson’s disease (PD) analysis by a sparse discriminative feature selection framework. Specifically, we propose a framework to construct a least square regression model based on the Fisher’s linear discriminant analysis (LDA) and locality preserving projection (LPP). This framework utilizes the global and local information to select the most relevant and discriminative features to boost classification performance. Differing in previous methods for binary classification, we perform a multi-class classification for PD diagnosis. Our proposed method is evaluated on the public available Parkinson’s progression markers initiative (PPMI) datasets. Extensive experimental results indicate that our proposed method identifies highly suitable regions for further PD analysis and diagnosis and outperforms state-of-the-art methods.

Typical unsupervised feature selection algorithms select a common feature subset for all the clusters. Consequently, clusters embedded in different feature subspaces are not discovered. In this paper, we propose a novel approach of simultaneous localized feature selection and model detection for unsupervised learning. In our approach, local feature saliency, together with other parameters of Gaussian mixtures, are estimated by Bayesian variational learning. Experiments performed on real-world datasets illustrate that our approach is superior over both global feature selection and subspace clustering methods.

We present an automatic method of designing correlation filters for pattern recognition that are composed of select local features (i.e., small parts of a reference object). The local features are selected for their ability to discriminate between the reference object and other known objects or patterns. In the basic localized feature selection problem, we design a correlation filter from a single optimal local feature. In the general localized feature selection problem, we design a correlation filter composed of several local features. We show that the discrimination ability of a correlation filter designed form properly selected local features is actually greater than the discrimination ability of a traditional matched filter.

Automatic age estimation relying on human facial images is a key technology of many real-world applications, which is still a challenging task in the computer vision field. There are three cascade modules for facial age estimation: facial aging feature extraction, dimension reduction (or feature selection) and estimation method. Many existing literatures focus on the first or last module while for an age estimation system, it's also important to construct a reasonable framework. Our work focuses on creating an effective framework by selecting methods for these modules reasonably. Firstly, a BIM (bio-inspired model) is employed to extract facial aging features because it can not only capture discriminative local and global features, but also overcome interferences of some 2D deformations to some extent. Then, LDA (linear discriminant analysis) is used for reducing the BIF (bio-inspired features) to lower dimensions and extracting more discriminative information at the same time. Finally, CS-OHRank (cost-sensitive ordinal hyperplane rank), which tackles with sparse data well and reflects the cumulative attributes of aging, is applied as the estimation method. Experimental results on benchmark dataset FG-NET show that our framework combining BIF, LDA and CS-OHRank is competitive among the state of the art, with MAE (mean absolute error) = 4.72 years..

Linear discriminant analysis (LDA) aims to find a low-dimensional space in which data points in the same class are to be close to each other while keeping data points from different classes apart. To improve the robustness of LDA to non-Gaussian distribution data, most existing discriminant analysis methods extend LDA by approximating the underlying manifold of data. However, these methods suffer from the following problems: 1) local affinity or reconstruction coefficients are learned on the basis of the relationships of all data pairs, which would lead to a sharp increase in the amount of computation and 2) they learn the manifold information in the original space, ignoring the interference of the noise and redundant features. Motivated by these challenges, this article represents a novel discriminant analysis model, called fast and adaptive locality discriminant analysis (FALDA), to improve the efficiency and robustness. First, with the anchor-based strategy, a bipartite graph of each class is constructed to characterize the local structure of data. Since the number of anchor points is far less than that of data points, learning of fuzzy membership relationships between data points and anchor points within each class can save training time. Second, a maximum entropy regularization is introduced to control the uniformity of the weights of graphs and avoid the trivial solution. Third, the above relationships are updated adaptively in the process of dimensionality reduction, which can suppress the interference of the noise and redundant features. Fourth, the whitening constraint is imposed on the projection matrix to remove the relevance between features and restrict the total scatter of data in the subspace. Last but not the least, data with complex distribution can be explicitly divided into sub-blocks according to the learned anchor points (or subclass center points). We test our proposed method on synthetic data, benchmark datasets, and imbalanced datasets. Promising experimental results demonstrate the success of this novel model.

Classical Linear Discriminant Analysis (LDA) is usually suffers from the small sample size (SSS) problem when dealing with the high dimensional face data. Many methods have been proposed for solving this problem such as Fisherface and Null Space LDA (N-LDA), but these methods are overfitted to the training set and inevitably lose some useful discriminative information in many cases. To effectively utilize nearly all useful discriminative information, a not completely random sampling framework for the integration of multiple features is developed. However, this method has the following main disadvantage: By directly employing feature extraction, the newly constructed variables may contain lots of information originated from those redundant features in the original space. So, in this paper, we introduce a new random sampling LDA by incorporating feature selection for face recognition, that is, some redundant features are removed using the given feature selection methods at first, and then PCA is employed, finally we use random sampling to generate multiple feature subsets. Along this, corresponding weak LDA classifiers are naturally generated and an integrated classifier is developed using a fusion rule. Experiments on 4 face datasets (AR, ORL, Yale, YaleB) show the effectiveness of our algorithm.

Network Intrusion Detection Systems (NIDSs) are an increasingly important tool for the prevention and mitigation of cyber attacks. Over the past years, a lot of research efforts have aimed at leveraging the increasingly powerful models of Machine Learning (ML) for this purpose. A number of labelled synthetic datasets have been generated and made publicly available by researchers, and they have become the benchmarks via which new ML-based NIDS classifiers are being evaluated. Recently published results show excellent classification performance with these datasets, increasingly approaching 100 percent performance across key evaluation metrics such as Accuracy, F1 score, AUC, etc. Unfortunately, we have not yet seen these excellent academic research results translated into practical NIDS systems with such near-perfect performance. This motivated our research presented in this paper, where we analyse the statistical properties of the benign traffic in three of the more recent and relevant NIDS datasets, (CIC_IDS, UNSW_NB15, TON_IOT), by converting them into a common flow format. As a comparison, we consider two datasets obtained from real-world production networks, one from a university network and one from a medium size Internet Service Provider (ISP). Our results show that the two real-world datasets are quite similar among themselves in regards to most of the considered statistical features. Equally, the three synthetic datasets are also relatively similar within their group. However, and most importantly, our results show a distinct difference of most of the considered statistical features between the three synthetic datasets and the two real-world datasets. Since ML relies on the basic assumption of training and test datasets being sampled from the same distribution, this raises the question of how well the performance results of ML-classifiers trained on the considered synthetic datasets can translate and generalise to real-world networks. We believe this is an interesting and relevant question which provides motivation for further research in this space.