Local structured feature learning with dynamic maximum entropy graph

Linear discriminant analysis (LDA), as an effective feature extraction method, has been widely applied in high-dimensional data analysis. However, its discriminative performance is still severely limited by the following factors. First, the restriction on the total number of features available from LDA has seriously limited its application to problems where the feature dimension is much larger than the number of classes. Second, LDA cannot deal with data containing multiple clusters (or subclasses) within a class because it cannot correctly depict the local structure of the data. To alleviate this issue, we propose a locality-preserving multiprojection discriminant analysis (LPMDA) model to extract more discriminative features preserving local structure. Specifically, LPMDA rephrases the objective function of LDA as a convex discriminant analysis framework from the perspective of metric learning, allowing for extracting more features than the number of classes. Furthermore, an auto-optimized graph technique is also integrated into the discriminant analysis framework to explore the local structure of the data. An efficient iterative optimization algorithm is presented to solve LPMDA. Extensive experiments on several benchmark datasets confirm the effectiveness of the proposed method.

Linear discriminant analysis (LDA) aims to find a low-dimensional space in which data points in the same class are to be close to each other while keeping data points from different classes apart. To improve the robustness of LDA to non-Gaussian distribution data, most existing discriminant analysis methods extend LDA by approximating the underlying manifold of data. However, these methods suffer from the following problems: 1) local affinity or reconstruction coefficients are learned on the basis of the relationships of all data pairs, which would lead to a sharp increase in the amount of computation and 2) they learn the manifold information in the original space, ignoring the interference of the noise and redundant features. Motivated by these challenges, this article represents a novel discriminant analysis model, called fast and adaptive locality discriminant analysis (FALDA), to improve the efficiency and robustness. First, with the anchor-based strategy, a bipartite graph of each class is constructed to characterize the local structure of data. Since the number of anchor points is far less than that of data points, learning of fuzzy membership relationships between data points and anchor points within each class can save training time. Second, a maximum entropy regularization is introduced to control the uniformity of the weights of graphs and avoid the trivial solution. Third, the above relationships are updated adaptively in the process of dimensionality reduction, which can suppress the interference of the noise and redundant features. Fourth, the whitening constraint is imposed on the projection matrix to remove the relevance between features and restrict the total scatter of data in the subspace. Last but not the least, data with complex distribution can be explicitly divided into sub-blocks according to the learned anchor points (or subclass center points). We test our proposed method on synthetic data, benchmark datasets, and imbalanced datasets. Promising experimental results demonstrate the success of this novel model.

Adaptive fast local discriminant analysis with whitening transform.

In this article, we propose a novel feature selection approach via explicitly addressing the long-standing subspace sparsity issue. Leveraging l2,1 -norm regularization for feature selection is the major strategy in existing methods, which, however, confronts sparsity limitation and parameter-tuning trouble. To circumvent this problem, employing the l2,0 -norm constraint to improve the sparsity of the model has gained more attention recently whereas, optimizing the subspace sparsity constraint is still an unsolved problem, which only can acquire an approximate solution and without convergence proof. To address the above challenges, we innovatively propose a novel subspace sparsity discriminative feature selection (S2DFS) method which leverages a subspace sparsity constraint to avoid tuning parameters. In addition, the trace ratio formulated objective function extremely ensures the discriminability of selected features. Most important, an efficient iterative optimization algorithm is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To the best of our knowledge, such an optimization algorithm of solving the subspace sparsity issue is first proposed in this article, and a general formulation of the optimization algorithm is provided for improving the extensibility and portability of our method. Extensive experiments conducted on several high-dimensional text and image datasets demonstrate that the proposed method outperforms related state-of-the-art methods in pattern classification and image retrieval tasks.

Linear discriminant analysis (LDA) is a classical statistical machine-learning method, which aims to find a linear data transformation increasing class discrimination in an optimal discriminant subspace. Traditional LDA sets assumptions related to the Gaussian class distributions and single-label data annotations. In this article, we propose a new variant of LDA to be used in multilabel classification tasks for dimensionality reduction on original data to enhance the subsequent performance of any multilabel classifier. A probabilistic class saliency estimation approach is introduced for computing saliency-based weights for all instances. We use the weights to redefine the between-class and within-class scatter matrices needed for calculating the projection matrix. We formulate six different variants of the proposed saliency-based multilabel LDA (SMLDA) based on different prior information on the importance of each instance for their class(es) extracted from labels and features. Our experiments show that the proposed SMLDA leads to performance improvements in various multilabel classification problems compared to several competing dimensionality reduction methods.

Clustering is one of the most important techniques in the field of data mining. It partitions data points into different groups based on the similarity of information provided. Many clustering methods calculate similarity matrix before clustering as a separated step. They ignore the fact that embedded subspace can better reveal the spatial distribution of the original data since practical data always exist in a large scale. In this paper, we proposed a model called Adaptive Projected Clustering with Graph Regularization (APCGR) where the similarity matrix calculating and clustering process are conducted simultaneously. To achieve a desirable clustering result, we impose a rank constraint on the Laplacian matrix of the similarity matrix to ensure that the number of the connected components are exactly that same as the clustering number. The proposed clustering framework is indeed executed on an optimal subspace where more accurate and reasonable solutions can be provided. Augmented Lagrangian Method (ALM) will be applied in the optimizing procedure to effectively solve the potential problems and show the clustering results explicitly. Experimental results on both synthetic and benchmark datasets exhibit the superior performance of the proposed method.

Linear Discriminant Analysis (LDA) is a popular statistical method for both feature extraction and dimensionality reduction in face recognition. The major drawback of LDA is the so-called small sample size (3S) problem. This problem always occurs when the total number of training samples is smaller than the dimension of feature space. Under this situation, the within-class scatter matrix Sw becomes singular and LDA approach cannot be implemented directly. To overcome the 3S problem, this paper proposes a novel wavelet-face based subspace LDA algorithm. Wavelet-face feature extraction and dimensionality reduction are based on two-level D4-filter wavelet transform and discarding the null space of total class scatter matrix St. It is shown that our obtained projection matrix satisfies the uncorrelated constraint conditions. Hence in the sense of statistical uncorrelation, this projection matrix is optimal. The proposed method for face recognition has been evaluated with two public available databases, namely ORL and FERET databases. Comparing with existing LDA-based methods to solve the 3S problem, our method gives the best performance.

Feature extraction methods have been successfully applied to many real-world applications. The classical Linear Discriminant Analysis (LDA) and its variants are widely used as feature extraction methods. Although they have been used for different classification tasks, these methods have some shortcomings. The main one is that the projection axes obtained are not informative about the relevance of original features. In this paper, we propose a linear embedding method that merges two interesting properties: Robust LDA and inter-class sparsity. Furthermore, the targeted projection transformation focuses on the most discriminant original features, The proposed method is called Robust Discriminant Analysis with Feature Selection and Inter-class Sparsity (RDA FSIS). Two kinds of sparsity are explicitly included in the proposed model. The first kind is obtained by imposing the `2;1 constraint on the projection matrix in order to perform feature ranking. The second kind is obtained by imposing the inter-class sparsity constraint used for getting a common sparsity structure in each class. Comprehensive experiments on five real-world image datasets demonstrate the effectiveness and advantages of our framework over existing linear methods.

Dimension reduction of multimodal data by auto-weighted local discriminant analysis

Nowadays, multi-view dataset have become ubiquitous along with more and more data are gathered from different measuring technologies or various sources, in which various aspects of dataset are formalized as multiple views. Although a variety of multi-view clustering analysis approaches have been put forward to uncover the cluster structure hidden in the data, most of these existing methods are based on such a hypothesis: the relationship between objects and clusters are definite. However, most of the data in our real life may have no clear cluster boundaries but have indistinct or overlapping boundaries. How to effectively reveal the uncertain cluster structure under multiview data is still a big challenge for multi-view clustering analysis. Inspired by the powerful uncertain information modeling and analysis capabilities of rough and fuzzy sets, this paper proposes a new multi-view clustering method to discover the uncertain cluster information. A rough set concept based cluster centroid updating strategy is designed to efficiently describe the uncertain construction of clusters. A view weight is introduced to capture the different importance of various views. A fuzzy-based iterative optimization objective function is developed to fuse different view information. Finally, an efficient iterative optimization algorithm is devised to solve the proposed rough fuzzy objective function. Experiments on widely used benchmark datasets prove that our proposed method is always superior to several latest clustering approaches.

Nonnegative Matrix Factorization (NMF) is a widely used technique in many applications such as face recognition, motion segmentation, etc. It approximates the nonnegative data in an original high dimensional space with a linear representation in a low dimensional space by using the product of two nonnegative matrices. In many applications data are often partially corrupted with large additive noise. When the positions of noise are known, some existing variants of N-MF can be applied by treating these corrupted entries as missing values. However, the positions are often unknown in many real world applications, which prevents the usage of traditional NMF or other existing variants of NMF. This paper proposes a Robust Nonnegative Matrix Factorization (RobustNMF) algorithm that explicitly models the partial corruption as large additive noise without requiring the information of positions of noise. In particular, the proposed method jointly approximates the clean data matrix with the product of two nonnegative matrices and estimates the positions and values of outliers/noise. An efficient iterative optimization algorithm with a solid theoretical justification has been proposed to learn the desired matrix factorization. Experimental results demonstrate the advantages of the proposed algorithm.

An optimized feature extraction algorithm for abnormal network traffic detection

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and $\ell _{2,p}$ l 2 , p -norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach.

A comparison of supervised machine learning algorithms for mosquito identification from backscattered optical signals

The performance of supervised learning-based seganalysis depends on the choice of both classifier and features which represent the image. Features extracted from images may contain irrelevant and redundant features which makes them inefficient for machine learning. Relevant features not only decrease the processing time to train a classifier but also provide better generalisation. Linear discriminant classifier which is commonly used for classification may not be able to classify in better way non-linearly separable data. Recently, exponential discriminant analysis, a variant of linear discriminant analysis (LDA), is proposed which transforms the scatter matrices to a new space by distance diffusion mapping. This provides exponential discriminant analysis (EDA) much more discriminant power to classify non-linearly separable data and helps in improving classification accuracy in comparison to LDA. In this paper, the performance of EDA in conjunction with feature selection methods has been investigated. For feature selection, Kullback divergence, Chernoff distance measures and linear regression measures are used to determine relevant features from higher-order statistics of images. The performance is evaluated in terms classification error and computation time. Experimental results show that exponential discriminate analysis in conjunction with linear regression significantly performs better in terms of both classification error and compilation time of training classifier. Defence Science Journal, 2012, 62(1), pp.19 -24 , DOI:http://dx.doi.org/10.14429/dsj. 62.1437