Orthogonal Semi-Supervised Marginal Fisher Analysis

The recently proposed Marginal Fisher Analysis (MFA) achieved high performance for face recognition. However, MFA suffers from the small sample size problem and the optimal basis vectors obtained by the MFA are nonorthogonal. In this paper, we present a new method called Direct Orthogonal Marginal Fisher Analysis (DOMFA), which is able to extract all the orthogonal discriminant vectors simultaneously in the high-dimensional feature space without pre-processing using PCA and does not suffer the small sample size problem. Experimental results on ORL database indicate that the proposed DOMFA method achieves better recognition rate than the MFA method and some other orthogonal feature extraction algorithms.

Marginal Fisher analysis (MFA) exploits the margin criterion to compact the intraclass data and separate the interclass data, and it is very useful to analyze the high-dimensional data. However, MFA just considers the structure relationships of neighbor points, and it cannot effectively represent the intrinsic structure of hyperspectral imagery (HSI) that possesses many homogenous areas. In this paper, we propose a new dimensionality reduction (DR) method, termed local geometric structure Fisher analysis (LGSFA), for HSI classification. Firstly, LGSFA uses the intraclass neighbor points of each point to compute its reconstruction point. Then, an intrinsic graph and a penalty graph are constructed to reveal the intraclass and interclass properties of hyperspectral data. Finally, the neighbor points and corresponding intraclass reconstruction points are used to enhance the intraclass-manifold compactness and the interclass-manifold separability. LGSFA can effectively reveal the intrinsic manifold structure and obtain the discriminating features of HSI data for classification. Experiments on the Salinas, Indian Pines, and Urban data sets show that the proposed LGSFA algorithm achieves the best classification results than other state-of-the-art methods.

In this paper, we propose a new supervised learning algorithm, which is named the Generalized Marginal Fisher Analysis (GMFA), to utilize the advantages of the Marginal Fisher Analysis (MFA) and the Generalized Singular Value Decomposition (GSVD) techniques for face recognition. The experimental results on several standard face databases demonstrate that GMFA outperforms LDA/Fisherface, LDA/GSVD and MFA.

In spite of extensive research in the last decade, activity recognition still faces many challenges for real-world applications. On one hand, when attempting to recognize various activities, different sensors play different on different activity classes. This heterogeneity raises the necessity of learning the optimal combination of sensor modalities for each activity. On the other hand, users may consistently or occasionally annotate activities. To boost recognition accuracy, we need to incorporate the user input and incrementally adjust the model. To tackle these challenges, we propose an adaptive activity recognition with dynamic heterogeneous sensor fusion framework.We dynamically fuse various modalities to characterize different activities. The model is consistently updated upon arrival of newly labeled data. To evaluate the effectiveness of the proposed framework, we incorporate it into popular feature transformation algorithms, e.g., Linear Discriminant Analysis, Marginal Fisher's Analysis, and Maximum Mutual Information in the proposed framework. Finally, we carry out experiments on a real-world dataset collected over two weeks. The result demonstrates the practical implication of our framework and its advantage over existing approaches.

The dimension reduction methods have been proved powerful and practical to extract latent features hidden in the signal for process monitoring. A novel nonlinear dimension reduction method called kernel orthogonal global-local preserving projections (KOGLPP) is proposed and applied for condition monitoring and fault detection. To overcome the shortcomings of kernel locality preserving projections (KLPP) and kernel principal component analysis (KPCA), the KOGLPP model aims at preserving the global and local data structures simultaneously by constructing a dual-objective optimization function, and a tuning parameter is introduced to adjust the trade-off between the global and local data structures. For the purpose of condition monitoring and fault detection, monitoring statistics are constructed in low dimensional feature space. As KOGLPP combines the advantages of both KPCA and KLPP, KOGLPP is also more powerful in extracting potential useful data characteristics. Finally, the effectiveness of the proposed nonlinear dimension reduction method is evaluated experimentally on a numerical example and a bearing test-rig. The results indicate its potential applications as an effective and reliable tool for condition monitoring and fault detection.

Determination of structural characteristics of forests at large scales is an important problem in both scientific studies and development of management practices. Light detection and ranging (LIDAR) waveform data have been demonstrated to be valuable for estimating forest structural parameters even in dense forests, although challenges inherent to the LIDAR acquisition systems must be addressed. A new approach for processing LIDAR waveform data to estimate forest structural parameters is proposed. It was applied to Laser Vegetation Imaging Sensor waveform data acquired over old-growth tropical forest in the La Selva Biological Station, Costa Rica. Linear and nonlinear feature extraction methods were utilized to derive a lower dimensional feature space from high-dimensional LIDAR waveform data. The resulting features were used to estimate mean canopy heights through multiple linear regression analysis. Experimental results obtained by the new approach were statistically comparable to estimates obtained using features extracted via traditional waveform analysis, and the proposed approach successfully discovered another meaningful lower dimensional feature space without manual interpretation.

Locally linear embedding (LLE) is an elegant nonlinear method for feature extraction and manifold learning, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. However, LLE algorithm fails when it is directly applied to video with multi-shot. In this paper, video manifold feature is defined firstly, and then using LLE we extract video manifold feature through adding virtual frames on an enriched set. The proposed manifold feature extraction method is applied to shot transition and video trajectories. The video manifold feature description gives a new tool for video analysis.

Subspace learning based face recognition methods have attracted considerable interests in recent years, including principal component analysis (PCA), linear discriminant analysis (LDA), locality preserving projection (LPP), neighborhood preserving embedding (NPE) and marginal Fisher analysis (MFA). However, a disadvantage of all these approaches is that their computations involve eigen- decomposition of dense matrices which is expensive in both time and memory. In this paper, we propose a novel dimensionality reduction framework, called spectral regression (SR), for efficient regularized subspace learning. SR casts the problem of learning the projective functions into a regression framework, which avoids eigen-decomposition of dense matrices. Also, with the regression based framework, different kinds of regularizes can be naturally incorporated into our algorithm which makes it more flexible. Computational analysis shows that SR has only linear-time complexity which is a huge speed up comparing to the cubic-time complexity of the ordinary approaches. Experimental results on face recognition demonstrate the effectiveness and efficiency of our method.

This paper develops a supervised discriminant technique based on graph embedding, called maximum inter-class and marginal discriminant embedding (MIMDE) for feature and classification in high-dimensional data space. In graph embedding, our objective is to find a linear transform matrix to make the samples in the same class as compact as possible and the samples belong to the different classes as dispersed as possible through local property (intra-class compactness graph and inter-class separability graph) and marginal property (margin separability graph). This characteristic makes MIMDE more intuitive and more powerful than marginal Fisher analysis which not considered inter-class separability graph. Experimental results on ORL and AR face databases show the effectiveness of the proposed method.

Learning universal multiview dictionary for human action recognition

In the last decade, many approaches have been developed to solve one-class classification (OCC) problems for anomaly detection. Many of them rely on estimating the statistical distribution of the data, find hidden patterns, or remap the data in advantageous feature spaces. This kind of techniques usually needs some a priori knowledge of the data distribution (i.e., Gaussian) or the setting of some parameters to achieve good classification performance, making their use less effective when the data distribution is unknown. In this paper, we propose a novel blind anomaly detection for low dimensional feature spaces, that exploits the flexibility of the neural network (NN) structure to find the class boundaries without any information about the shape of the data distribution. To prove the generality of the solution, we tested many different class shapes, and we applied it to a structural health monitoring (SHM) case study. Without requiring the tuning of hyperparameters, the performance of the proposed algorithm overcomes that of some known approaches like principal component analysis (PCA), kernel principal component analysis (KPCA), Gaussian mixture model (GMM), and autoassociative neural network (ANN) in many cases, and performs well in the specific SHM setting.

Manifold learning aims to map the original data from a high-dimensional space into a low-dimensional feature space with possible better discriminative structure. In this paper, we propose a supervised manifold learning approach called SubManifold Individuality LEarning (SMILE). In SMILE, the linear subspace derived from the principal component analysis based on data with the same label is named as "the individual subspace", while the linear subspace learned from all data is defined as "the global subspace". For each data sample, the aim of SMILE is to enlarge the diversity between its reconstructed data from individual subspace and global subspace, respectively, so that the intrinsic character of each class can be stimulated in the feature space. SMILE also utilizes the Laplacian matrix to restrict the local structure of data in the low-dimensional feature space in order to preserve the locality of the high-dimensional data. The proposed method is validated in appearance-based face recognition problem on some typical facial image databases via extracting discriminative features. Experimental results show that the proposed approach can obtain the discriminative structure of facial manifold and extract better features for face recognition than other counterparts approaches.

To efficiently solve Web document classification problem,a novel document classification algorithm based on kernel discriminant analysis(KDA) and SVM was proposed.The proposed algorithm firstly reduced the high dimensional Web document space in the training sets to the lower dimensional space with KDA algorithm,then the classification and predication in the lower dimensional feature space were implemented with the multiplicative update-based optimal SVM.The experimental evaluations were performed on the Reuters-21578 and 20-Newsgroup which are two well-known data sets in the field of document classification.Experimental results show that the proposed algorithm not only achieves higher classification accuracy,but also has lower running time.

Text categorization is an important research topic in Information Retrieval area and it is one of the key techniques for handling and organizing the huge amount of text data available on the Internet and other digital format in our daily life. In this paper, we propose a method for text categorization based on Mean Shift. Mean Shift algorithm is a well developed technique in computer vision researches. We extend the application of Mean Shift to text categorization by reducing the dimensions of text vector space to a proper scale. Firstly, a low-dimensional feature space is constructed using the feature selection method by the theory of information gain. Secondly, an adaptive Mean Shift algorithm is applied for detecting the centers (centroids) of each category on the feature space above. Finally, each document will be added to its most similar category by calculating the similarities between the document and the center of every category. Experimental results on 20NewsGroup and Rueters-21578 corpus show that this method can achieve higher performance than some classic text categorization method like KNN, Naive Bayes and SVM.

Multiple classification systems play an important role in increasing recognition performance, especially when using heterogeneous classifiers that effectively improve performance. In this study, a new hybrid classifier was designed using heterogeneous Fisherface and discriminative common vector approach (DCVA) subspace recognition methods, which gave successful results in face recognition. While the classification process of DCVA is based on the common properties of signals belonging to the classes, the classification process of Fisherface is based on the different properties of signals. To create a hybrid classifier, called the Hybrid DCVA-Fisherface, the classifiers' decision rules were combined using the Minimum Proportional Score Algorithm and Recognition Update Algorithm. In addition to the proposed subspace classifiers, convolutional neural networks, Transform learning-Alexnet, Alexnet + SVM, and Alexnet + KNN were used for classification. Studies were conducted using the ORL, YALE, Extended YALE B and Face Research Lab London Set (FRLL). To better examine the efficiency of the algorithms, tests were also carried out by downsampling the images. When the experimental results were analysed, the proposed hybrid classifier gave higher recognition rates than all classifiers for ORL, YALE, and Extended YALE B. However, deep learning methods generally achieved better recognition performance than subspace classifiers for the FRLL database, which has more classes than other databases.