Two-Dimensional Principal Component Analysis and Its Extensions

Although two-dimensional principal component analysis (2DPCA) extracts image features directly from 2D image matrices rather than one dimensional vectors, 2DPCA is only based on the whole images to preserve total variances by maximizing the trace of feature covariance matrix. Thus, 2DPCA cannot extract localized components, which are usually important for face recognition. Inspired by nonnegative matrix factorization (NMF), which is based on localized features, we propose a novel algorithm for face recognition called two-dimensional nonnegative principal component analysis (2DNPCA) to extract localized components and maintain the maximal variance property of 2DPCA. 2DNPCA is a matrix-based algorithm to preserve the local structure of facial images and has the nonnegative constraint to learn localized components. Therefore, 2DNPCA has both advantages of 2DPCA and NMF. Furthermore, 2DNPCA solves the time-consuming problem by removing the restriction of minimizing the cost function and extracting only the base matrix. The nearest neighbor (NN) classifier and linear regression (LR) classifier are used for classification and extensive experimental results show that 2DNPCA plus NN and 2DNPCA plus LR are both very efficient approaches for face recognition.

In this paper, we propose a disturbance covariance matrix estimation method for radar signal processing when the number of training samples is severely limited. Several maximum likelihood (ML) estimation methods based on the structured of the covariance matrix, which is modeled as a sum of an unknown positive semi-definite Hermitian matrix and a matrix proportional to identity, have been proposed to solve the problem. Based on the covariance structure, we incorporate additional constraint on the low rank property of the covariance matrix, and propose a low rank regularized ML estimation method for covariance matrix estimation. The low rank is achieved by solving a reweighted trace minimization problem. Experimental results demonstrate the effective of the proposed method, both in the accuracy of rank estimation and the improvement of signal interference and noise ratio (SINR).

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

The covariance matrix (or its inverse, the precision matrix) is central to many chemometric techniques. Traditional sample estimators perform poorly for high‐dimensional data such as metabolomics data. Because of this, many traditional inference techniques break down or produce unreliable results. In this paper, we selectively review several modern estimators of the covariance and precision matrix that improve upon the traditional sample estimator. We focus on 3 general techniques: eigenvalue‐shrinkage estimation, ridge‐type estimation, and structured estimation. These methods rely on different assumptions regarding the structure of the covariance or precision matrix. Various examples, in particular using metabolomics data, are used to compare these techniques and to demonstrate that in concert with, eg, principal component analysis, multivariate analysis of variance, and Gaussian graphical models, better results are obtained.

We examined the ability of eigenvalue tests to distinguish field—collected from random, assemblage structure data sets. Eight published time series of species abundances were used in the analysis, including data sets for: fishes, birds, mammals, stream benthos, and crabs. To test the efficacy of eigenvalue tests, we constructed 1000 randomly generated data sets for each real data set, whose means and variances were identical to the means and variances of the original data matrices. The data sets were then subjected to a principal components analysis (PCA) and eigenvalue tests used to identify significant eigenvalues for both correlation and covariance matrix solutions. We also examined the effects of: (1) number of species (= number of variables), (2) number of samples (= replication), (3) variance structure, on the performance of the test. Using PCA's based on the correlation matrix and with sample sizes typically encountered in the field, the eigenvalue tests generally performed at the .05 level when a = .01. Slightly poorer results were obtained with the covariance matrix. Increasing the number of samples to at least three times the number of species generally gave a level coverage of an a level test (i.e., a = .05, .01). Increasing variance in the data set only affected test outcomes at levels of replication less than twice the number of species. We conclude that the eigenvalue tests can be used to detect patterns in PCA's of assemblage structure data, if the number of samples is at least three times the number of species and either a covariance or correlation matrix solution is used. It is assumed that these patterns represent ecologically meaningful patterns of variation.

Journal Article Nonparametric estimation of large covariance matrices of longitudinal data Get access Wei Biao Wu, Wei Biao Wu Search for other works by this author on: Oxford Academic Google Scholar Mohsen Pourahmadi Mohsen Pourahmadi Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 90, Issue 4, December 2003, Pages 831–844, https://doi.org/10.1093/biomet/90.4.831 Published: 01 December 2003

Adaptive beamformers are sensitive to model mismatch, especially when the number of training samples is small or the training samples are contaminated by the signal component. In this paper, we consider an extreme scenario where only a single signal-contaminated snapshot is available for adaptive beamformer design. In such a case, we cannot perform direct inversion or eigen-decomposition of the rank-one sample covariance matrix required in adaptive beamformer design. To address this issue, we formulate a sparsity-constrained covariance matrix fitting problem to estimate the spatial spectrum distribution over the observed spatial domain, which is then used for adaptive beamformer design via the sparse reconstruction of the interference-plus-noise covariance matrix. Simulation results demonstrate the performance advantage of the proposed adaptive beamforming algorithm over other beamforming algorithms suitable for the single-snapshot scenario.

An efficient method of face detection based on two-dimensional principal component analysis (PCA) incorporating with support vector machine (SVM) is proposed in this paper. Firstly, a 2DPCA coarse filter with relatively lower computational complexity is applied to the whole input image to filter out most of the non-face, then follows the SVM classifier to make the final decision, so the detection process is speeded up. As opposed to PCA, 2DPCA is based on 2D image matrices rather than ID vector so the image matrix does not need to be transformed into a vector prior to feature extraction. The experiment results show that the method can effectively detect faces under complicated background, and the processing time is shorter than using SVM alone

This paper reports the scaling behavior of the fuzzy c-means based classifier (FCMC) with many parameters. FCMC is a classifier based on clustering approaches. The classification accuracy on test sets (i.e., the generalization capability) is not necessarily improved by increasing the number of clusters. Especially when the number of training samples is relatively small, not only the classification boundary over-fits the data, but also covariance matrices and cluster centers are computed incorrectly, since the number of samples in each cluster becomes smaller. Hence, the test set accuracy deteriorates. The performance of FCMC with two clusters in each class and the number of training samples less than 1000, was reported in the literature. This paper reports the scaling behavior of FCMC by testing with variously-sized training samples. The number of clusters of FCMC is increased up to eight. The number of clusters used in this paper is not very large but the number of parameters is relatively large. So, the parameters are optimized to training sets. LibSVM is one of the widely known state of the art tools for support vector machines (SVM). The test set accuracy, training time and testing time (i.e., the detection time) of FCMC are compared with LibSVM by varying the size of training sets. FCMC shows a good generalization capability, though the parameters are optimized to training sets. When the number of training samples is increased by 10 times, the training time of FCMC increases by 10 times, but that of LibSVM increases by a factor of 100. The testing time is also much shorter than LibSVM when the size of the training set is large.

For multivariate analysis with p variables the problem that often arises is the ambiguous nature of the correlation or covariance matrix. When p is moderately or very large it is generally difficult to identify the true nature of relationship among the variables as well as observations from the covariance or correlation matrix. Under such situations a very common way to simplify the matter is to reduce the dimension by considering only those variables (actual or derived) which are truly responsible for the overall variation. Important and useful dimension reduction techniques are Principal Component Analysis (PCA), Factor Analysis, Multidimensional Scaling, Independent Component Analysis (ICA), etc. Among them PCA is the most popular one. One may look at this method in three different ways. It may be considered as a method of transforming correlated variables into uncorrelated one or a method of finding linear combinations with relatively small or large variability or a tool for data reduction. The third criterion is more data oriented. In PCA primarily it is not necessary to make any assumption regarding the underlying multivariate distribution but if we are interested in some inference problems related to PCA then assumption of multivariate normality is necessary. The eigen values and eigen vectors of the covariance or correlation matrix are the main contributors of a PCA. The eigen vectors determine the directions of maximum variability whereas the eigen values specify the variances. In practice, decisions regarding the quality of the principal component approximation should be made on the basis of eigen value–eigen vector pairs.

Covariance matrix plays an important role in risk management, asset pricing, and portfolio allocation. Covariance matrix estimation becomes challenging when the dimensionality is comparable or much larger than the sample size. A widely used approach for reducing dimensionality is based on multi-factor models. Although it has been well studied and quite successful in many applications, the quality of the estimated covariance matrix is often degraded due to a nontrivial amount of missing data in the factor matrix for both technical and cost reasons. Since the factor matrix is only approximately low rank or even has full rank, existing matrix completion algorithms are not applicable. We consider a new matrix completion paradigm using the factor models directly and apply the alternating direction method of multipliers for the recovery. Numerical experiments show that the nuclear-norm matrix completion approaches are not suitable but our proposed models and algorithms are promising.

In this paper, based on the study of the Two-Dimensional Principal Component Analysis (2DPCA), Two-Dimensional Principal Component Analysis (2DPCA) and fuzzy set theory, we propose a integrated face recognition algorithm based on wavelet subspace. This method can make good use of the advantages of each single method, and also can make up for the defect of each other. The comparison of the results of the different methods identification effect on the ORL、YALE and FERET face database show, the integrated method proposed in this paper improves the recognition rate, and it also reduces the training and classification time as well. Key word: face recognition; Two-Dimensional Principal Component Analysis (2DPCA); Two-Dimensional Linear Discriminant Analysis (2DLDA); fuzzy set theory; feature extraction be divided into several categories:the method based on geometric features、the method based on model、the method based on the statistical、the method based on neural network method and the method of combining multiple classifiers, this paper mainly focuses on the method based on statistics. The method based on statistical faced the image as random vector, thus using some statistical methods to analyze the face model, the most representative methods include the eigenface method based on principal component analysis which proposed by Turk M (2)

As training samples are not always target-free in space-time processing for airborne radar, the traditional methods usually use the sample covariance matrix (SCM) as the test covariance matrix (TCM) to censor contaminated training samples. However, the SCM cannot represent the property of the cell under test (CUT) accurately, resulting in low selection efficiency. To deal with this problem, this letter proposes a novel training sample selection method based on covariance matrix whitening. Specifically, we utilize the reconstructed subaperture's clutter covariance matrix (RSCCM) of the CUT as the TCM. The RSCCM is only determined by the CUT and can characterize the CUT directly. Then, we use the RSCCM to whiten the subaperture's covariance matrix of the training sample. A criterion for selecting the training samples is derived based on the maximum eigenvalue of the whitened subaperture's covariance matrix, which is related to the energy of the outliers and more stable than the statistic of the generalized inner product method. Simulations are conducted to evaluate the performance of the proposed method.

Dimensionality reduction methods play an important role in face recognition. Principal component analysis (PCA) and two-dimensional principal component analysis (2DPCA) are two kinds of important methods in this field. Recent research seems like that 2DPCA method is superior to PCA method. To prove if this conclusion is always true, a comprehensive comparison study between PCA and 2DPCA methods was carried out. A novel concept, called column-image difference (CID), was proposed to analyze the difference between PCA and 2DPCA methods in theory. It is found that there exist some restrictive conditions when 2DPCA outperforms PCA. After theoretical analysis, the experiments were conducted on four famous face image databases. The experiment results confirm the validity of theoretical claim.

This paper presents a comparison between one-dimensional component analysis (1D-PCA) and two-dimensional principal component analysis (2D-PCA) under two different types of classification techniques namely k-nearest neighbor (kNN) and Support Vector Machines (SVM). These two techniques differ in the method to determine the image covariance matrix. 2DPCA used 2D image matrices instead of column vectors in 1DPCA. The eigenvectors derived from these matrices will result in images in reduced dimensions to be used for classification. K-nearest neighbor algorithm (kNN) and Support Vector Machines (SVM) were used for classification. The performance measures used for comparison were classification accuracy and computational time. The tests were performed on the ORL image database. The experimental results indicate that 2DPCA outperforms in terms of classification accuracy and computational complexity.