Subspace Methods with Globally/Locally Weighted Correlation Matrix

Correlation matrix plays a key role in many multivariate methods (e.g., graphical model estimation and factor analysis). The current state-of-the-art in estimating large correlation matrices focuses on the use of Pearson's sample correlation matrix. Although Pearson's sample correlation matrix enjoys various good properties under Gaussian models, its not an effective estimator when facing heavy-tail distributions with possible outliers. As a robust alternative, Han and Liu (2013b) advocated the use of a transformed version of the Kendall's tau sample correlation matrix in estimating high dimensional latent generalized correlation matrix under the transelliptical distribution family (or elliptical copula). The transelliptical family assumes that after unspecified marginal monotone transformations, the data follow an elliptical distribution. In this paper, we study the theoretical properties of the Kendall's tau sample correlation matrix and its transformed version proposed in Han and Liu (2013b) for estimating the population Kendall's tau correlation matrix and the latent Pearson's correlation matrix under both spectral and restricted spectral norms. With regard to the spectral norm, we highlight the role of "effective rank" in quantifying the rate of convergence. With regard to the restricted spectral norm, we for the first time present a "sign subgaussian condition" which is sufficient to guarantee that the rank-based correlation matrix estimator attains the optimal rate of convergence. In both cases, we do not need any moment condition.

On using prototype reduction schemes to optimize kernel-based nonlinear subspace methods

Asymptotic normal distributions of eigenvalue-normed eigenvectors (Pearson-Hotelling principal-component vectors) of sample variance and correlation matrices are derived. Each distribution follows from the asymptotic distribution of the whole matrix of eigenvectors, also obtained in the paper. The results are presented for the general case, where the existence of fourth-order finite population moments is assumed, and also specified for normal and elliptical populations. Population variance and correlation matrices are supposed to be nonsingular and without multiple eigenvalues. A comparison with unit-length eigenvectors (Anderson principal-component vectors) is being made. Special attention is being paid to asymptotic variances of eigenvectors of both sample variance and correlation matrices in the two approaches. It is shown that the asymptotic variance matrices of principal-component vectors for sample variance and correlation matrices are singular in the Anderson approach. In the Pearson-Hotelling approach, however, the asymptotic variance of eigenvectors of the sample variance matrix is nonsingular, whereas in the sample correlation matrix case the asymptotic variance matrix of eigenvectors is nonsingular under certain conditions.

Testing the equality of two or multiple correlation or covariance matrices is an important problem in biology, finance and many other areas. High dimensionality, where the number of features is much larger than the sample size, causes conventional procedures to perform poorly, as they are often based on limiting distributions of test statistics in the classical large sample size setting. Moreover, their performance is often contingent on whether the matrix of differences is sparse or dense, while such information is rarely available. In this article, we develop a new family of permutation testing procedures to tackle these challenges. The introduced tests are demonstrated to outperform many other competing procedures in terms of size control and power under various settings. In particular, using our variance-stabilizing transformation, the proposed methods provide the best performance for testing correlation or covariance matrix differences in both sparse and dense settings. We establish non-asymptotic guarantees on the power of our test, which ensure its reliability for sparse and dense differential correlation matrices. Through the analysis of gene-expression and brain imaging data, we showcase the high power and accurate size control of our test in high-dimensional statistical applications.

Executive Summary. The fact that portfolio variances must be positive implies that the correlation matrix for asset returns should be positive definite. As such, a test for positive definiteness in the correlation matrix should be routinely implemented prior to any portfolio optimization exercise. A program is developed to test for positive definiteness in correlation matrices and to detect correlation coefficients that violate the positive definiteness condition. The correlation matrices from five randomly selected papers are tested and violations revealed in larger correlation matrices. Where violations are detected, the portfolio manager or analyst should proceed with the optimization exercise only if the alternative bounds on the correlation coefficients computed by our program are acceptable.

The Nearest Correlation Matrix Problem: Solution by Differential Evolution Method of Global Optimization

Accurately capturing the spatial correlations of traffic network significantly benefits short-term traffic forecasting. Some existing works represent spatial correlations in a simple one-dimensional space, but they cannot represent the real spatial correlations among sensors comprehensively. The other existing works represent the spatial correlations through grid-based method, but the local correlation of constructed spatial map is too superficial to extract deep spatial features effectively. Therefore, a novel deep learning model is proposed, which aims to represent the spatial correlations more effectively through a new correlation matrix structure. In the proposed model, the correlations among sensors are calculated from multiple perspectives to construct the speed, volume, and occupancy correlation matrices respectively. Then, considering that highly correlated sensors are close in the spatial dimension, an enhanced heuristic optimization algorithm is proposed to evolve these three correlation matrices into optimal ones by reorganizing the highly correlated sensors into each others neighborhood. Finally, the three optimal correlation matrices are combined to form a three-dimensional multivariate correlation matrix characterized by locally high correlation, which is beneficial to exploit the deep spatial features of traffic network. The experiments show that the proposed model has better accuracy and stability than other commonly used baseline models.

Most simulation studies in factor analysis follow a process of constructing population correlation matrices from the common-factor model and generating sample correlation matrices from the population matrices. In the common-factor model, the population correlation matrix is perfectly fit by the model's containing common and unique factors. However, since no mathematical model accounts exactly for the real-world phenomena that it is intended to represent, the Tucker-Koopman-Linn model (1969) is more realistic for generating correlation matrices than the conventional common-factor model because the former incorporates model error. In this paper, a procedure for generating population and sample correlation matrices with model error by combining the Tucker-Koopman-Linn model and Wijsman's algorithm (1959) is presented. The SAS/IML program for generating correlation matrices is described, and an example is also provided.

The simplest way to describe the dependence for a set of financial assets is their correlation matrix. This correlation matrix can be improper when it is specified element-wise. We describe a new method for obtaining a positive definite correlation matrix starting from an improper one. The expert's opinion and trust in each pairwise correlation is described by a beta distribution. Then, by combining these individual distributions, a joint distribution over the space of positive definite correlation matrices is obtained using Cholesky factorization, and its mode constitutes the new proper correlation matrix. The optimization is complemented by a visual representation of the entries that were most affected by the legalization procedure. We also sketch a Bayesian approach to the same problem.

Problem statement: In the presence of multicollinearity, the estimation of parameters in multiple linear regression models by means of Ordinary Least Squares (OLS) is known to suffer severe distortion. An alternative approach was to use the modified OLS which was based on the latent roots and latent vectors of the correlation matrix of the independent and dependent variables. This procedure is called the Latent Root Regression (LRR) which serves the purpose to improve the stability of the estimates for data plagued by multicollinearity. However, there was evidence that the LRR estimators were easily affected by a few atypical observations that we call outliers. It is now evident that the robust method alone cannot rectify the combined problems of multicollinearity and outliers. Approach: In this study, we proposed a robust procedure for the estimation of the regression parameters in the presence of multicollinearity and outliers. We called this method Latent Root-M based Regression (LRMB) because here we employed the weight of the M-estimator in the weighted correlation matrix. Numerical examples and some simulation studies were presented to illustrate the performance of the newly proposed method. Results: Results of the study showed that the LRMB method is more efficient than the existing methods. Conclusion/Recommendations: In order to get a reliable estimate, we recommend using the LRMB when both multicollinearity and outliers are present in the data.

This paper describes how new learning methods may make it possible for a large‐scale, hierarchical neural network to recognize most Japanese handwritten characters. This is a very large and complex task, as the Japanese character set consists of about 3000 categories which can be written in many different ways. Such a difficult task can lead a neural network to converge very slowly and to yield recognition rates that are uneven between categories. To address these problems we here propose five learning methods as modifications of the conventional back‐propagation learning rule. These methods produce fast convergence, even recognition rates over all categories, and adequate recognition of test samples. We also describe how a large‐scale neural network can be built by dividing the recognition task into several subtasks, with networks for each subtask, and then integrating these subnetworks in a large network with a hierarchical structure. In a hierarchical network, the upper level network directly integrates outputs from each lower level network. Application of that network to handwritten Japanese character recognition has resulted in poor recognition, because lower level networks do not know about unknown input patterns, and the direct integration of ambiguous outputs from many lower level networks confuses the upper level network. We propose a new integration method which provides each subnetwork with more information as to how close an input pattern is to the categories of that subnetwork. This method resulted in high recognition performance for character recognition. We here described the above methods, and report the performance of our implementation of a neural network for the recognition of 71 Hiragana characters, and describe our implementation of this network on a hypercube concurrent computer.

Meta-analysis plays a key role in combining studies to obtain more reliable results. In social, behavioral, and health sciences, measurement units are typically not well defined. More meaningful results can be obtained by standardizing the variables and via the analysis of the correlation matrix. Structural equation modeling (SEM) with the combined correlations, called meta-analytical SEM (MASEM), is a powerful tool for examining the relationship among latent constructs as well as those between the latent constructs and the manifest variables. Three classes of methods have been proposed for MASEM: (1) generalized least squares (GLS) in combining correlations and in estimating the structural model, (2) normal-distribution-based maximum likelihood (ML) in combining the correlations and then GLS in estimating the structural model (ML-GLS), and (3) ML in combining correlations and in estimating the structural model (ML). The current article shows that these three methods are equivalent. In particular, (a) the GLS method for combining correlation matrices in meta-analysis is asymptotically equivalent to ML, (b) the three methods (GLS, ML-GLS, ML) for MASEM with correlation matrices are asymptotically equivalent, (c) they also perform equally well empirically, and (d) the GLS method for SEM with the sample correlation matrix in a single study is asymptotically equivalent to ML, which has being discussed extensively in the SEM literature regarding whether the analysis of a correlation matrix yields consistent standard errors and asymptotically valid test statistics. The results and analysis suggest that a sample-size weighted GLS method is preferred for combining correlations and for MASEM.

A correlation matrix between three variables has to satisfy certain conditions. Such a matrix essentially contains three numbers and thus can be represented by a point in three dimensions. The set of all possible correlation matrices yields a convex solid body with an uncommon shape. All its cross sections perpendicular to the axes are ellipses. At the same time, its surface contains the vertices and edges of a regular tetrahedron. Another unusual shape is obtained for banded correlation matrices between four variables.

Let $\widetilde{X}_{M\times N}$ be a rectangular data matrix with independent real-valued entries $[\widetilde{x}_{ij}]$ satisfying $\mathbb {E}\widetilde{x}_{ij}=0$ and $\mathbb {E}\widetilde{x}^2_{ij}=\frac{1}{M}$, $N,M\to\infty$. These entries have a subexponential decay at the tails. We will be working in the regime $N/M=d_N,\lim_{N\to\infty}d_N\neq0,1,\infty$. In this paper we prove the edge universality of correlation matrices ${X}^{\dagger}X$, where the rectangular matrix $X$ (called the standardized matrix) is obtained by normalizing each column of the data matrix $\widetilde{X}$ by its Euclidean norm. Our main result states that asymptotically the $k$-point ($k\geq1$) correlation functions of the extreme eigenvalues (at both edges of the spectrum) of the correlation matrix ${X}^{\dagger}X$ converge to those of the Gaussian correlation matrix, that is, Tracy-Widom law, and, thus, in particular, the largest and the smallest eigenvalues of ${X}^{\dagger}X$ after appropriate centering and rescaling converge to the Tracy-Widom distribution. The asymptotic distribution of extreme eigenvalues of the Gaussian correlation matrix has been worked out only recently. As a corollary of the main result in this paper, we also obtain that the extreme eigenvalues of Gaussian correlation matrices are asymptotically distributed according to the Tracy-Widom law. The proof is based on the comparison of Green functions, but the key obstacle to be surmounted is the strong dependence of the entries of the correlation matrix. We achieve this via a novel argument which involves comparing the moments of product of the entries of the standardized data matrix to those of the raw data matrix. Our proof strategy may be extended for proving the edge universality of other random matrix ensembles with dependent entries and hence is of independent interest.

Practical meta-analysis of correlation matrices generally ignores covariances (and hence correlations) between correlation estimates. The authors consider various methods for allowing for covariances, including generalized least squares, maximum marginal likelihood, and Bayesian approaches, illustrated using a 6-dimensional response in a series of psychological studies concerning prediction of exercise behavior change. Quantities of interest include the overall population mean correlation matrix, the contrast between the mean correlations, the predicted correlation matrix in a new study, and the conflict between the existing studies and a new correlation matrix. The authors conclude that accounting for correlations between correlations is unnecessary when interested in individual correlations but potentially important if concerned with a composite measure involving 2 or more correlations. A simulation study indicates the asymptotic normal assumption appears reasonable. Because of potential instability in the generalized least squares methods, they recommend a model-based approach, either the maximum marginal likelihood approach or a full Bayesian analysis.