Inferences in panel data with interactive effects using large covariance matrices

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

The smoothness of subdiagonals of the Cholesky factor of large covariance matrices is closely related to the degree of nonstationarity of autoregressive models for time series data. Heuristically, one expects for nearly stationary covariance matrix entries in each subdiagonal of the Cholesky factor of its inverse to be approximately the same in the sense that the sum of the absolute values of successive differences is small. Statistically, such smoothness is achieved by regularizing each subdiagonal using fused-type lasso penalties. We rely on the standard Cholesky factor as the new parameter within a regularized normal likelihood setup which guarantees: (a) joint convexity of the likelihood function, (b) strict convexity of the likelihood function restricted to each subdiagonal even when n < p, and (c) positive-definiteness of the estimated covariance matrix. A block coordinate descent algorithm, where each block is a subdiagonal, is proposed, and its convergence is established under mild conditions. Lack of decoupling of the penalized likelihood function into a sum of functions involving individual subdiagonals gives rise to some computational challenges and advantages relative to two recent algorithms for sparse estimation of the Cholesky factor, which decouple row-wise. Simulation results and real data analysis show the scope and good performance of the proposed methodology. Software for our method is freely available in R language. Supplementary materials for this article are available online.

Journal Article Nonparametric estimation of large covariance matrices of longitudinal data Get access Wei Biao Wu, Wei Biao Wu Search for other works by this author on: Oxford Academic Google Scholar Mohsen Pourahmadi Mohsen Pourahmadi Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 90, Issue 4, December 2003, Pages 831–844, https://doi.org/10.1093/biomet/90.4.831 Published: 01 December 2003

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&amp;P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.

Efficient estimation of approximate factor models via penalized maximum likelihood

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

This paper develops an improved covariance matrix estimator in the mean-variance optimisation setting. Well-known problems with the sample covariance matrix are that it is singular when the number of observations is less than the number of assets, and can be nearly singular when the number of observations exceeds the number of assets. Therefore, using the sample covariance matrix as an input in mean-variance optimisation can result in unreasonable optimal portfolios and badly biased estimates of Sharpe ratios. We address this problem by imposing structure on the estimated covariance matrix by putting constraints on the Sharpe ratio, asset return variances, and the variance of the global minimum variance portfolio. We show that the constrained maximum likelihood estimator (CMLE) performs better than the sample covariance matrix. Moreover, when the shrinkage approach is applied to the CMLE and single index covariance matrix, it performs better than the shrinkage estimator of Ledoit and Wolf (2004).

Many applications require precise estimates of high-dimensional covariance matrices. The standard estimator is the sample covariance matrix, which is conceptually simple, fast to compute and has favorable properties in the limit of infinitely many observations. The picture changes when the dimensionality is of the same order as the number of observations. In such cases, the eigenvalues of the sample covariance matrix are highly biased, the condition number becomes large and the inversion of the matrix gets numerically unstable.&#13;\nA number of alternative estimators are superior in the high-dimensional setting, which include as subcategories structured estimators, regularized estimators and spectrum correction methods. In this thesis I contribute to all three areas. &#13;\nIn the area of structured estimation, I focus on models with low intrinsic dimensionality. I analyze the bias in Factor Analysis, the state-of-the-art factor model and propose Directional Variance Adjustment (DVA) Factor Analysis, which reduces bias and yields improved estimates of the covariance matrix.&#13;\nAnalytical shrinkage of Ledoit and Wolf (LW-Shrinkage) is the most popular regularized estimator. I contribute in three aspects: first, I provide a theoretical analysis of the behavior of LW-Shrinkage in the presence of pronounced eigendirections, a case of great practical relevance. I show that LW-Shrinkage does not perform well in this setting and propose aoc-Shrinkage which yields significant improvements. Second, I discuss the effect of autocorrelation on LW-Shrinkage and review the Sancetta-Estimator, an extension of LW-Shrinkage to autocorrelated data. I show that the Sancetta-Estimator is biased and propose a theoretically and empirically superior estimator with reduced bias. Third, I propose an extension of shrinkage to multiple shrinkage targets. Multi-Target Shrinkage is not restricted to covariance estimation and allows for many interesting applications which go beyond regularization, including transfer learning. I provide a detailed theoretical and empirical analysis.&#13;\nSpectrum correction approaches the problem of covariance estimation by improving the estimates of the eigenvalues of the sample covariance matrix. I discuss the state-of-the-art approach, Nonlinear Shrinkage, and propose a cross-validation based covariance (CVC) estimator which yields competitive performance at increased numerical stability and greatly reduced complexity and computational cost. On all data sets considered, CVC is on par or superior in comparison to the regularized and structured estimators.&#13;\nIn the last chapter, I conclude with a discussion of the advantages and disadvantages of all covariance estimators presented in this thesis and give situation-specific recommendations. In addition, the appendix contains a systematic analysis of Linear Discriminant Analysis as a model application, which sheds light on the interdependency between the generative model of the data and various covariance estimators.

We consider large complex random sample covariance matrices obtained from “spiked populations,” that is, when the true covariance matrix is diagonal with all but finitely many eigenvalues equal to 1. We investigate the limiting behavior of the largest eigenvalues when the population and the sample sizes both become large. Under some conditions on moments of the sample distribution, we prove that the asymptotic fluctuations of the largest eigenvalues are the same as for a complex Gaussian sample with the same true covariance. The real setting is also considered.

The Riemannian geometry of covariance matrices has been essential to several successful applications, in computer vision, biomedical signal and image processing, and radar data processing. For these applications, an important ongoing challenge is to develop Riemannian-geometric tools which are adapted to structured covariance matrices. This paper proposes to meet this challenge by introducing a new class of probability distributions, Gaussian distributions of structured covariance matrices . These are Riemannian analogs of Gaussian distributions, which only sample from covariance matrices having a preassigned structure, such as complex, Toeplitz, or block-Toeplitz. The usefulness of these distributions stems from three features: 1) they are completely tractable, analytically, or numerically, when dealing with large covariance matrices; 2) they provide a statistical foundation to the concept of structured Riemannian barycentre ( i.e. , Frechet or geometric mean); and 3) they lead to efficient statistical learning algorithms, which realise, among others, density estimation and classification of structured covariance matrices. This paper starts from the observation that several spaces of structured covariance matrices, considered from a geometric point of view, are Riemannian symmetric spaces. Accordingly, it develops an original theory of Gaussian distributions on Riemannian symmetric spaces, of their statistical inference, and of their relationship to the concept of Riemannian barycentre. Then, it uses this original theory to give a detailed description of Gaussian distributions of three kinds of structured covariance matrices, complex, Toeplitz, and block-Toeplitz. Finally, it describes algorithms for density estimation and classification of structured covariance matrices, based on Gaussian distribution mixture models.

One of the major challenges in multivariate analysis is the estimation of population covariance matrix from the sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Both of these techniques try to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. Both methods have their respective pros and cons. In this paper, we propose an improved estimator which exploits the advantages of these techniques by taking optimally weighted convex combination of covariance matrices estimated by shrinkage transformation and a filter based on RMT. It is a generalized estimator which can adapt to changing sampling noise conditions by performing hyperparameter optimization. Using data from six of the world's biggest stock exchanges, we show that the proposed estimator outperforms the existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. The proposed estimator can be useful in a wide range of machine learning and signal processing applications. © 1994-2012 IEEE.

The major problem in adaptively processing data from a large array is inversion of the covariance matrix. The matrix must be well-conditioned to be inverted. The number of independent samples that is needed to form a well-conditioned matrix is at least twice the number of array elements. The integration time to form a well-conditioned large array covariance matrix may then be unreasonably long relative to the coherent time of signal in a changing environment. A sub-optimal subarray processing has been developed. The adaptive weight vectors are calculated for each subarray. These weight vectors are linearly combined to form a full-array adaptive weight vector and are applied to the full-array covariance matrix. The required integration time is reduced to that which is needed to form the well-conditioned subarray covariance matrices. In a recent shallow water experiment, a three-dimensional array had been deployed. The array consisted of 5 VLAs each with 30-element spaced evenly on a circular baseline of a radius of 200 m. The subarray processing approach had been used to localize a moving source that had a fast range/bearing change relative to the baseline.

A key component in most parametric classifiers is the estimation of an inverse covariance matrix. In hyperspectral images, the number of bands can be in the hundreds, leading to covariance matrices having tens of thousands of elements. Lately, the use of linear regression in estimating the inverse covariance matrix has been introduced in the time-series literature. This paper adopts and expands these ideas to ill-posed hyperspectral image classification problems. The results indicate that at least some of the approaches can give a lower classification error than traditional methods such as the linear discriminant analysis and the regularized discriminant analysis. Furthermore, the results show that, contrary to earlier beliefs, estimating long-range dependencies between bands appears necessary to build an effective hyperspectral classifier and that the high correlations between neighboring bands seem to allow differing sparsity configurations of the inverse covariance matrix to obtain similar classification results.