Posterior contraction in sparse Bayesian factor models for massive covariance matrices

High dimensional covariance matrices have attracted much attention of statisticians and econometricians during the past decades. Vast literature is devoted to the research in high dimensional covariance matrices. However, most of them are for constant covariance matrices. In many applications, constant covariance matrices are not appropriate, e.g. in portfolio allocation, dynamic covariance matrices would make much more sense. Simply assuming each entry of a covariance matrix is a function of time to introduce a dynamic structure would not work. In this paper, we are going to introduce a class of high dimensional dynamic covariance matrices in which a kind of additive structure is embedded. We will show the proposed high dimensional dynamic covariance matrices have many advantages in applications. An estimation procedure is also proposed to estimate the proposed high dimensional dynamic covariance matrices. Asymptotic properties are built to justify the proposed estimation procedure. Intensive simulation studies show the proposed estimation procedure works very well when sample size is finite. Finally, we apply the proposed high dimensional dynamic covariance matrices, together with the proposed estimation procedure, to portfolio allocation. The results look very interesting.

Statistical inference for high-dimensional data is increasingly essential due to the prevalence of big data. While recent advancements have improved testing linear hypotheses for high-dimensional mean vectors, exploration of testing linear hypotheses for high-dimensional covariance matrices remains limited. This article focuses on a general linear hypothesis testing (GLHT) problem for high-dimensional covariance matrices, encompassing various specific cases such as testing the equality of variances, testing for a given covariance matrix, assessing the homogeneity of covariance matrices, and examining linear combinations of covariance matrices. We propose and study a new test statistic for the GLHT problem on high-dimensional covariance matrices. Under certain regularity conditions and the null hypothesis, we demonstrate that the test statistic shares the same limiting distribution as a Chi-squared-type mixture. This mixture’s distribution can be accurately approximated using a three-cumulant matched Chi-squared-approximation approach. Additionally, we establish the asymptotic power of the proposed test against some local alternatives. Simulation studies and a real-world financial data analysis demonstrate that the proposed test outperforms several competitors in terms of size control.

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

We suggest a method for estimating a covariance matrix that can be represented as a sum of a sparse low-rank matrix and a diagonal matrix. Our formulation is based on penalized quadratic loss, which is a convex problem that can be solved via incremental gradient and proximal method. In contrast to other spiked covariance matrix estimation approaches that are related to principal component analysis and factor analysis, our method has a simple formulation and does not constrain entire rows and columns of the matrix to be zero. We further discuss a penalized entropy loss method that is nevertheless nonconvex and necessitates a majorization-minimization algorithm in combination with the incremental gradient and proximal method. We carry out simulations to demonstrate the finite-sample properties focusing on high-dimensional covariance matrices. Finally, the proposed method is illustrated using a gene expression data set.

Since the advent of Cognitive Radio and exploitation of unused spectrum by secondary devices, spectrum has gained immense importance. In cognitive radio network setting, due to very low received signal to noise ratio (SNR), spectrum sensing has been a challenging task. Further, signal detection is obscured due to presence of accumulated interference for which the statistical distribution is unknown and has significant distributional uncertainty. Covariance based signal detection methods have been proposed by various authors and proved to be better than other methods due to its independence on SNR, accuracy, and medium algorithmic complexity. Some results from Random Matrix Theory (RMT) on modelling of covariance matrices can be used in signal detection. This paper presents an algorithm for detection of Chi-Square distributed independent random signals using Wishart matrix which model the covariance matrix. A centering matrix is employed to scale the sample covariance matrix with Wishart distribution. The signals under various fading environment are used to evaluate the performance of the proposed algorithm. Also, a DVBT-2K signal is used to evaluate the performance for more realistic conditions. Furthermore, quantification of distributional uncertainty of accumulated interference in DTV based CRN is calculated using differential entropy of pdf of the interference. The quantified uncertainty is used to calculate the new threshold for detection. As per our knowledge, it is observed in the proposed work that the Maximum-Minimum Eigenvalue (MME) based proposed algorithm performs better than other proposed schemes in terms of probability of detection and complexity.

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

A sample \( X_{1},\ldots ,X_{n} \) consisting of independent identically distributed random vectors in \( \mathbb {R}^{p} \) with zero mean and covariance matrix \( \mathbf {\Sigma }\) is considered. The recovery of spectral projectors of high-dimensional covariance matrices from a sample of observations is a key problem in statistics arising in numerous applications. This chapter describes a bootstrap procedure for constructing confidence sets for the spectral projector \( \mathbf {P}_{r} \) related to rth eigenvalue of the covariance matrix \(\mathbf {\Sigma }\) from given data on the base of corresponding spectral projector \(\widehat{\mathbf {P}}_{r}\) of the sample covariance matrix \(\widehat{\mathbf {\Sigma }}\). This approach does not use the asymptotical distribution of \( \Vert \mathbf {P}_{r} - \widehat{\mathbf {P}}_{r} \Vert _{2} \) and does not require the computation of its moment characteristics. The performance of the bootstrap approximation procedure is analyzed.

We consider large complex random sample covariance matrices obtained from “spiked populations,” that is, when the true covariance matrix is diagonal with all but finitely many eigenvalues equal to 1. We investigate the limiting behavior of the largest eigenvalues when the population and the sample sizes both become large. Under some conditions on moments of the sample distribution, we prove that the asymptotic fluctuations of the largest eigenvalues are the same as for a complex Gaussian sample with the same true covariance. The real setting is also considered.

We address the problem of estimating high-dimensional covariance matrices (CM) for the explicit purpose of supervised anomaly detection, in the case when the number n of data points is lower than their dimensionality p. This is increasingly common with the emergence of the Internet of Things that makes it possible to collect data from many sensors simultaneously, resulting in very high-dimensional data points. When we attempt to perform anomaly detection for such data by modeling the normal behavior of the system by means of a multivariate Gaussian distribution, and n < p, the sample CM is singular, and cannot be used directly without some form of regularization. In contrast to existing methods for CM regularization that aim to fit the training data accurately, we propose a regularization algorithm for CM estimation that directly aims to maximize the area under the resulting receiver-operator characteristic (AUROC) for the ultimate decision problem that needs to be solved: anomaly detection. Experiments on test problems demonstrate the ability of the proposed algorithm to find CM estimates significantly better at anomaly detection than existing estimation methods that are unaware of the decision task that the CMs they produce will be used in.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

Radar data suffers from the curse of dimensionality, and with the deluge of data, due to advances in sensors, and technology, it has becomes increasingly challenging to interpret, visualize and process data. In this paper, we assume that the dimensions of the data matrix is large. That is, we assume that the number of variables as well as the number of samples is large, however, their ratio is a finite constant. We further assume a structured covariance matrix, comprised of low rank matrices added to a scaled identity matrix. The behavior of their principal components (PCs) and eigenvalues of the estimated covariance matrix is analyzed using well known tools in random matrix theory (RMT) and almost sure limits are studied. Simulations reveal that the analysis is valid for many distributions, not necessarily Gaussian, provided these samples are statistically independent. The theory is applied to the covariance matrices generated from the KASSPER database.

One of the major challenges in multivariate analysis is the estimation of population covariance matrix from the sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Both of these techniques try to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. Both methods have their respective pros and cons. In this paper, we propose an improved estimator which exploits the advantages of these techniques by taking optimally weighted convex combination of covariance matrices estimated by shrinkage transformation and a filter based on RMT. It is a generalized estimator which can adapt to changing sampling noise conditions by performing hyperparameter optimization. Using data from six of the world's biggest stock exchanges, we show that the proposed estimator outperforms the existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. The proposed estimator can be useful in a wide range of machine learning and signal processing applications. © 1994-2012 IEEE.

Estimating high dimensional covariance matrices: A new look at the Gaussian conjugate framework

We demonstrate that by adopting the new class of "expected-likelihood" (EL) covariance matrix estimates, instead of the traditional maximum-likelihood (ML) estimates, we can significantly enhance adaptive detection performance. These new estimates are found by searching within the properly parameterized class of admissible covariance matrices for the one that produces the likelihood ratio (LR) that is "closest possible" to the LR generated by the true (exact) covariance matrix.