A Class of Structured High-Dimensional Dynamic Covariance Matrices.

Statistical inference for high-dimensional data is increasingly essential due to the prevalence of big data. While recent advancements have improved testing linear hypotheses for high-dimensional mean vectors, exploration of testing linear hypotheses for high-dimensional covariance matrices remains limited. This article focuses on a general linear hypothesis testing (GLHT) problem for high-dimensional covariance matrices, encompassing various specific cases such as testing the equality of variances, testing for a given covariance matrix, assessing the homogeneity of covariance matrices, and examining linear combinations of covariance matrices. We propose and study a new test statistic for the GLHT problem on high-dimensional covariance matrices. Under certain regularity conditions and the null hypothesis, we demonstrate that the test statistic shares the same limiting distribution as a Chi-squared-type mixture. This mixture’s distribution can be accurately approximated using a three-cumulant matched Chi-squared-approximation approach. Additionally, we establish the asymptotic power of the proposed test against some local alternatives. Simulation studies and a real-world financial data analysis demonstrate that the proposed test outperforms several competitors in terms of size control.

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when $p\gg n$. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a $\sqrt{\log n}$ term.

We suggest a method for estimating a covariance matrix that can be represented as a sum of a sparse low-rank matrix and a diagonal matrix. Our formulation is based on penalized quadratic loss, which is a convex problem that can be solved via incremental gradient and proximal method. In contrast to other spiked covariance matrix estimation approaches that are related to principal component analysis and factor analysis, our method has a simple formulation and does not constrain entire rows and columns of the matrix to be zero. We further discuss a penalized entropy loss method that is nevertheless nonconvex and necessitates a majorization-minimization algorithm in combination with the incremental gradient and proximal method. We carry out simulations to demonstrate the finite-sample properties focusing on high-dimensional covariance matrices. Finally, the proposed method is illustrated using a gene expression data set.

Estimating high dimensional covariance matrices: A new look at the Gaussian conjugate framework

A sample \( X_{1},\ldots ,X_{n} \) consisting of independent identically distributed random vectors in \( \mathbb {R}^{p} \) with zero mean and covariance matrix \( \mathbf {\Sigma }\) is considered. The recovery of spectral projectors of high-dimensional covariance matrices from a sample of observations is a key problem in statistics arising in numerous applications. This chapter describes a bootstrap procedure for constructing confidence sets for the spectral projector \( \mathbf {P}_{r} \) related to rth eigenvalue of the covariance matrix \(\mathbf {\Sigma }\) from given data on the base of corresponding spectral projector \(\widehat{\mathbf {P}}_{r}\) of the sample covariance matrix \(\widehat{\mathbf {\Sigma }}\). This approach does not use the asymptotical distribution of \( \Vert \mathbf {P}_{r} - \widehat{\mathbf {P}}_{r} \Vert _{2} \) and does not require the computation of its moment characteristics. The performance of the bootstrap approximation procedure is analyzed.

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

We address the problem of estimating high-dimensional covariance matrices (CM) for the explicit purpose of supervised anomaly detection, in the case when the number n of data points is lower than their dimensionality p. This is increasingly common with the emergence of the Internet of Things that makes it possible to collect data from many sensors simultaneously, resulting in very high-dimensional data points. When we attempt to perform anomaly detection for such data by modeling the normal behavior of the system by means of a multivariate Gaussian distribution, and n < p, the sample CM is singular, and cannot be used directly without some form of regularization. In contrast to existing methods for CM regularization that aim to fit the training data accurately, we propose a regularization algorithm for CM estimation that directly aims to maximize the area under the resulting receiver-operator characteristic (AUROC) for the ultimate decision problem that needs to be solved: anomaly detection. Experiments on test problems demonstrate the ability of the proposed algorithm to find CM estimates significantly better at anomaly detection than existing estimation methods that are unaware of the decision task that the CMs they produce will be used in.

In this paper we present different approaches to structuring covariance matrices within Statistical classifiers. This is motivated by the fact that the use of full covariance matrices is infeasible in many applications. On the one hand, this is due to the high number of model Parameters that have to be estimated, on the other hand the computational complexity of a classifier based on full covariance matrices is very high. We propose the use of diagonal and band-matrices to replace full covariance matrices and we also show that computation of tangent distance is equivalent to using a structured covariance matrix within a Statistical classifier.

In this paper the model of experiment in which several characteristics are measured or the measurements are repeated in time, is considered. It is assumed that the covariance matrix of the characteristics/time points is unknown, but has the banded Toeplitz structure. The problem of estimating such a covariance structure was considered by e.g. Cui et al. (2016), Filipiak et al. (2018). We discuss two methods that aim to deal with singularity and numerical ill-conditioning of the estimators of covariance matrix under high-dimensional regime. The method of estimation of banded Toeplitz covariance matrix based on a shrinking, described by Ledoit and Wolf (2004) for unstructured covariance matrices is proposed. The estimators obtained with the use of the proposed method are then compared with the estimators obtained by the projection of sample covariance matrix onto the asymptotic cone of the nonnegative definite Toeplitz matrices (Filipiak et al. 2018). For this purpose simulation studies concerning bias and risk are conducted for several sets of parameters.

We propose a simple forward adaptive banding method for estimating large covariance matrices using the modified Cholesky decomposition. This approach requires the fitting of a prespecified set of models due to the adaptive banding structure and can be efficiently implemented. Aside from its computational attractiveness, we propose a novel Bayes information criterion that gives consistent model selection for estimating high dimensional covariance matrices. The method compares favourably to its competitors in simulation study.

A robust test for sphericity of high-dimensional covariance matrices

Testing the equality of two or multiple correlation or covariance matrices is an important problem in biology, finance and many other areas. High dimensionality, where the number of features is much larger than the sample size, causes conventional procedures to perform poorly, as they are often based on limiting distributions of test statistics in the classical large sample size setting. Moreover, their performance is often contingent on whether the matrix of differences is sparse or dense, while such information is rarely available. In this article, we develop a new family of permutation testing procedures to tackle these challenges. The introduced tests are demonstrated to outperform many other competing procedures in terms of size control and power under various settings. In particular, using our variance-stabilizing transformation, the proposed methods provide the best performance for testing correlation or covariance matrix differences in both sparse and dense settings. We establish non-asymptotic guarantees on the power of our test, which ensure its reliability for sparse and dense differential correlation matrices. Through the analysis of gene-expression and brain imaging data, we showcase the high power and accurate size control of our test in high-dimensional statistical applications.

As the field of big data continues to evolve, there is an increasing necessity to evaluate the equality of multiple high-dimensional covariance matrices. Many existing methods rely on approximations to the null distribution of the test statistic or its extreme-value distributions under stringent conditions, leading to outcomes that are either overly permissive or excessively cautious. Consequently, these methods often lack robustness when applied to real-world data, as verifying the required assumptions can be arduous. In response to these challenges, we introduce a novel test statistic utilizing the normal-reference approach. We demonstrate that the null distribution of this test statistic shares the same limiting distribution as a chi-square-type mixture under certain regularity conditions, with the latter reliably estimable from data using the three-cumulant matched chi-square-approximation. Additionally, we establish the asymptotic power of our proposed test. Through comprehensive simulation studies and real data analysis, our proposed test demonstrates superior performance in terms of size control compared to several competing methods.