Robust estimation of structured covariance matrix for heavy-tailed distributions

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.

This paper analyzes the performance of Tyler's M-estimator of the scatter matrix in elliptical populations. We focus on non-asymptotic performance analysis of Tyler's estimator. Given n samples of dimension p < n, we show that the squared Frobenius norm of the error of the inverse estimator is proportional to p2/(1−c2)2n with high probability, where c is the coherence coefficient of the properly scaled estimator. Under additional group symmetry conditions we improve the obtained bound, utilizing the inherent sparsity properties of group symmetry.

In the context of robust covariance matrix estimation, this work generalizes the shrinkage covariance matrix estimator introduced in [1, 2]. The shrinkage method is a way to improve and to regularize the Tyler's estimator [3, 4]. This paper proves that the shrinkage estimator does not require any trace constraint to be well-defined, as it has been previously developed in [1]. The existence and the uniqueness of this estimator, defined through a fixed point equation, is given according to the values of the shrinkage parameter. Moreover, it is shown that the shrinkage estimator converges to a particular Tyler's estimator when the shrinkage parameter tends to 0. Then, results on real STAP data show the improvement of using such a robust estimator to perform target detection in cases where the data sample size is less than the dimension.

An approach of regularizing Tyler's robust M-estimator of the co-variance matrix is proposed. We also provide an automatic choice of the regularization parameter in the high-dimensional regime. Simulations show its advantage over the sample covariance estimator and Tyler's M-estimator when data is heavy-tailed and the number of samples is small. Compared with the previous approaches of regularizing Tyler's M-estimator, our approach has a similar performance and a much simpler way of choosing the regularization parameter automatically.

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

Matrices attaining the minimum semidefinite rank of a chordal graph

In this paper, we propose an analytical solution for recovering a low-rank positive semi-definite (PSD) matrix from its rank-one measurements. We show that by utilizing a set of structured measurement vectors, we can analytically determine the null space of this low-rank PSD matrix. Based on the result, the PSD matrix can be efficiently recovered. Our analysis shows that the proposed method only requires <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(N-K)(2K+1) + K^{2}$</tex-math></inline-formula> measurements to guarantee exact recovery of the PSD matrix, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> respectively denote the dimension and the rank of the PSD matrix. Numerical results show that the proposed method achieves a considerable improvement over existing state-of-the-art methods in terms of both sample complexity and computational efficiency. Specifically, the proposed method helps improve the computational efficiency by an order of magnitude as compared with existing methods.

We address structured covariance estimation in elliptical distributions by assuming that the covariance is a priori known to belong to a given convex set, e.g., the set of Toeplitz or banded matrices. We consider the General Method of Moments (GMM) optimization applied to robust Tyler's scatter M-estimator subject to these convex constraints. Unfortunately, GMM turns out to be non-convex due to the objective. Instead, we propose a new COCA estimator-a convex relaxation which can be efficiently solved. We prove that the relaxation is tight in the unconstrained case for a finite number of samples, and in the constrained case asymptotically. We then illustrate the advantages of COCA in synthetic simulations with structured compound Gaussian distributions. In these examples, COCA outperforms competing methods such as Tyler's estimator and its projection onto the structure set.

We propose a new kernel entropy component analysis approach to object recognition with image set by fusing the information entropy. Since the geometry of Symmetric Positive Definite (SPD) matrices, we model the image set with its covariance matrix (nonsingular). Thus the object recognition with image set can be model as classifying problem on the riemannian manifold space. Given the proper kernel function derived from efficient metric for SPD matrices, points lie on manifold space represented by covariance matrices can be cast into high dimensional euclidean space. In this euclidean space, traditional methods used for dimensional reduction can be applied directly. Accounting for the measurability to information of the Renyi entropy, we fuse the information entropy to the dimensional reduction progress of our method. Similar to Kernel Principal Component Analysis (KPCA), the method accomplish data dimensionality reduction by projection onto a subset of entropy preserving KPCA axes. But this subset does not need to correspond to the top eigenvalues of the kernel matrix. The positive results of experiment on datasets demonstrated effectiveness of our method.

Thetdistribution is a useful extension of the normal distribution, which can be used for statistical modeling of data sets with heavy tails, and provides robust estimation. In this paper, in view of the advantages of Bayesian analysis, we propose a new robust coefficient estimation and variable selection method based on Bayesian adaptive Lassotregression. A Gibbs sampler is developed based on the Bayesian hierarchical model framework, where we treat thetdistribution as a mixture of normal and gamma distributions and put different penalization parameters for different regression coefficients. We also consider the Bayesiantregression with adaptive group Lasso and obtain the Gibbs sampler from the posterior distributions. Both simulation studies and real data example show that our method performs well compared with other existing methods when the error distribution has heavy tails and/or outliers.

We explore why many recently proposed robust estimation problems are efficiently solvable, even though the underlying optimization problems are non-convex. We study the loss landscape of these robust estimation problems, and identify the existence of ’generalized quasi-gradients’. Whenever these quasi-gradients exist, a large family of no-regret algorithms are guaranteed to approximate the global minimum; this includes the commonly used filtering algorithm. For robust mean estimation of distributions under bounded covariance, we show that any first-order stationary point of the associated optimization problem is an approximate global minimum if and only if the corruption level $\epsilon &amp;lt; 1/3$. Consequently, any optimization algorithm that approaches a stationary point yields an efficient robust estimator with breakdown point $1/3$. With carefully designed initialization and step size, we improve this to $1/2$, which is optimal. For other tasks, including linear regression and joint mean and covariance estimation, the loss landscape is more rugged: there are stationary points arbitrarily far from the global minimum. Nevertheless, we show that generalized quasi-gradients exist and construct efficient algorithms. These algorithms are simpler than previous ones in the literature, and for linear regression we improve the estimation error from $O(\sqrt{\epsilon })$ to the optimal rate of $O(\epsilon )$ for small $\epsilon $ assuming certified hypercontractivity. For mean estimation with near-identity covariance, we show that a simple gradient descent algorithm achieves breakdown point $1/3$ and iteration complexity $\tilde{O}(d/\epsilon ^2)$.

This paper addresses a simple, and yet effective approach to the design of block adaptive beamformers via parallel projection method (PPM), which is an extension of the classic projection onto convex set (POCS) method to inconsistent sets scenarios. The proposed approach begins with the construction of the convex constraint sets which the weight vector of the adaptive beamformer lies in. The convex sets are judiciously chosen to force the weights to possess some desirable properties or to meet some prescribed rules. Based on the minimum variance criterion and a fixed gain at the look direction, two constraint sets including the minimum variance constraint set and the gain constraint set are considered. For every input block of data, the weights of the proposed beamformer can then be determined by iteratively projecting the weight vector onto these convex sets until it converges. Furnished simulations show that the proposed beamformer provides superior performance compared with previous works in various scenarios but yet in general with lower computational overhead.

Bivariate optimization of pedalling rate and crank arm length in cycling

Recently, in the context of covariance matrix estimation, in order to improve as well as to regularize the performance of the Tyler's estimator [1] also called the Fixed-Point Estimator (FPE) [2], a "shrinkage" fixed-point estimator has been introduced in [3]. First, this work extends the results of [3,4] by giving the general solution of the "shrinkage" fixed-point algorithm. Secondly, by analyzing this solution, called the generalized robust shrinkage estimator, we prove that this solution converges to a unique solution when the shrinkage parameter $\beta$ (losing factor) tends to 0. This solution is exactly the FPE with the trace of its inverse equal to the dimension of the problem. This general result allows one to give another interpretation of the FPE and more generally, on the Maximum Likelihood approach for covariance matrix estimation when constraints are added. Then, some simulations illustrate our theoretical results as well as the way to choose an optimal shrinkage factor. Finally, this work is applied to a Space-Time Adaptive Processing (STAP) detection problem on real STAP data.