The application of improved covariance estimation to adaptive beamforming and detection

Many applications require precise estimates of high-dimensional covariance matrices. The standard estimator is the sample covariance matrix, which is conceptually simple, fast to compute and has favorable properties in the limit of infinitely many observations. The picture changes when the dimensionality is of the same order as the number of observations. In such cases, the eigenvalues of the sample covariance matrix are highly biased, the condition number becomes large and the inversion of the matrix gets numerically unstable.
\nA number of alternative estimators are superior in the high-dimensional setting, which include as subcategories structured estimators, regularized estimators and spectrum correction methods. In this thesis I contribute to all three areas. 
\nIn the area of structured estimation, I focus on models with low intrinsic dimensionality. I analyze the bias in Factor Analysis, the state-of-the-art factor model and propose Directional Variance Adjustment (DVA) Factor Analysis, which reduces bias and yields improved estimates of the covariance matrix.
\nAnalytical shrinkage of Ledoit and Wolf (LW-Shrinkage) is the most popular regularized estimator. I contribute in three aspects: first, I provide a theoretical analysis of the behavior of LW-Shrinkage in the presence of pronounced eigendirections, a case of great practical relevance. I show that LW-Shrinkage does not perform well in this setting and propose aoc-Shrinkage which yields significant improvements. Second, I discuss the effect of autocorrelation on LW-Shrinkage and review the Sancetta-Estimator, an extension of LW-Shrinkage to autocorrelated data. I show that the Sancetta-Estimator is biased and propose a theoretically and empirically superior estimator with reduced bias. Third, I propose an extension of shrinkage to multiple shrinkage targets. Multi-Target Shrinkage is not restricted to covariance estimation and allows for many interesting applications which go beyond regularization, including transfer learning. I provide a detailed theoretical and empirical analysis.
\nSpectrum correction approaches the problem of covariance estimation by improving the estimates of the eigenvalues of the sample covariance matrix. I discuss the state-of-the-art approach, Nonlinear Shrinkage, and propose a cross-validation based covariance (CVC) estimator which yields competitive performance at increased numerical stability and greatly reduced complexity and computational cost. On all data sets considered, CVC is on par or superior in comparison to the regularized and structured estimators.
\nIn the last chapter, I conclude with a discussion of the advantages and disadvantages of all covariance estimators presented in this thesis and give situation-specific recommendations. In addition, the appendix contains a systematic analysis of Linear Discriminant Analysis as a model application, which sheds light on the interdependency between the generative model of the data and various covariance estimators.

We consider the estimation of a Gaussian random vector x observed through a linear transformation H and corrupted by additive Gaussian noise with a known covariance matrix, where the covariance matrix of x is known to lie in a given region of uncertainty that is described using bounds on the eigenvalues and on the elements of the covariance matrix. Recently, two criteria for minimax estimation called difference regret (DR) and ratio regret (RR) were proposed and their closed form solutions were presented assuming that the eigenvalues of the covariance matrix of x are known to lie in a given region of uncertainty, and assuming that the matrices H T Cw -1 H and C x are jointly diagonalizable, where C w and C x denote the covariance matrices of the additive noise and of x respectively. In this work we present a new criterion for the minimax estimation problem which we call the generalized difference regret (GDR), and derive a new minimax estimator which is based on the GDR criterion where the region of uncertainty is defined not only using upper and lower bounds on the eigenvalues of the parameter's covariance matrix, but also using upper and lower bounds on the individual elements of the covariance matrix itself. Furthermore, the new estimator does not require the assumption of joint diagonalizability and it can be obtained efficiently using semidefinite programming. We also show that when the joint diagonalizability assumption holds and when there are only eigenvalue uncertainties, then the new estimator is identical to the difference regret estimator. The experimental results show that we can obtain improved mean squared error (MSE) results compared to the MMSE, DR, and RR estimators.

In the last few years, the demand of robust beamforming techniques has been increased significantly to serve numerous purposes. However, beamforming under low Signal to Noise Ratio (SNR) condition has always been a challenging task, which becomes more tedious in case of multiple sources. On the other hand, there are numerous factors such as steering vector error or similar estimation error problems that confine the efficiency of major beamforming techniques. Classical methods employ Eigen-Value Decomposition (EVD) and Singular Value Decomposition (SVD) over covariance matrix of the received signal; however their efficiency is limited under low SNR and multi-source condition, and perform inferiorly in case of steering vector errors. Such approaches are limited to achieve optimal Direction-of-Arrival (DOA) estimation that degrades the overall performance of the beamformer. Considering all these limitations, in this research, the focus is made on designing a robust DOA estimation model, followed by a novel beamformer for 8-element Uniform Linear Array (ULA). The simulation work consists two parts DOA estimation and beamforming. Unlike classical MUSIC and ESPRIT algorithms popular for DOA estimation our proposed model exploits the propagation method based subspace method where at first covariance matrix is obtained for the received signal, which is followed by normalization and Cross-Correlation matrix estimation. The obtained correlation matrix is processed for Generalized Eigen-Value Decomposition (GEVD) that efficiently obtained (orthogonal) subspace and SOI separation to yield DOA information. This approach enables more efficient DOA estimation even under low-SNR and steering vector error conditions. After obtaining DOA information, a novel adaptive beamformer is designed by using MVDR and FROST in parallel. DOA information from proposed model is given as input to the adaptive MVDR FROST beamforming model. Exploiting dynamic Signal-to-Interference ratio (SINR) by each beamformer our proposed method introduces an adaptation model that intends to achieve higher SINR value. MATLAB 2018a based simulation confirmed the robustness of the proposed model in terms of DOA estimation and beamforming results.

Using conditional bias in principal component analysis for the evaluation of joint influence on the eigenvalues of the covariance matrix

Robust adaptive beamforming via covariance matrix reconstruction with diagonal loading on interference sources covariance matrix

We introduce nonparametric regularization of the eigenvalues of a sample covariance matrix through splitting of the data (NERCOME), and prove that NERCOME enjoys asymptotic optimal nonlinear shrinkage of eigenvalues with respect to the Frobenius norm. One advantage of NERCOME is its computational speed when the dimension is not too large. We prove that NERCOME is positive definite almost surely, as long as the true covariance matrix is so, even when the dimension is larger than the sample size. With respect to the Stein’s loss function, the inverse of our estimator is asymptotically the optimal precision matrix estimator. Asymptotic efficiency loss is defined through comparison with an ideal estimator, which assumed the knowledge of the true covariance matrix. We show that the asymptotic efficiency loss of NERCOME is almost surely 0 with a suitable split location of the data. We also show that all the aforementioned optimality holds for data with a factor structure. Our method avoids the need to first estimate any unknowns from a factor model, and directly gives the covariance or precision matrix estimator, which can be useful when factor analysis is not the ultimate goal. We compare the performance of our estimators with other methods through extensive simulations and real data analysis.

In this article, we propose a network planning algorithm for IEEE802.11g, which minimises the total interferences between access points while maintaining a required quality of service (QoS), measured by users' signal to interference ratio (SIR). Three planning problems are formulated as integer linear programs. (1) A frequency assignment problem, which assigns frequencies to access points while minimising total interferences and ensuring users' SIR higher than a required threshold. (2) Assuming the optimal frequency assignment, a power management problem is modelled to assign five different power levels to access points while reducing the network total interference and ensuring the required QoS is met. (3) Given the frequency and power assignments, a load balancing problem is formulated to minimise the maximum number of users assigned per access point, taking into consideration users' QoS. Once the planning is done, new joining users are dynamically assigned to the less loaded acceptable access point. Based on realistic scenarios analysis, we show that the proposed planning achieves significant improvement in average user SIR and power savings while balancing the network. In fact, for one of the analysed scenarios the algorithm achieved 55% power savings while the SIR increased by 6 dB improving the QoS.

Satellite cognitive network is currently facing a lot of complex spectrum environment with a lot of interference, and the required user signal strength will change with a variety of external factors, which directly affects the above series of results obtained through the decision mechanism and it can not be well applied to satellite cognitive network. So a new blind detection algorithm based on maximum and minimum eigenvalues of sampling covariance matrix is proposed. In this algorithm, the ratio of the difference and sum of the maximum and minimum eigenvalues of the sampling covariance matrix is used as the perceptual decision quantity. Then, by introducing the latest results of the distribution of the maximum and minimum eigenvalues of the sampling covariance matrix in large dimensional random matrix, an effective decision threshold calculation method is designed. Compared with the classical eigenvalue detection algorithm, the new algorithm has the advantage of accurate calculation of perceptual decision threshold, and can effectively improve the detection performance and the reliability of decision results.

We consider the problem of approximating the set of eigenvalues of the covariance matrix of a multivariate distribution (equivalently, the problem of approximating the "population spectrum"), given access to samples drawn from the distribution. The eigenvalues of the covariance of a distribution contain basic information about the distribution, including the presence or lack of structure in the distribution, the effective dimensionality of the distribution, and the applicability of higher-level machine learning and multivariate statistical tools. We consider this fundamental recovery problem in the regime where the number of samples is comparable, or even sublinear in the dimensionality of the distribution in question. First, we propose a theoretically optimal and computationally efficient algorithm for recovering the moments of the eigenvalues of the population covariance matrix. We then leverage this accurate moment recovery, via a Wasserstein distance argument, to show that the vector of eigenvalues can be accurately recovered. We provide finite--sample bounds on the expected error of the recovered eigenvalues, which imply that our estimator is asymptotically consistent as the dimensionality of the distribution and sample size tend towards infinity, even in the sublinear sample regime where the ratio of the sample size to the dimensionality tends to zero. In addition to our theoretical results, we show that our approach performs well in practice for a broad range of distributions and sample sizes.

The evaluation of the Signal to Interference Ratio (SIR) in cellular networks is of primary importance for network dimensioning. For static studies, which evaluate cell capacity and coverage, as well as for dynamic studies, which consider arrivals and departures of mobile stations (MS), the SIR is always an important input. Contrary to most of the analytical works evaluating SIR, we assume in this paper that the MS is attached to the best server, i.e., to the base station (BS) from which it receives the highest power. This is a policy that is more realistic than the classical assumption that considers MSs to be attached to the nearest BS. The exact formulation of the SIR is however in this case uneasy to handle and numerical methods remain heavy. In this paper, we thus propose an approximate analytical study based on truncated lognormal distributions that provides very close results to Monte Carlo simulations.

The ensemble covariance matrix of a wide sense stationary signal spatially sampled by a full linear array is positive semi-definite and Toeplitz. However, the direct augmented covariance matrix of an augmentable sparse array is Toeplitz but not positive semi-definite, resulting in negative eigenvalues that pose inherent challenges in signal direction estimation problems. The positive eigenvalues-based covariance matrix for augmentable sparse arrays is robust but the matrix is unobtainable when all noise eigenvalues of the direct augmented matrix are negative, which is a possible case. To address this problem, we propose a robust covariance matrix for augmentable sparse arrays that leverages both positive and negative noise eigenvalues. The proposed covariance matrix estimate can be used in conjunction with subspace based algorithms such as multiple signal classification or adaptive beamformers such as minimum variance distortionless response beamformer to yield accurate signal direction estimates.

A robust bi-orthogonal/dynamically-orthogonal method using the covariance pseudo-inverse with application to stochastic flow problems

In a clutter-dominant target detection problem, an adaptive detector needs to either whiten or null the clutter prior to signal detection. To whiten clutter, it is required to estimate the covariance matrix of a received clutter, while a proper set or estimate of the parameters of clutter subspace model is required to null the received clutter. In this paper, we investigate the detection performance of a subspace-based detector and a sample covariance matrix(SCM)-based detector presented in [1] and [2], respectively, but for a mono-static radar system. Our simulation results show that the subspace-based detector not only can reach a predetermined probability of false alarm (fully nulling clutter) but also offers better detection performance as compared with its counterpart. In this case, the SCM-based detector only offers better detection performance for detecting targets with Doppler frequency resided in clutter region as compared with that of subspace-based one.

Traditional adaptive beamformers are sensitive to model mismatch and they have poor performances when the input signal-to-noise ratio (SNR) is close to or more than interference-to-noise ratio (INR). In this letter, a novel robust adaptive beamforming (RAB) algorithm based on twice interference-plus-noise covariance (INC) matrix reconstruction is proposed. Firstly, the desired signal (DS) steering vector is estimated by the alternating projection algorithm based on the antenna array geometry and angular sector prior information. Then, the proposed method obtains the eigenvalue corresponding to DS by correlation coefficient and takes the place of it by noise power to obtain the first INC matrix. Furthermore, the second precise covariance matrix is reconstructed based on the theoretical INC matrix expression. The interferences powers are estimated by the theoretical formulation of the interference covariance matrix (ICM), which is derived from the first INC matrix. The simulation results demonstrate that the proposed algorithm achieves good performance especially when the input SNR is high, and its performance is close to the optimal beamformer in the case of look direction error.

Robust adaptive beamforming (RAB) can be used to suppress interference signals while retaining the desired signals received by a sensor array. However, desired signal self-cancellation and model mismatch can affect RAB performance. In this paper, a novel interference-plus-noise covariance matrix (INCM) reconstruction method is proposed for RAB to solve these problems. The proposed method divides the desired signal into two ranges according to the input signal-to-noise ratio (SNR), namely low SNR and high SNR. In the low SNR range, INCM reconstruction directly uses the same sample covariance matrix as the sample matrix inversion (SMI) beamformer to retain the advantages of the traditional SMI algorithm. In the high SNR range, the eigenvalues of the sample covariance matrix are used to estimate the interference power and noise power. The optimized interference steering vector (SV) is obtained by solving a quadratic convex optimization problem in an interference subspace. The INCM is reconstructed from the interference SVs, interference power, and noise power. The reconstructed INCM is then used to correct the desired signal SV via maximizing the beamformer output power. This is achieved by solving a quadratically constrained quadratic programming (QCQP) problem. Analysis and simulation results are presented which demonstrate that the proposed method performs well under a variety of mismatch conditions.