Optimal Design of the Adaptive Normalized Matched Filter Detector Using Regularized Tyler Estimators

Many applications require precise estimates of high-dimensional covariance matrices. The standard estimator is the sample covariance matrix, which is conceptually simple, fast to compute and has favorable properties in the limit of infinitely many observations. The picture changes when the dimensionality is of the same order as the number of observations. In such cases, the eigenvalues of the sample covariance matrix are highly biased, the condition number becomes large and the inversion of the matrix gets numerically unstable.
\nA number of alternative estimators are superior in the high-dimensional setting, which include as subcategories structured estimators, regularized estimators and spectrum correction methods. In this thesis I contribute to all three areas. 
\nIn the area of structured estimation, I focus on models with low intrinsic dimensionality. I analyze the bias in Factor Analysis, the state-of-the-art factor model and propose Directional Variance Adjustment (DVA) Factor Analysis, which reduces bias and yields improved estimates of the covariance matrix.
\nAnalytical shrinkage of Ledoit and Wolf (LW-Shrinkage) is the most popular regularized estimator. I contribute in three aspects: first, I provide a theoretical analysis of the behavior of LW-Shrinkage in the presence of pronounced eigendirections, a case of great practical relevance. I show that LW-Shrinkage does not perform well in this setting and propose aoc-Shrinkage which yields significant improvements. Second, I discuss the effect of autocorrelation on LW-Shrinkage and review the Sancetta-Estimator, an extension of LW-Shrinkage to autocorrelated data. I show that the Sancetta-Estimator is biased and propose a theoretically and empirically superior estimator with reduced bias. Third, I propose an extension of shrinkage to multiple shrinkage targets. Multi-Target Shrinkage is not restricted to covariance estimation and allows for many interesting applications which go beyond regularization, including transfer learning. I provide a detailed theoretical and empirical analysis.
\nSpectrum correction approaches the problem of covariance estimation by improving the estimates of the eigenvalues of the sample covariance matrix. I discuss the state-of-the-art approach, Nonlinear Shrinkage, and propose a cross-validation based covariance (CVC) estimator which yields competitive performance at increased numerical stability and greatly reduced complexity and computational cost. On all data sets considered, CVC is on par or superior in comparison to the regularized and structured estimators.
\nIn the last chapter, I conclude with a discussion of the advantages and disadvantages of all covariance estimators presented in this thesis and give situation-specific recommendations. In addition, the appendix contains a systematic analysis of Linear Discriminant Analysis as a model application, which sheds light on the interdependency between the generative model of the data and various covariance estimators.

In the non-Gaussian clutter modeled as independent and identically distributed spherically invariant random vectors, three estimators of sample covariance matrix (SCM), normalized sample covariance matrix (NSCM) and the corresponding recursive estimator (NSCM-RE) are analyzed. Based on the uniform theorem, three corresponding adaptive normalized matched filters (ANMF) are evaluated from the standpoint of constant false alarm rate (CFAR) property. The theoretical results demonstrate that the SCM-ANMF is only CFAR to the normalized clutter covariance matrix (NCCM); the NSCM-ANMF is only CFAR to the clutter power level; and the NSCM-RE-ANMF with finite number of iterations is still not CFAR to the NCCM. To ensure CFAR property of ANMF, an adaptive estimator (AE) is devised. Moreover, with AE as the initialization matrix for the iterations, the AE-RE is proposed. With finite number of iterations, the corresponding AE-REANMF guarantees CFAR property to both of the NCCM and the clutter power level. Finally, the performance assessment conducted by Monte Carlo simulation confirms the effectiveness of the proposed detectors.

One of the major challenges in multivariate analysis is the estimation of population covariance matrix from the sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Both of these techniques try to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. Both methods have their respective pros and cons. In this paper, we propose an improved estimator which exploits the advantages of these techniques by taking optimally weighted convex combination of covariance matrices estimated by shrinkage transformation and a filter based on RMT. It is a generalized estimator which can adapt to changing sampling noise conditions by performing hyperparameter optimization. Using data from six of the world's biggest stock exchanges, we show that the proposed estimator outperforms the existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. The proposed estimator can be useful in a wide range of machine learning and signal processing applications. © 1994-2012 IEEE.

Multiple input multiple output (MIMO) radars have been intensively studied for their superior performance merits compared to conventional radars including improved spatial resolution and parameter estimation. To achieve these merits, the transmitted waveforms from the MIMO transmitting antennas should be mutually orthogonal. Herein, the orthogonality is attained in the Doppler domain, namely Doppler division multiple access (DDMA), as it enjoys the simplest structure among other orthogonality schemes. Heretofore, the research in MIMO radars has focused on optimizing MIMO radar performance from different aspects including efficient waveform design, antenna beam-forming, and adaptive processing techniques. However, MIMO radars have been also a target of deliberate jamming scenarios. In this paper, the focus is turned to the evaluation of the detection performance of DDMA MIMO radars in jamming environments by creating a two-sided engagement scenario. On the jammer side, a jamming technique is proposed, Doppler-range block jamming (DRBJ), to deny the detection of a DDMA MIMO radar equipped with a moving target detector (MTD). On the radar side, a comparative detection performance analysis of two different MIMO radar detectors, normalized adaptive matched filter (NAMF) and fast 3D constant false alarm rate (CFAR), is conducted using Monte Carlo simulations. This analysis unveils divergent design philosophies of the two detectors, with each prioritizing distinct performance objectives.

In radar target detection, the adaptive normalised matched filter (ANMF) detector is often used to detect moving targets in sea and land clutter environment. Its performance depends on estimation of clutter covariance matrix. The sample covariance matrix (SCM) estimator has been thoroughly analysed for independent and identically distributed Gaussian clutter vectors. Many radar systems on service use range-oversampling receivers. Referred to the range-oversampling model in the weather and oceanic radars, the spatial correlation model of range-oversampled clutter vectors is established, the error of the SCM estimator is analysed, and the range-oversampling gain, relevant to the oversampling factor and the receiver's bandwidth, is derived. The experiments using raw radar clutter data are made to verify the range-oversampling gain, showing that the range oversampling improves the performance of the ANMF detector.

Covariance matrix estimation is one of the most important problems in statistics. To accommodate the complexity of modern datasets, it is desired to have estimation procedures that not only can incorporate the structural assumptions of covariance matrices, but are also robust to outliers from arbitrary sources. In this paper, we define a new concept called matrix depth and then propose a robust covariance matrix estimator by maximizing the empirical depth function. The proposed estimator is shown to achieve minimax optimal rate under Huber’s $\\varepsilon$-contamination model for estimating covariance/scatter matrices with various structures including bandedness and sparsity.

This paper develops an improved covariance matrix estimator in the mean-variance optimisation setting. Well-known problems with the sample covariance matrix are that it is singular when the number of observations is less than the number of assets, and can be nearly singular when the number of observations exceeds the number of assets. Therefore, using the sample covariance matrix as an input in mean-variance optimisation can result in unreasonable optimal portfolios and badly biased estimates of Sharpe ratios. We address this problem by imposing structure on the estimated covariance matrix by putting constraints on the Sharpe ratio, asset return variances, and the variance of the global minimum variance portfolio. We show that the constrained maximum likelihood estimator (CMLE) performs better than the sample covariance matrix. Moreover, when the shrinkage approach is applied to the CMLE and single index covariance matrix, it performs better than the shrinkage estimator of Ledoit and Wolf (2004).

Journal Article Nonparametric estimation of large covariance matrices of longitudinal data Get access Wei Biao Wu, Wei Biao Wu Search for other works by this author on: Oxford Academic Google Scholar Mohsen Pourahmadi Mohsen Pourahmadi Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 90, Issue 4, December 2003, Pages 831–844, https://doi.org/10.1093/biomet/90.4.831 Published: 01 December 2003

Constant false alarm rate (CFAR), a target detection method commonly used in the radar systems, has an inconsistent performance against various environments. For improving the radar detectability, this paper proposes a novel scheme of radar target detection using neural network-based adaptive selection CFAR. The proposed method employs cell-averaging, ordered-statistic, greatest-of and smallest-of CFAR thresholds as the basis of references. The pattern of those threshold values combined with the cell under test signal value will be identified and classified by the neural network to compute the raw threshold. Then, the final threshold is selected depending on the nearest value between raw and four referenced CFARs. The performance of the proposed method is examined against three possible cases of the radar systems including homogeneous background, multiple targets and clutter boundary. The result of this research shows that the proposed method outperforms the classical CFARs due to the adaptive selection algorithm can select properly among referenced CFARs against the given cases particularly in the homogeneous and multiple target environments.

Accurate estimation of covariance matrix is the basis of effective development of high-dimensional optimal portfolio. Random matrix theory provides an effective means to improve the estimation of high-dimensional covariance matrix. Based on the empirical research on the export excess return rate of traditional cultural industry, the result is a combination with higher precision and lower risk. Based on relevant data, this paper constructs a stochastic matrix theoretical model to analyze the impact of human capital, new fixed asset investment, independent innovation and technology purchase, financing sources, and other factors on high-tech industry export. Based on the results of empirical analysis, some policy suggestions are put forward, such as increasing r&d investment, improving enterprises’ innovation ability, and introducing core scientific and technological talents to promote the export scale growth of high-tech industries. Model comparative analysis is made on the influence of traditional industries’ concentrated export, foreign trade environment, and cost advantage on the export of high-tech products. Through comparative analysis of the influencing factors of the export of high-tech products in the two periods before and after the export crisis of traditional culture, the new changes of the influencing factors of the export of high-tech products are analyzed. In addition, through the change of the internal and external environment of the current high-tech product export, the reasons for the new change of the influencing factors are discussed.

This letter introduces a stochastic partially homogeneous model for adaptive signal detection. In this model, the disturbance covariance matrix of training signals, ${\bf R}$ , is assumed to be a random matrix with some a priori information, while the disturbance covariance matrix of the test signal, ${\bf R}_{0}$ , is assumed to be equal to $\lambda{\bf R}$ , i.e., ${\bf R}_{0}=\lambda{\bf R}$ . On one hand, this model extends the stochastic homogeneous model by introducing an unknown power scaling factor $\lambda$ between the test and training signals. On the other hand, it can be considered as a generalization of the standard partially homogeneous model to the stochastic Bayesian framework, which treats the covariance matrix as a random matrix. According to the stochastic partially homogeneous model, a scale-invariant generalized likelihood ratio test (GLRT) for the adaptive signal detection is developed, which is a knowledge-aided version of the well-known adaptive coherence estimator (ACE). The resulting knowledge-aided ACE (KA-ACE) employs a colored loading step utilizing the a priori knowledge and the sample covariance matrix. Various simulation results and comparison with respect to other detectors confirm the scale-invariance and the effectiveness of the KA-ACE.

The problem of Krylov subspace estimation based on the sample covariance matrix is addressed. The focus is on signal processing applications where the Krylov subspace is defined by the unknown second-order statistics of the observed samples and the signature vector associated with the desired parameter. In particular, the consistency of traditionally optimal sample estimators is revised and analytically characterized under a practically more relevant asymptotic regime, according to which not only the number of samples but also the observation dimension grow without bound at the same rate. Furthermore, an improved construction of a class of Krylov subspace estimators is proposed based on the generalized consistent estimation of a set of vector-valued power functions of the observation covariance matrix. To that effect, an extension of some known results from random matrix theory on the estimation of certain spectral functions of the covariance matrix to the convergence of not only the covariance eigenspectrum but also the associated eigensubspaces is provided. A new family of estimators is derived that generalizes conventional implementations by proving to be consistent for observations of arbitrarily high dimension. The proposed estimators are shown to outperform traditional constructions via the numerical evaluation of the solution to two fundamental problems in sensor array signal processing, namely the problem of estimating the power of an intended source and the estimation of the principal eigenspace and dominant eigenmodes of a structured covariance matrix.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

We present the problem of polarization diversity detection in compound-Gaussian clutter with unknown covariance matrix. To this end we assume that a set of secondary data, free of signal components and with the same covariance structure of the cell under test, is available. Due to the lack of a uniformly most powerful (UMP) detector we resort to a design procedure based upon the Rao and the Wald tests. Specifically we first derive the Rao and the Wald tests assuming that the covariance matrix is known, and then we substitute into the derived decision rules a suitable estimate of the clutter covariance. Interestingly, the newly proposed detectors share the constant false alarm rate (CFAR) property with respect to the texture statistical characterization. Moreover simulation results have shown that the Wald test based detector ensures a performance level higher than the Rao test. We have also conducted a further performance analysis, in the presence of real clutter data and in comparison with the previously proposed generalized likelihood ratio test (GLRT) based receivers, which highlights that, in general, the Wald test receiver outperforms its counterparts. Finally, since the newly proposed decision rules as well as the previously designed GLRTs do not ensure the CFAR property with respect to the clutter covariance matrix, we have developed a sensitivity analysis on the probability of false alarm (P/sub fa/), based on simulated clutter with covariance matrix estimated from real radar data. The results have shown that (P/sub fa/) is only slightly affected by variations in the clutter correlation properties and hence the CFARness is substantially achieved.

Arrays can be designed as a full array of sensors or a sparse array of sensors with an equivalent aperture. Reductions in cost, size, weight and power consumption are important factors to consider when designing an array of sensors. Sparse arrays can provide a reduction in these factors. It is important to observe the performance of sparse arrays in relation to full arrays to see if they are able to achieve similar performance. The ability for the full and sparse arrays of sensors to accurately estimate the direction of arrival (DOA) of plane waves impinging them is observed in this thesis. The Mean-squared error (MSE) of the DOA estimation is measured against the Creamer-Rao bound (CRB) as a benchmark for optimal performance for an unbiased estimator. We derive an optimal way to remove sensors for one signal impinging on an array by finding the minimum CRB. This method considers all the unique combinations a sparse array could be assembled for a given number of sensors and aperture. Parameters such as number of sensors, sensor locations, signal-to-noise ratio (SNR), number of snapshots, and signal correlation affect the MSE and CRB. Through observing trends in the MSE as a result of changing these parameters we can come to conclusions to how full and sparse arrays perform as a result of adjusting these parameters. The MSE of the DOA of the full and sparse arrays is obtained through a covariance estimation from subspace-based DOA algorithms. The sample covariance and the diagonal-averaged sample covariance matrix performances are compared. The sample covariance matrix for a full array provides the best estimations at high SNR while the diagonal-averaging method provides a more accurate estimation at low SNR. The sparse array diagonal-averaged covariance matrix estimation performs similarly to the diagonal-averaged covariance matrix of the full array, but is still slightly worse due to missing information. How sparsity affects performance was also observed. As sparsity increases for an array, the performance change is non-linear for the MSE and CRB. Increases in signal correlation cause an increase in the overall MSE and a decrease in the overall CRB for both sparse and full arrays.