Knowledge-Aided Adaptive Coherence Estimator in Stochastic Partially Homogeneous Environments

We present two methods to improve three hyperspectral stochastic algorithms for target detection; the algorithms are the constrained energy minimization, the generalized likelihood ratio test, and the adaptive coherence estimator. The original algorithms rely solely on spectral information and do not use spatial information; this usage is normally justified in subpixel target detection, since the target size is smaller than the size of a pixel. However, we found that since the background (and the false alarms) may be spatially correlated and the point spread function can distribute the energy of a point target between several neighboring pixels, the implementation of spatial filtering algorithms considerably improved target detection. Our first improvement used the local spatial mean and covariance matrices, which take into account the spatial local mean instead of the global mean. While this concept has been found in the literature, the effect of its implementation in both the estimated mean and the covariance matrix is examined quantitatively here. The second was based on the fact that the effect of a target of physical subpixel size will extend to a cluster of pixels. We tested our algorithms by using the data set and scoring methodology of the Rochester Institute of Technology Target Detection Blind Test project. The results showed that both spatial methods independently improved the basic spectral algorithms mentioned above, and when the two methods were used together, the results were even better.

Adaptive detectors for mismatched subspace target in clutter with lognormal texture

As it is not suffered by the well-known noise uncertainty problem of energy detection (ED), generalized likelihood ratio test (GLRT) based spectrum sensing schemes for a single secondary user node have recently been extensively addressed. In this paper, we extend the existing research to an optimal GLRT based cooperative spectrum sensing scheme in a heterogeneous network scenario, where each node can be equipped with different number of antennas and have different signal processing capacity, such as sampling rate. By considering the discrepancy of different secondary user nodes' sensing reliability due to different hardware configuration, we demonstrate that the optimal cooperative spectrum sensing scheme based on GLRT is a linear combination of each node's local test statistic. Moreover, the local test statistic is the logarithm of the arithmetic-to-geometric mean ratio of the eigenvalues associated with the sample covariance matrix. Furthermore, the optimal combining coefficient is a simple function of the numbers of the antennas and the received signal samples at each secondary user node, which can be intrinsically provided by each secondary user node without any prior information. Numerical results show that the proposed GLRT based cooperative spectrum sensing scheme performs better than the practical ED based scheme, i.e., equal gain combination based ED, especially under the case where the sensing reliability discrepancy of each secondary user node is significant.

This paper considers the problem of knowledge-aided space-time adaptive processing (STAP) in nonhomogeneous environments, where the covariance matrices of the training and test signals are assumed random and different from each other. A Bayesian detector is proposed by incorporating some a priori knowledge of the disturbance covariance matrices, and exploring their inherent block-Toeplitz structure. Specifically, the block-Toeplitz structure of the covariance matrix allows us to model the training signals as a multichannel auto-regressive (AR) process. The resulting detector is referred to as the Bayesian parametric adaptive matched filter (B-PAMF) which, compared with nonparametric Bayesian detectors, entails a lower training requirement and alleviates the computational complexity. Numerical results show that the proposed B-PAMF detector outperforms the standard PAMF test in nonhomogeneous environments.

One of the major challenges in multivariate analysis is the estimation of population covariance matrix from the sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Both of these techniques try to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. Both methods have their respective pros and cons. In this paper, we propose an improved estimator which exploits the advantages of these techniques by taking optimally weighted convex combination of covariance matrices estimated by shrinkage transformation and a filter based on RMT. It is a generalized estimator which can adapt to changing sampling noise conditions by performing hyperparameter optimization. Using data from six of the world's biggest stock exchanges, we show that the proposed estimator outperforms the existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. The proposed estimator can be useful in a wide range of machine learning and signal processing applications. © 1994-2012 IEEE.

Detecting and identifying targets or objects that are present inhyperspectral ground images are of great interest. Applicationsinclude land and environmental monitoring, mining, military, civilsearch-and-rescue operations, and so on. We propose and analyze anextremely simple and efficient idea for template matching based on$l_1$ minimization. The designed algorithm can be applied inhyperspectral classification and target detection. Synthetic imagedata and real hyperspectral image (HSI) data are used to assess theperformance, with comparisons to other approaches, e.g. spectralangle map (SAM), adaptive coherence estimator (ACE),generalized-likelihood ratio test (GLRT) and matched filter. Wedemonstrate that this algorithm achieves excellent results with bothhigh speed and accuracy by using Bregman iteration.

This paper addresses improvements on the design of the adaptive normalized matched filter (ANMF) for radar detection. It is well acknowledged that the estimation of the noise-clutter covariance matrix is a fundamental step in adaptive radar detection. In this paper, we consider regularized estimation methods that force by construction the eigenvalues of the covariance estimates to be greater than a positive regularization parameter $\rho$ . This makes them more suitable for high-dimensional problems with a limited number of secondary data samples than traditional sample covariance estimates. The motivation behind this paper is to understand the effect and properly set the value of $\rho$ that improves estimate conditioning while maintaining a low-estimation bias. More specifically, we consider the design of the ANMF detector for two kinds of regularized estimators, namely the regularized sample covariance matrix, the regularized Tyler estimator (RTE). The rationale behind this choice is that the RTE is efficient in mitigating the degradation caused by the presence of impulsive noises while inducing little loss when the noise is Gaussian. Based on asymptotic results brought by recent tools from random matrix theory, we propose a design for the regularization parameter that maximizes the asymptotic detection probability under constant asymptotic false alarm rates. Provided simulations support the efficiency of the proposed method, illustrating its gain over conventional settings of the regularization parameter.

The problem of Krylov subspace estimation based on the sample covariance matrix is addressed. The focus is on signal processing applications where the Krylov subspace is defined by the unknown second-order statistics of the observed samples and the signature vector associated with the desired parameter. In particular, the consistency of traditionally optimal sample estimators is revised and analytically characterized under a practically more relevant asymptotic regime, according to which not only the number of samples but also the observation dimension grow without bound at the same rate. Furthermore, an improved construction of a class of Krylov subspace estimators is proposed based on the generalized consistent estimation of a set of vector-valued power functions of the observation covariance matrix. To that effect, an extension of some known results from random matrix theory on the estimation of certain spectral functions of the covariance matrix to the convergence of not only the covariance eigenspectrum but also the associated eigensubspaces is provided. A new family of estimators is derived that generalizes conventional implementations by proving to be consistent for observations of arbitrarily high dimension. The proposed estimators are shown to outperform traditional constructions via the numerical evaluation of the solution to two fundamental problems in sensor array signal processing, namely the problem of estimating the power of an intended source and the estimation of the principal eigenspace and dominant eigenmodes of a structured covariance matrix.

This paper addresses jamming attack detection issue in a millimeter Wave (mmWave) massive MIMO system under noise uncertainty. Specifically, we apply the generalized likelihood ratio test (GLRT) to develop a one-step GLRT scheme for detecting jamming attack under the homogeneous and partially homogeneous noise environments, and exploit training data to estimate the unknown noise statistical information and replace it with resulting estimation in deriving GLRT procedure to obtain adaptive GLRT detection scheme. We also design a two-step GLRT scheme, where we first assume the unknown noise statistical information is known, and derive GLRT based on test data, and then replace it by the sample covariance matrix based on training data only to achieve a fully adaptive jamming attack detector. With the help of statistical signal subspace analysis and composite hypothesis testing theories, we further examine the statistical distributions of the jamming attack detection schemes and present the closed-form expressions of false alarm and detection probabilities for the proposed schemes under different noise environments. Finally, we implement extensive simulations to validate the theoretical results and evaluate the detection efficiency under various parameters.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

In this two-part paper, we consider the problem of adaptive multidimensional/multichannel signal detection in Gaussian noise with unknown covariance matrix. The test data (primary data) is assumed as a collection of sample vectors, arranged as the columns of a rectangular data array. The rows and columns of the signal matrix are both assumed to lie in known subspaces, but with unknown coordinates. Due to this feature of the signal structure, we name this kind of signal as the double subspace signal. Part I of this paper focuses on the adaptive detection in homogeneous environments, while Part II deals with the adaptive detection in partially homogeneous environments. Precisely, in this part, we derive the generalized likelihood ratio test (GLRT), Rao test, Wald test, as well as their two-step variations, in homogeneous environments. Three types of spectral norm tests (SNTs) are also introduced. All these detectors are shown to possess the constant false alarm rate (CFAR) property. Moreover, we discuss the differences between them and show how they work. Another contribution is that we investigate various special cases of these detectors. Remarkably, some of them are well-known existing detectors, while some others are still new. At the stage of performance evaluation, conducted by Monte Carlo simulations, both matched and mismatched signals are dealt with. For each case, more than one scenario is considered.

The adaptive matched filter, like many other adaptive detection schemes, uses in its test statistic the data under test whitened by the sample covariance matrix S of the training samples. Actually, it is a generalized likelihood ratio test (GLRT) based on the conditional (i.e., for given S) distribution of the adaptively whitened data. In this letter, we investigate detection of a Gaussian rank-one signal using the marginal (unconditional) distribution of the adaptively whitened data. A first contribution is to derive the latter and to show that it only depends on a scalar parameter, namely the signal to noise ratio. Then, a GLRT is formulated from this unconditional distribution and shown to have the constant false alarm rate property. We show that it bears close resemblance with the plain GLRT based on the whole data set (data under test and training samples). The new detector performs as well as the plain GLRT and even better with multiple cells under test and low training sample support.

Persymmetric adaptive detection of distributed targets in compound-Gaussian sea clutter with Gamma texture

This paper deals with the adaptive detection problem of multidimensional subspace signals in compound-Gaussian clutter. Owing to the limited number of training data, this paper exploits the persymmetric structural information and uses the two-step generalized likelihood ratio test (GLRT) approach to propose a persymmetric adaptive detector. Theoretical analysis prove that the detector ensures the constant false alarm rate (CFAR) property with respect to the texture and covariance matrix. In addition, numerical simulation shows that the proposed detector has better performance than the traditional detectors. Meanwhile, the effects of the covariance matrix estimation approaches, signal subspace dimension and mismatched signal subspace on the detection performance is assessed.

In the non-Gaussian clutter modeled as independent and identically distributed spherically invariant random vectors, three estimators of sample covariance matrix (SCM), normalized sample covariance matrix (NSCM) and the corresponding recursive estimator (NSCM-RE) are analyzed. Based on the uniform theorem, three corresponding adaptive normalized matched filters (ANMF) are evaluated from the standpoint of constant false alarm rate (CFAR) property. The theoretical results demonstrate that the SCM-ANMF is only CFAR to the normalized clutter covariance matrix (NCCM); the NSCM-ANMF is only CFAR to the clutter power level; and the NSCM-RE-ANMF with finite number of iterations is still not CFAR to the NCCM. To ensure CFAR property of ANMF, an adaptive estimator (AE) is devised. Moreover, with AE as the initialization matrix for the iterations, the AE-RE is proposed. With finite number of iterations, the corresponding AE-REANMF guarantees CFAR property to both of the NCCM and the clutter power level. Finally, the performance assessment conducted by Monte Carlo simulation confirms the effectiveness of the proposed detectors.