An MCD-based local ACE algorithm for hyperspectral imagery target detection

In this paper we present a new methodology for automated target detection and identification in hyperspectral imagery. The standard paradigm for target detection in hyperspectral imagery is to run a detection algorithm, typically statistical in nature, and visually inspect each high-scoring pixel to decide whether it is a true detection or a false alarm. Detection filters have constant false alarm rates (CFARs) approaching 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> , but these can still result in a large number of false alarms given multiple images and a large number of target materials. Here we introduce a new methodology for target detection and identification in hyperspectral imagery in conjunction with a data-driven target detection algorithm that shows promise for hard targets. The result is a greatly reduced false alarm rate and a practical methodology for aiding an analyst in quantitatively evaluating detected pixels.

Aims: The aim of this study is to investigate the impact of thresholds on the detection of outliers by comparing the performance of two estimators, namely the minimum covariance determinant (MCD) and minimum regularized covariance determinant (MRCD), at different sample sizes. The study uses simulated data generated from the standard normal distribution to assess how varying thresholds affect the ability of these estimators to detect outliers.&#x0D; Study Design: This study employs a quantitative research design. It involves the generation of simulated data, the application of the MCD and MRCD estimators for outlier detection, and the systematic manipulation of thresholds and sample size as independent variables.&#x0D; Place and Duration: The study is conducted using computational tools and did not require a physical location.&#x0D; Methodology: Simulated data is generated from the standard normal distribution to create a controlled environment for outlier detection experiments. The MCD and MRCD estimators are applied to the simulated data to detect outliers. These estimators are sensitive to deviations from the norm in the data. Different thresholds are systematically applied to the data, and the performance of the estimators is assessed at each threshold level. Thresholds may vary in their extremeness. The study investigates the impact of different sample sizes on outlier detection. This involves using datasets with varying numbers of observations. The r programming language and associated packages are used as the statistical tool for data generation, analysis, and visualization.&#x0D; Results: The study's findings indicate that the choice of thresholds in data analysis significantly affects the performance of the MCD and MRCD estimators in outlier detection. If the thresholds used for both estimators are the same, their performance is similar. However, differences emerge when thresholds differ from each other. Higher thresholds are shown to identify less extreme outliers, while lower thresholds are effective at identifying more extreme outliers. These results provide insights into the behavior of these estimators in outlier detection scenarios, shedding light on their sensitivity to threshold choices and sample size.Conclusion: Our study has shed light on the critical interdependencies among threshold choices, sample sizes, and the performance of the minimum covariance determinant (MCD) and minimum regularized covariance determinant (MRCD) estimators in the context of outlier detection. By conducting a systematic exploration in a controlled environment with simulated data, we have gleaned valuable insights that can inform both researchers and practitioners in the field of organizational science research.

The Minimum Covariance Determinant (MCD) estimator proposed by Rousseeuw (1985) is a highly robust way to estimate location and scatter of multivariate and possibly contaminated data sets. Recently Rousseeuw and Van Driessen (1999) have proposed a fast algorithm which allows practical application of MCD even in very large and contaminated data sets. As an initial estimate of location and scatter, MCD could be a better alternative than the so-called minimum volume ellipsoid (MVE) estimator proposed by Rousseeuw (1985), because MCD is asymptotically normal and MVE has a lower convergence rate. Robust distances using MVE have been used for labeling multiple outliers, as it was suggested by Rousseeuw and van Zomeren (1990). For small - sample cases Rousseeuw and van Zomeren (1991) computed by simulation a correction factor in terms to be able to compare MVE robust distances against distribution quantiles. In accordance with its superior theoretical performance, MCD estimate might be used for the same purpose; however it also requires a correction factor for small samples. We present in this paper the results from a simulation study for small - sample cases, with two objectives: first, when both types of estimators are used to build tolerance regions, we compare them in relation to their actual coverage probabilities; and second, to compute required correction factors. We followed the methodology from Rousseeuw and van Zomeren (1991).

In this paper, we propose a collaborative representation-based binary hypothesis model with multi-features learning (CRTDBH-MTL) for target detection in hyperspectral imagery. The proposed method contained the following aspects. First, two complementary features extracted by different algorithms are implemented for describing hyperspectral imageries. Next, we apply these features into the unified collaborative representation-based binary hypothesis model (CRTDBH) to acquire a collaborative vector (CV) for each feature. Once the CV is obtained, the sample can be sparsely represented by the training samples from the background-only dictionary under the null hypothesis and the training samples from the target and background dictionaries under the alternative hypothesis. Finally, spatial correlation and spectral similarity of adjacent neighboring pixels are exploited to improve the detection performance. The experimental results suggest that the proposed algorithm shows an outstanding detection performance.

Problem statement: High leverage points are extreme outliers in the X-direction. In regression analysis, the detection of these leverage points becomes important due to their arbitrary large effects on the estimations as well as multicollinearity problems. Mahalanobis Distance (MD) has been used as a diagnostic tool for identification of outliers in multivariate analysis where it finds the distance between normal and abnormal groups of the data. Since the computation of MD relies on non-robust classical estimations, the classical MD can hardly detect outliers accurately. As an alternative, Robust MD (RMD) methods such as Minimum Covariance Determinant (MCD) and Minimum Volume Ellipsoid (MVE) estimators had been used to identify the existence of high leverage points in the data set. However, these methods tended to swamp some low leverage points even though they can identify high leverage points correctly. Since, the detection of leverage points is one of the most important issues in regression analysis, it is imperative to introduce a novel detection method of high leverage points. Approach: In this study, we proposed a relatively new two-step method for detection of high leverage points by utilizing the RMD (MVE) and RMD (MCD) in the first step to identify the suspected outlier points. Then, in the second step the MD was used based on the mean and covariance of the clean data set. We called this method two-step Robust Diagnostic Mahalanobis Distance (RDMDTS) which could identify high leverage points correctly and also swamps less low leverage points. Results: The merit of the newly proposed method was investigated extensively by real data sets and Monte Carlo Simulations study. The results of this study indicated that, for small sample sizes, the best detection method is (RDMDTS) (MVE)-mad while there was not much difference between (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad for large sample sizes. Conclusion/Recommendations: In order to swamp less low leverage as high leverage point, the proposed robust diagnostic methods, (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad were recommended.

The minimum covariance determinant (MCD) estimator is a highly robust estimator of multivariate location and scatter. It can be computed efficiently with the FAST‐MCD algorithm of Rousseeuw and Van Driessen. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD has also been used to develop robust and computationally efficient multivariate techniques.In this paper, we review the MCD estimator, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in theoretical and applied multivariate statistics. Copyright © 2009 John Wiley &amp; Sons, Inc.This article is categorized under: Statistical and Graphical Methods of Data Analysis &gt; Robust Methods

Kernel-based regularized-angle spectral matching for target detection in hyperspectral imagery

This paper introduces Cointegration Theory to address the problem of adaptive target detection in hyperspectral imagery. Cointegration Theory aims at mining a long-term equilibrium relationship, which refers to the condition that an appropriate linear combination of several non-stationary series can be stationary as long as they have similar or related drift. Hyperspectral response sequences, which are highly non-stationary, have similar patterns among the same materials. To be treated as a time series, each given hyperspectral curve is matched with the reference spectrum via the Johansen Cointegration Test. The statistic of the test is then used for target detection. Experimental results indicate that our proposed method is effective and has a strong capacity to identify interesting objects from their background.

Although sparse representation based classification (SRC) has gained great success, doubts on the necessity of sparse constraint come in recent years. And collaborative representation based classification (CRC) has attracted much attention from researchers in fields of signal processing, image processing and pattern recognition. In this paper, an algorithm called collaborative dictionary learning with structured incoherence (CDLSI) is proposed for collaborative representation based detection (CRD), which can be viewed as a binary classification problem, in hyperspectral imagery (HSI). An inter-class incoherence term is added to make sub-dictionaries to be as independent as possible. During the optimizing procedure, sub-dictionaries are updated atoms-by-atoms with metaface method. Specifically, considering the non-sparse representation of CRC, the coefficients are iteratively optimized with l₂ -norm regularization during the coding procedure in CDLSI. Once the sub-dictionaries are obtained, the collaborative representation based technique is then used for detection. The proposed algorithm is applied to several real hyperspectral images for detection. Experimental results confirm the effectiveness of the proposed approach, and prove the superiority to the traditional algorithms.

Subpixel target detection in hyperspectral imagery is challenging since subpixel targets are smaller in size than the resolution of a single pixel and accurate pixel-level labels on subpixel targets are often unavailable. In particular, this article addresses the problem of learning a prime prototype target signature from imprecisely labeled highly mixed hyperspectral data. Two algorithms, multiple-instance subpixel adaptive cosine estimator (MI-SPACE) and multiple-instance subpixel spectral matched filter (MI-SPSMF), based on multiple-instance learning framework are presented. The proposed methods aim to learn a discriminative prime target signature by maximizing the posterior detection statistics of subpixel hyperspectral targets for the correspondingly proposed subpixel adaptive cosine estimator (SPACE) and subpixel spectral matched filter (SPSMF) detectors, which are also developed in this article. Experimental results demonstrate the effectiveness of the proposed methods on both simulated and real-field hyperspectral subpixel target detection tasks.

Orthogonal subspace projection (OSP) has been used in hyperspectral image processing for automatic target detection and image classification. Existing OSP based approaches for target detection require a priori knowledge of all undesired signatures present in the input scene. In this paper, we proposed a new technique for target detection which does not require a priori knowledge of the non-target signatures present in the input scene. The length of any pixel vector containing the target reduces significantly when it is projected in a direction orthogonal to the target signature. Thus the ratio between the original pixel vector to the projected pixel vector yields a high value for the pixels containing the target. Therefore, an OSP based parameter along with noise adjusted principal component analysis (NAPCA) was introduced in this paper for target detection in hyperspectral images. For noisy images, NAPCA is used as a preprocessing step to reduce the effects of noise as well as to reduce the spectral dimension thereby yielding better target detection capability while enhancing the computational efficiency. For noise-free input scenes or when very small amount of noise is present in the input scene, principal component analysis (PCA) may be used instead of NAPCA. The OSP based technique requires that the number of spectrally distinct signatures present in the input scene must be less than the number of spectral bands. The proposed algorithm yields very good results even when this criterion is not satisfied.

Target detection in hyperspectral imagery is a complex process due to many factors. Exploiting the hyperspectral image for analysis is very challenging due to large information and low spatial resolution. However, hyperspectral target detection has numerous applications. Hence, it is important to pursue research in target detection. In this paper, a literature survey of target detection algorithms for hyperspectral imagery is presented along with scope for future research. An overview of theory behind the hyperspectral imaging is detailed. Also, various challenges involved in exploring the hyperspectral data are discussed.

Metric learning-based methods, which yield great performance and show considerable potential to improve the performance of hyperspectral image processing, aim to calculate the Mahalanobis distance metric matrix. In this article, we proposed a symmetric information-theoretic metric learning (SITML) method for hyperspectral target detection. The SITML algorithm is designed based on the classical information-theoretic metric learning (ITML) and, minimizes the differential Kullback–Leibler (KL) divergence. To enhance both of the detection performance and the generalization ability, we build metric spaces from the neighborhood of training samples to preserve the local discriminative information. Then, we conduct local pairwise constraints to maximize the Jeffery divergence (also named the symmetric KL divergence) of two multivariate Gaussian distributions to solve the problem of an asymmetric KL divergence. Finally, we use a closed-form solution to solve the optimization problem. Intensive experiments on three hyperspectral datasets indicate that SITML outperforms the classical ITML algorithm and other state-of-the-art target detection methods.

Sparse representation based on stacked kernel for target detection in hyperspectral imagery

For multivariate data, tandem clustering is a well-known technique aiming to improve cluster identification through initial dimension reduction. Nevertheless, the usual approach using principal component analysis (PCA) has been criticized for focusing solely on inertia so that the first components do not necessarily retain the structure of interest for clustering. To address this limitation, a new tandem clustering approach based on invariant coordinate selection (ICS) is proposed. By jointly diagonalizing two scatter matrices, ICS is designed to find structure in the data while providing affine invariant components. Certain theoretical results have been previously derived and guarantee that under some elliptical mixture models, the group structure can be highlighted on a subset of the first and/or last components. However, ICS has garnered minimal attention within the context of clustering. Two challenges associated with ICS include choosing the pair of scatter matrices and selecting the components to retain. For effective clustering purposes, it is demonstrated that the best scatter pairs consist of one scatter matrix capturing the within-cluster structure and another capturing the global structure. For the former, local shape or pairwise scatters are of great interest, as is the minimum covariance determinant (MCD) estimator based on a carefully chosen subset size that is smaller than usual. The performance of ICS as a dimension reduction method is evaluated in terms of preserving the cluster structure in the data. In an extensive simulation study and empirical applications with benchmark data sets, various combinations of scatter matrices as well as component selection criteria are compared in situations with and without outliers. Overall, the new approach of tandem clustering with ICS shows promising results and clearly outperforms the PCA-based approach.