Dual Collaborative Constraints Regularized Low-Rank and Sparse Representation via Robust Dictionaries Construction for Hyperspectral Anomaly Detection

Sparse representation-based background modeling facilitates much recent progress in hyperspectral anomaly detection (AD). The sparse representation of background often exhibits underlying structure, which is crucial to distinguish between background and anomaly. However, how to exploit such underlying structure is still challenging. To address this problem, we present a novel hyperspectral AD method, which can exploit the structured sparsity in modeling the background more accurately. With the plausible background area detected by a local RX detector, a robust background spectrum dictionary is learned in a principal component analysis way. A reweighted Laplace prior-based structured sparse representation model is then employed to reconstruct the spectrum of each pixel. With considering the structured sparsity in representation, the background pixels can be reconstructed more accurately than the anomaly ones, which thus can be detected based on the reconstruction error. To further improve the detection performance, an intracluster reconstruction model is developed to exploit the spatial similarity among the background pixels in the same cluster. The anomaly pixels can then be detected based on the cost of intracluster reconstruction error. By linearly combining these two detection results, improvement is obviously achieved on detection accuracy. Experimental results on both simulated and real-world data sets demonstrate that the proposed method outperforms several state-of-the-art hyperspectral AD methods.

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

Hyperspectral anomaly detection (HAD) aims to separate subtle anomalies of a given hyperspectral image (HSI) from its background, which is a hot topic as well as a challenging inverse problem. Despite the significant success of the deep learning-based HAD methods, they are hard to interpret due to their black-box nature. Meanwhile, deep learning methods suffer from the identity mapping (IM) problem, referring to the network excessively focusing on the precise reconstruction of the background while neglecting the appropriate representation of anomalies. To this end, this paper proposes a self-attention-aided deep unfolding tensor representation network (SATRNet) for interpretable HAD by solving the tensor representation (TR)-based optimization model within the framework of deep networks. In particular, a Self-Attention Learning Module (SALM) was first designed to extract discriminative features of the input HSI. The HAD problem was then formulated as a tensor representation problem by exploring both the low-rankness of the background and the sparsity of the anomaly. A Weight Learning Module (WLM) exploring local details was also generated for precise background reconstruction. Finally, a deep network was built to solve the TR-based problem through unfolding and parameterizing the iterative optimization algorithm. The proposed SATRNet prevents the network from learning meaningless mappings, making the network interpretable to some extent while essentially solving the IM problem. The effectiveness of the proposed SATRNet is validated on 11 benchmark HSI datasets. Notably, the performance of SATRNet against adversarial attacks is also investigated in the experimentation, which is the first work exploring adversarial robustness in HAD to the best of our knowledge.

Hyperspectral anomaly detection (HAD), a challenging inverse problem, has found numerous scientific applications. Although extant HAD algorithms have achieved remarkable results, there are still several issues remained unresolved: 1) low spatial resolution (and spectral redundancy) in typical hyperspectral images prevents effectively distinguishing the abnormal pixels from those normal ones and 2) the reconstruction from existing residual-based frameworks would not completely remove anomaly effects, making the detection solely from the residual impractical. In this article, we propose a novel HAD method, termed transformer-guided fractional attention within the abundance domain (TGFA-AD), which substitutes raw input image with the abundance matrix obtained via blind source separation (BSS). First, the proposed abundance spatial-channel reconstruction transformer (ASCR-Former) is customized for rebuilding the abundance matrix. According to the image self-similarity, the abundance is patch-wisely encoded with class (CLS) tokens. The transformer encoders intensify the spatial and channel characteristics between tokens for reconstructing the abundance, followed by deriving the initial detection from the abundance residual matrix. Second, a novel fractional abundance attention (FAA) mechanism is proposed, where the attention weights coming from a specific linear combination of abundances are guided by the initial detection with convex $ Q$ -quadratic norm. Finally, the fractional convolution is incorporated to fuse the abundance and residual into the fractional feature for yielding the final detection result. Real data experiments quantitatively and qualitatively exhibit the state-of-the-art performance of TGFA-AD.

Hyperspectral anomaly detection, which is aimed at locating anomaly, has received widespread attention. In this article, a new anomaly detector, named local spatial constraint and total variation (LSC-TV), is proposed for hyperspectral imagery. In anomaly detection methods based on low-rank representation, background pixels are usually considered to have a global low-dimensional structure. However, the complex background distribution in hyperspectral images (HSIs) means that this global low-dimensional structure rarely occurs. In LSC-TV, the effective local spatial information is extracted by superpixel segmentation, and the regularization based on the F-norm is used to force the background within the same superpixel to show uniform spectral features. Moreover, each pixel is given a penalty based on the degree of anomaly determined during model iteration, while the anomaly is not considered by the background constraint. In addition, the background pixels in the neighborhood often show a high correlation, whereas the anomaly does not possess this feature. Nonisotropic TV is introduced into the proposed LSC model using the correlation of first-order neighborhoods to make it easier for anomalies to be separated. The proposed LSC-TV method and current state-of-the-art methods are tested on a set of simulated data and four sets of real data. The experimental results demonstrate that the proposed method is superior to the comparative method in terms of both color map detection and quantitative evaluation.

Accurately distinguishing between background and anomalous objects within hyperspectral images poses a significant challenge. The primary obstacle lies in the inadequate modeling of prior knowledge, leading to a performance bottleneck in hyperspectral anomaly detection (HAD). In response to this challenge, we put forth a groundbreaking coupling paradigm that combines model-driven low-rank representation (LRR) methods with data-driven deep learning techniques by learning disentangled priors (LDP). LDP seeks to capture complete priors for effectively modeling the background, thereby extracting anomalies from hyperspectral images more accurately. LDP follows a model-driven deep unfolding architecture, where the prior knowledge is separated into the explicit low-rank prior formulated by expert knowledge and implicit learnable priors by means of deep networks. The internal relationships between explicit and implicit priors within LDP are elegantly modeled through a skip residual connection. Furthermore, we provide a mathematical proof of the convergence of our proposed model. Our experiments, conducted on multiple widely recognized datasets, demonstrate that LDP surpasses most of the current advanced HAD techniques, exceling in both detection performance and generalization capability.

Outlier detection is the process of detecting the data objects which are grossly different from or inconsistent with the remaining set of data. Some of the important applications in the field of data mining are fraud detection, customer behavior analysis, and intrusion detection. There are number of good research algorithms for detecting outliers if the entire data is available and algorithms can operate in more than single passes to achieve the required results. Among the existing methods, LOF (Local outlier Factor) a density based method is very efficient in detecting all forms of outliers. LOF algorithm can not be directly applied to the datastream as the large number of nearest neighbor searches, LOF computation and lrd (local reachability distances) can make it highly inefficient for datastream. In this paper we propose a cluster based partitioning algorithm which can divide the stream in safe region and candidate regions. In Second phase apply LOF algorithm over these partitions separately with some slight enhancement for LOF computation over candidate region to achieve accurate results for finding most outstanding outliers. Several experiments on different dataset confirm that our technique can find better outliers with low computational cost than the direct LOF or compared to the other enhancements proposed for LOF.

Tactical reconnaissance using small unmanned aerial vehicles has become a common military scenario. However, since their sensor systems are usually limited to rudimentary visual or thermal imaging, the detection of camouflaged objects can be a particularly hard challenge. With respect to SWaP-C criteria, multispectral sensors represent a promising solution to increase the spectral information that could lead to unveiling camouflage. Therefore, this paper investigates and evaluates the applicability of four well-known hyperspectral anomaly detection methods (RX, LRX, CRD, and AED) and a method developed by the authors called local point density (LPD) for near real-time camouflage detection in multispectral imagery based on a specially created dataset. Results show that all targets in the dataset could successfully be detected with an AUC greater than 0.9 by multiple methods, with some methods even reaching an AUC relatively close to 1.0 for certain targets. Yet, great variations in detection performance over all targets and methods were observed. The dataset was additionally enhanced by multiple vegetation indices (BNDVI, GNDVI, and NDRE), which resulted in generally higher detection performances of all methods. Overall, the results demonstrated the general applicability of the hyperspectral anomaly detection methods for camouflage detection in multispectral imagery.

Anomaly detection is a fundamental task in hyperspectral image (HSI) processing. However, most existing methods rely on pixel feature vectors and overlook the relational structure information between pixels, limiting the detection performance. In this article, we propose a novel approach to hyperspectral anomaly detection that characterizes the HSI data using a vertex- and edge-weighted graph with the pixels as vertices. The constructed graph encodes rich structural information in an affinity matrix. A crucial innovation of our method is the ability to obtain internal relations between pixels at multiple topological scales by processing different powers of the affinity matrix. This power processing is viewed as a graph evolution, which enables anomaly detection using vertex extraction formulated as a quadratic programming problem on graphs of varying topological scales. We also design a hierarchical guided filtering architecture to fuse multiscale detection results derived from graph evolution, which significantly reduces the false alarm rate. Our approach effectively characterizes the topological properties of HSIs, leveraging the structural information between pixels to improve anomaly detection accuracy. Experimental results on four real HSIs demonstrate the superior detection performance of our proposed approach compared to some state-of-the-art hyperspectral anomaly detection methods.

Many local outlier detection algorithms have been proposed inspired by the idea of local outlier factor (LOF). However, they often have low detection performance and are sensitive to neighborhood size because there is a major defect in their calculation formulas of outlier degree and the kNN (k-nearest neighbors) method is widely used to quantify a neighborhood of an instance. To address these issues, we define a novel nearest neighbors tree (NNT) to measure a neighborhood of an instance. Meanwhile, we propose a local structure outlier factor (LSOF), which score each local structure instead of each data point and report the top-scored local structures as anomalous local structures, where outliers and groups of outliers are easily divided according to characteristics of the NNT. Our experimental results demonstrate that the competitive behavior of our method on both synthetic and real-world datasets.

Over the last two decades, anomaly detection (AD) has been known to play a critical role in hyperspectral image analysis, which provides a new way to distinguish the targets from the background without prior knowledge. Recently, the representation-based methods were proposed and soon became a significant type of methods on hyperspectral AD. In this paper, a novel AD algorithm based on convolutional neural network (CNN) and low-rank representation (LRR) is proposed. First, a CNN model is built and trained on hyperspectral image (HSI) datasets to accurately obtain the resulting abundance maps. Compared with the raw dataset, abundance maps contain more distinctive features to identify anomalies from the background. Second, a dictionary is constructed by the density-based spatial clustering of applications with noise (DBSCAN) algorithm to stably represent the background component. Third, a matrix decomposition method based on LRR is adopted. In this way, a coefficient matrix corresponding to the constructed dictionary is obtained, which is low rank. At the same time, a residual matrix can be obtained as well, which is sparse. Finally, anomalies can be extracted from the residual matrix. The experimental results show that the proposed method achieves a superior performance compared to some of the state-of-the-art methods in the field of hyperspectral AD.

The ubiquity of smartphones in our daily lives has made them attractive targets for malicious actors seeking to compromise user data and device functionality. Android malware detection has become imperative to protect user privacy and device integrity. This paper presents a focused study on leveraging the Local Outlier Factor (LOF) method for Android malware detection using the DREBIN dataset. Our research addresses the need for accurate and efficient Android malware detection. We explore the LOF method, an anomaly-based detection technique, to assess its effectiveness in distinguishing malicious applications from benign ones within the Android ecosystem. Rigorous experiments using the extensive DREBIN dataset reveal LOF's superiority in accuracy, precision, recall, and False Positive Rate (FPR). We introduce additional metrics like Area Under the Curve (AUC), Matthews Correlation Coefficient (MCC), and True Negative Rate (TNR) to comprehensively evaluate LOF. Our findings highlight LOF's ability to balance false positives and false negatives, making it an ideal choice for Android malware detection. We emphasize the importance of representative datasets, such as DREBIN, for validation. In conclusion, this research positions LOF as a reliable tool for Android malware detection, offering robust protection against emerging threats. As mobile technology evolves, our study encourages further exploration of advanced techniques and real-world deployment scenarios.

Sparse representation and modified tensor projection for hyperspectral anomaly detection

Hyperspectral anomaly detection based on weighted low-rank sparse dictionary learning

After the local outlier factor was first proposed, there is a large family of local outlier detection approaches derived from it. Since the existing approaches only focus on the extent of overall separation between an object and its neighbors, and ignore the degree of dispersion between them, the precision of these approaches will be affected by various degrees in the scattered datasets. In addition, the outlier data occupy a relatively small amount in the dataset, but the existing approaches need to perform local outlier factor calculation on all data during the outlier detection, which greatly reduces the efficiency of the algorithms. In this paper, we redefine a local outlier factor called local deviation coefficient (LDC) by taking full advantage of the distribution of the object and its neighbors. And then, we propose a safe non-outlier objects elimination approach named as rough clustering based on multi-level queries (RCMLQ) to preprocess the datasets to eliminate the non-outlier objects to the utmost. Finally, an efficient local outlier detection approach named as efficient density-based local outlier detection for scattered data (E2DLOS) is proposed based on the LDC and RCMLQ. The RCMLQ greatly reduces the amount of data that needs to be quantified for local outlier factor and the LDC is more sensitive to the degree of anomaly of the scattered datasets, and so the E2DLOS improves the existing local outlier detection approaches in time efficiency and detection accuracy. Experiments show that the LDC can better reflect the true abnormal situations of the data for the scattered datasets. And the RCMLQ can be used in parallel with the traditional methods of improving the efficiency of the nearest neighbor search, which can further improve the efficiency of the E2DLOS algorithm by about 16%.