Local Sub-Block Contrast and Spatial–Spectral Gradient Feature Fusion for Hyperspectral Anomaly Detection

Analysis of hyperspectral images is of great importance in many scientific disciplines. Obtaining the spectral and spatial information simultaneously from time-varying hyperspectral images is a challenging task due to their high dimensionality. In this paper, we design an interface that allows users to study hyperspectral images interactively and obtain spectral features and enhanced images at the same time. The image fusion results change dynamically with the regions of interest selected by users and convey both the spatial and spectral information. We show the usefulness of our approach using time-varying hyperspectral plant images. We compare our method with existing hyperspectral image analysis techniques. Our evaluation indicates that our interface can help users determine important bands, identify regions of interest, and generate image fusion results for time-varying hyperspectral plant images.

We present a proof-of-principle demonstration using Sony's IBM Cell processor-based PlayStation 3 (PS3) to run-in near real-time-a hyperspectral anomaly detection algorithm (HADA) on real hyperspectral (HS) long-wave infrared imagery. The PS3 console proved to be ideal for doing precisely the kind of heavy computational lifting HS based algorithms require, and the fact that it is a relatively open platform makes programming scientific applications feasible. The PS3 HADA is a unique parallel-random sampling based anomaly detection approach that does not require prior spectra of the clutter background. The PS3 HADA is designed to handle known underlying difficulties (e.g., target shape/scale uncertainties) often ignored in the development of autonomous anomaly detection algorithms. The effort is part of an ongoing cooperative contribution between the Army Research Laboratory and the Army's Armament, Research, Development and Engineering Center, which aims at demonstrating performance of innovative algorithmic approaches for applications requiring autonomous anomaly detection using passive sensors.

Deep learning has achieved excellent performance in the field of computer vision and gained attention in the field of hyperspectral anomaly detection (HAD). Convolutional neural networks (CNN) based on pixel pairing features have achieved a certain detection effect, which is limited by background complexity in practical applications. This study proposes an adaptive weighted HAD algorithm that combines an autoencoder (AE) and CNN. Labelled hyperspectral images (HSIs) are used to train a CNN as a binary classifier for similarity discrimination in the data preparation stage, where morphological attribute filtering (MAF) is performed on images in the spatial dimension of the HSIs to highlight anomaly. The spectral angle is used as a measure of reconstruction error to train the AE, and the reconstruction error obtained using the AE is used as an adaptive weight to calculate the anomaly score, which considerably eliminates the blurring of the boundary between the background and anomalies. Through comparisons with some state-of-the-art methods, the effectiveness of the proposed method in improving detection accuracy and increasing background and anomaly discrimination is verified on three real datasets.

Most of the existing hyperspectral image change detection methods based on deep learning utilize convolutional neural networks (CNN), recurrent neural networks (RNN), and long-short-term memory (LSTM). However, CNNs primarily focus on extracting spatial information from hyperspectral images, thereby failing to fully utilize channel and temporal information. In this paper, we propose a network called STSTNet that can jointly extract spatial, spectral, and temporal feature information. To achieve a richer feature representation space, we employ a spatial attention module to extract spatial feature information from feature maps, and a spectral transformer encoder structure to extract abundant spectral sequence information. Specifically, we design two temporal information extraction structures after each spatial information extraction structure or spectral information extraction structure to capture correlations and interactions within the hyperspectral images. Extensive experiments have been conducted on three hyperspectral datasets, and the experimental results demonstrate the network’s excellent performance.

Rooibos (Aspalathus linearis) and Honeybush (Cyclopia species) are teas indigenous to South Africa that are popularly consumed locally as well as internationally for its purported health benefits. Studies have confirmed anti-oxidant, antimutagenic and anticancer activity mainly due to the presence of polyphenols. Hyperspectral imaging (HSI) integrates conventional spectroscopy and imaging to obtain spectral and spatial information from a sample. The potential of short wave infrared (SWIR) hyperspectral imaging in combination with chemometric data analysis as a rapid quality control method for commercially important herbal teas was investigated. Images were acquired with the sisuChema short wave infrared (SWIR) hyperspectral pushbroom imaging system at a spectral range of 920 – 2514nm. Principal Component Analysis (PCA) was performed to remove background pixels and to visualize the data. Classes (i.e. Rooibos or Honeybush) were interactively assigned using the score images and plots. Partial Least Squares Discriminant Analysis (PLS-DA) classification models developed were subsequently used to accurately predict the species identity of commercially available herbal tea samples introduced as an external dataset, producing a visually interpretable result. The results showed that the labeling information is correct but that Honeybush tea is in most cases present in very low amounts. It is evident that hyperspectral imaging is an objective and non-destructive quality control method that can be successfully used to determine the botanical species included in herbal teas in addition to giving an indication of the abundance of each species in polyherbal tea mixtures.

Target detection is a key issue in processing hyperspectral images (HSIs). Spectral-identification-based algorithms are sensitive to spectral variability and noise in acquisition. In most cases, both the target spatial distributions and the spectral signatures are unknown, so each pixel is separately tested and appears as a target when it significantly differs from the background. In this paper, we propose two algorithms to improve the signal-to-noise ratio (SNR) of hyperspectral data, leading to detectors that are robust to noise. These algorithms consist in integrating adaptive spatial/spectral filtering into the adaptive matched filter and adaptive coherence estimator. Considering the HSIs as tensor data, our approach introduces a data representation involving multidimensional processing. It combines the advantages of spatial and spectral information using an alternating least square algorithm. To estimate the signal subspace dimension in each spatial mode, we extend the Akaike information criterion, and we develop an iterative algorithm for spectral-mode rank estimation. We demonstrate the interest of integrating the quadtree decomposition to perform an adaptive 3-D filtering and thereby preserve the local image characteristics. This leads to a significant improvement in terms of denoised tensor SNR and, consequently, in terms of detection probability. The performance of our method is exemplified using simulated and real-world HYperspectral Digital Imagery Collection Experiment images.

Fusion of hyperspectral and panchromatic images using structure tensor and matting model

In order to quickly distinguish infertile eggs from fertile eggs, the hyperspectral imaging technology consisting of imaging and spectral information was used for detecting the fertile information of eggs. Before hatching eggs were incubated, a hyperspectral imaging system (wavelength between 400 to 1 000 nm) was used to acquire the images one-by-one manually. The characteristic information of ratios of length to short axis, elongation, roundness and the ratios of the yolk area to the whole area was extracted based on the images. The normalization method was used as the spectral data preprocessing, and then 155 spectral characteristic variables were extracted from 520 nm waveband through the correlation coefficient method. Principal component analysis (PCA) method was adopted to reduce the dimensions of image-spectrum fusion information; the top six principal components were extracted. Support vector machine (SVM) method was used to establish classification of fertile and infertile eggs models, which are based on image, spectrum and image-spectrum fusion information respectively. The accuracy rates of the SVM models were 84.00%, 90.00% and 93.00% respectively. The experimental results show that the model based on image-spectrum fusion information technology is superior to the single information model. Hyperspectral transmission imaging technology is effective and feasible to detect the fertile hatching eggs before incubation. Keywords: hyperspectral image, hatching eggs, information fusion, nondestructive detection DOI: 10.3965/j.ijabe.20150804.1672 Citation: Zhu Z H, Liu T, Xie D J, Wang Q H, Ma M H. Nondestructive detection of infertile hatching eggs based on spectral and imaging information. Int J Agric & Biol Eng, 2015; 8(4): 69－76.

A spatial-spectral SIFT for hyperspectral image matching and classification

In this paper, a soft classification algorithm based on composite kernels, which incorporate both spectral and spatial information, is proposed for hyperspectral image. Compared with hard classification, soft classification provides more information about the probabilities one pixel belongs to each class. To calculate these probabilities, the proposed algorithm uses Support Vector Machine (SVM), and it successfully converts SVM output values into probabilities, while at the same time integrates spatial and spectral information by composite kernels. To validate the proposed algorithm, experiments are conducted on hyperspectral images with 126 and 186 bands, and experimental results show that soft classification using SVM can yield better results compared with Maximum Likelihood Classifier (MLC),and the introduction of spectral-spatial kernels can greatly improve classification accuracies.

Fusing a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI), aiming to produce a super-resolution hyperspectral image, has recently attracted increasing research interest. In this paper, a novel approach based on coupled non-negative tensor decomposition is proposed. The proposed method performs a tucker tensor factorization of a low resolution hyperspectral image and a high resolution multispectral image under the constraint of non-negative tensor decomposition (NTD). The conventional matrix factorization methods essentially lose spatio-spectral structure information when stacking the 3D data structure of a hyperspectral image into a matrix form. Moreover, the spectral, spatial, or their joint structural features have to be imposed from the outside as a constraint to well pose the matrix factorization problem. The proposed method has the advantage of preserving the spatio-spectral structure of hyperspectral images. In this paper, the NTD is directly imposed on the coupled tensors of the HSI and MSI. Hence, the intrinsic spatio-spectral structure of the HSI is represented without loss, and spatial and spectral information can be interdependently exploited. Furthermore, multilinear interactions of different modes of the HSIs can be exactly modeled with the core tensor of the Tucker tensor decomposition. The proposed method is straightforward and easy to implement. Unlike other state-of-the-art approaches, the complexity of the proposed approach is linear with the size of the HSI cube. Experiments on two well-known datasets give promising results when compared with some recent methods from the literature.

Deep learning target detection based on faster regions with convolutional neural network (Faster R-CNN) features has been applied in image processing successfully, however, it is rarely introduced to the field of hyperspectral image (HSI) target detection due to the tensor characteristics and the lack of training samples of HSI data. In this paper, the target detection based on Faster R-CNN is proposed to HSI with data set adjustment and parameter turning. As a typical tensor data, HSIs contain two-dimensional (2-D) spatial information and one dimensional (1-D) spectral information. It contains more information than ordinary images, and has unique advantages in the field of ground object and sea target detection. Therefore, the original HSI is firstly adjusted to the data set format required by the model, and the final Faster R-CNN sample data set can be achieved by combining the data set of Google Earth images. Next, a Faster R-CNN network suitable for HSI data could be built. Finally, to improve the accuracy of target detection, some parameters of Faster R-CNN would be tuned. The numerical results show that the method has the potential advantages of high precision and high speed in HSI target detection, and will have broad application prospects.

Designing active learning (AL) techniques to determine informative training samples for hyperspectral image (HSI) classification is an open research issue. In this chapter, several spectral–spatial AL techniques are designed by exploiting both, an interesting approach to fuse spectral and spatial information and different combinations of existing uncertainty and diversity criteria. In order to fuse spectral and spatial information, the dimensionality of HSI is reduced and mean filtering (for incorporation of spatial information) is applied to each component image in the reduced domain of HSI considering multiple windows of different sizes. The filtered images are concatenated with original component images to form an extended spatial profile for the HSI. These spectral–spatial features are used with different combinations of uncertainty and diversity criteria to design several spectral–spatial AL techniques for determining most informative pixels. The experiments carried out on two real HSI data sets show that the spectral–spatial AL methods are more robust than the AL methods based on spectral measurements only.

Hyperspectral images (HSI) have high-dimensional and complex spectral characteristics, with dozens or even hundreds of bands covering the same area of pixels. The rich information of the ground objects makes hyperspectral images widely used in satellite remote sensing. Due to the limitations of remote sensing satellite sensors, hyperspectral images suffer from insufficient spatial resolution. Therefore, utilizing software algorithms to improve the spatial resolution of hyperspectral images has become an urgent problem that needs to be solved. The spatial information and spectral information of hyperspectral images are strongly correlated. If only the spatial resolution is improved, it often damages the spectral information. Inspired by the high correlation between spectral information in adjacent spectral bands of hyperspectral images, a hybrid convolution and spectral symmetry preservation network has been proposed for hyperspectral super-resolution reconstruction. This includes a model to integrate information from neighboring spectral bands to supplement target band feature information. The proposed model introduces flexible spatial-spectral symmetric 3D convolution in the network structure to extract low-resolution and neighboring band features. At the same time, a combination of deformable convolution and attention mechanisms is used to extract information from low-resolution bands. Finally, multiple bands are fused in the reconstruction module, and the high-resolution hyperspectral image containing global information is obtained by Fourier transform upsampling. Experiments were conducted on the indoor hyperspectral image dataset CAVE, the airborne hyperspectral dataset Pavia Center, and Chikusei. In the X2 super-resolution task, the PSNR values achieved on the CAVE, Pavia Center, and Chikusei datasets were 46.335, 36.321, and 46.310, respectively. In the X4 super-resolution task, the PSNR values achieved on the CAVE, Pavia Center, and Chikusei datasets were 41.218, 30.377, and 38.365, respectively. The results show that our method outperforms many advanced algorithms in objective indicators such as PSNR and SSIM while maintaining the spectral characteristics of hyperspectral images.

Spectral unmixing is an important problem for remotely sensed hyperspectral data exploitation. Automatic spectral unmixing can be viewed as a three-stage problem, where the first stage is subspace identification, the next one is endmember extraction, and the final one is abundance estimation. In this sequence, endmember extraction is the most challenging problem. Many researchers have attempted to extract endmembers from hyperspectral images using spectral information only. However, it is well known that the inclusion of spatial information can improve the endmember extraction task. In this article, we introduce a new endmember extraction algorithm that exploits both spectral and spatial information. A main innovation of the proposed algorithm is that spatial information is exploited using entropy, while spectral information is exploited using convex set optimization. In the literature, none of the spatial–spectral algorithms has used entropy as spatial information. The inclusion of this entropy-based spatial information improves the accuracy of the endmember extraction process. The results obtained by the proposed algorithm are compared (using a variety of metrics) with those obtained by other state-of-the-art methods, using both synthetic and real datasets. Our experimental results demonstrate that the proposed algorithm outperforms many available algorithms.