Unsupervised hyperspectral imagery classification via sparse multi-way models and image fusion

This article proposes a symmetric sparse representation (SSR) method to extract pure endmembers from hyperspectral imagery (HSI). The SSR combines the features of the linear unmixing model and the sparse subspace clustering model of endmembers, and it assumes that the desired endmembers and all the HSI pixel points can be sparsely represented by each other. It formulates the endmember extraction problem into a famous program of archetypal analysis, and accordingly, extracting pure endmembers can be transformed as finding the archetypes in the minimal convex hull containing all the HSI pixel points. The vector quantization scheme is adopted to help in carefully choosing the initial pure endmembers, and the archetypal analysis program is solved using the simple projected gradient algorithm. Seven state-of-the-art methods are implemented to make comparisons with the SSR on both synthetic and real hyperspectral images. Experimental results show that the SSR outperforms all the seven methods in spectral angle distance and root-mean-square error, and it can be a good alternative choice for extracting pure endmembers from HSI data.

The amount of hyperspectral imagery (HSI) data currently available is relatively small compared to other imaging modalities, and what is suitable for developing, testing, and evaluating spatial-spectral algorithms is virtually nonexistent. In this work, a significant amount of coincident airborne hyperspectral and high spatial resolution panchromatic imagery that supports the advancement of spatial-spectral feature extraction algorithms was collected to address this need. The imagery was collected in April 2013 for Ohio University by the Civil Air Patrol, with their Airborne Real-time Cueing Hyperspectral Enhanced Reconnaissance (ARCHER) sensor. The target materials, shapes, and movements throughout the collection area were chosen such that evaluation of change detection algorithms, atmospheric compensation techniques, image fusion methods, and material detection and identification algorithms is possible. This paper describes the collection plan, data acquisition, and initial analysis of the collected imagery.

To understand surface biogeophysical processes, accurately evaluating the geographical and temporal fluctuations of soil moisture is crucial. It is well known that the surface soil moisture content (SMC) affects soil reflectance at all solar spectrum wavelengths. Therefore, future satellite missions, such as the NASA Surface Biology and Geology mission, will be essential for mapping and monitoring global soil moisture changes. Our study compares two widely used moisture retrieval models: the multilayer radiative transfer model of soil reflectance (MARMIT) and the soil water parametric (SWAP)‐Hapke model. We evaluated the SMC retrieval accuracy of these models using unmanned aerial systems (UAS) hyperspectral imagery and goniometer hyperspectral data. Laboratory analysis employed hyperspectral goniometer data of sediment samples from four locations reflecting diverse environments, while field validation used hyperspectral UAS imaging and coordinated ground truth collected in 2018 and 2019 from a barrier island beach at the Virginia Coast Reserve Long‐Term Ecological Research site. The (SWAP)‐Hapke model achieves comparable accuracy to MARMIT using laboratory hyperspectral data but is less accurate when applied to UAS hyperspectral imagery than the MARMIT model. We proposed a modified version of the (SWAP)‐Hapke model, which achieves better results than MARMIT when applied to laboratory spectral measurements; however, MARMIT's performance is still more accurate when applied to UAS imagery. These results are likely due to differences in the models' descriptions of multiply‐scattered light and MARMIT's more detailed description of air‐water interactions.

Unmanned aerial vehicles (UAVs) provide a new research tool to obtain high spatial and temporal resolution imagery at a reduced cost. Rapid advances in miniature sensor technology are leading to greater potentials for ecological research. We demonstrate one of the first applications of UAV lidar and hyperspectral imagery and a fusion method for individual plant species identification and 3D characterization at submeter scales in south‐eastern Arizona, USA. The UAV lidar scanner characterized the individual vegetation canopy structure and bare ground elevation, whereas the hyperspectral sensor provided species‐specific spectral signatures for the dominant and target species at our study area in leaf‐on condition. We hypothesized that the fusion of the two different data sources would perform better than either data type alone in the arid and semi‐arid ecosystems with sparse vegetation. The fusion approach provides 84–89% overall accuracy (kappa values of 0.80–0.86) in target species classification at the canopy scale, leveraging a wide range of target spectral responses in the hyperspectral data and a high point density (50 points/m2) in the lidar data. In comparison, the hyperspectral image classification alone produced 72–76% overall accuracies (kappa values of 0.70 and 0.71). The UAV lidar‐derived digital elevation model (DEM) is also strongly correlated with manned airborne lidar‐derived DEM (R2 = 0.98 and 0.96), but was obtained at a lower cost. The lidar and hyperspectral data as well as the fusion method demonstrated here can be widely applied across a gradient of vegetation and topography to monitor and detect ecological changes at a local scale.

The fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data is widely used for land cover classification. However, due to different imaging mechanisms, HSI and LiDAR data always present significant image differences, and the dimensions and feature distributions of HSI and LiDAR are highly dissimilar. This makes it challenging to represent and correlate semantic information from multimodal data. Current methods for classifying pixel-by-pixel features, which rely on cascaded or attention-based fusion, cannot effectively use multimodal features. To achieve accurate classification results, extracting and fusing similar high-order semantic information and complementary discriminative information contained in multimodal data is vital. In this paper, we propose a cross-modal semantic enhancement network (CMSE) for multimodal semantic information mining and fusion. Our proposed CMSE framework extracts features from the image on multiple scales, capturing more representative local sparse features with different sizes of convolution kernels. To represent high-level semantic features related to land cover, we establish a Gaussian-weighted matrix and semantically transform the spatial and spectral features of distinct branches. Finally, we build a multi-level residual fusion module to incrementally fuse spectral features from HSI and elevation features from LiDAR. Additionally, we introduce a cross-modal semantically constrained loss to guide multimodal semantic feature alignment. We evaluate our approach on three multimodal remote sensing datasets, namely the Houston2013, Trento, and MUUFL datasets. The experimental results demonstrate that our proposed CMSE model achieves superior performance in terms of accuracy and robustness compared to other related deep networks.

Multi-modality image fusion applied to improve image quality has drawn great attention from researchers in recent years. However, noise is actually generated in images captured by different types of imaging sensors, which can seriously affect the performance of multi-modality image fusion. As the fundamental method of noisy image fusion, source images are denoised first, and then the denoised images are fused. However, image denoising can decrease the sharpness of source images to affect the fusion performance. Additionally, denoising and fusion are processed in separate processing modes, which causes an increase in computation cost. To fuse noisy multi-modality image pairs accurately and efficiently, a multi-modality image simultaneous fusion and denoising method is proposed. In the proposed method, noisy source images are decomposed into cartoon and texture components. Cartoon-texture decomposition not only decomposes source images into detail and structure components for different image fusion schemes, but also isolates image noise from texture components. A Gaussian scale mixture (GSM) based sparse representation model is presented for the denoising and fusion of texture components. A spatial domain fusion rule is applied to cartoon components. The comparative experimental results confirm the proposed simultaneous image denoising and fusion method is superior to the state-of-the-art methods in terms of visual and quantitative evaluations.

: This one-year effort will focus on the transition of FERI's machine learning algorithms for HyperSpectral Imagery (HSI) in the VSW into a distributable code set. This will provide a stable code platform for the application and transition of machine learning-based hyperspectral classification techniques into 6.3/6.4 programs. (This work was funded mid-year 2008.) Our objective is to focus on three areas of application research and transitions. First, we will transition our machine learning-based algorithms and computer code for the determination of bathymetry, bottom type, and water column Inherent Optical Properties from HyperSpectral Imagery (HSI) into a deliverable Message Passing Interface (MPI) program that may be easily used by other research and military operators. Second, we will use this program to determine the impacts of the granularity of the classification database on the inversion bathymetry, bottom type, and IOPs. Third, we will move beyond the use of single pixel HSI inversion to the use of spatial context-filtering to remove pixel-topixel noise inherent in the HSI data.

Hyperspectral imagery (HSI) has high spectral and spatial resolutions, which are essential for anomaly detection (AD). Many anomaly detectors assume that the spectrum signature of HSI pixels can be modeled with a Gaussian distribution, which is actually not accurate and often leads to many false alarms. Therefore, a sparsity model without any distribution hypothesis is usually employed. Dimensionality reduction (DR) as a preprocessing step for HSI is important. Principal component analysis as a conventional DR method is a linear projection and cannot exploit the nonlinear properties in hyperspectral data, whereas locally linear embedding (LLE) as a local, nonlinear manifold learning algorithm works well for DR of HSI. A modified algorithm of sparsity divergence index based on locally linear embedding (SDI-LLE) is thus proposed. First, kernel collaborative representation detection is adopted to calculate the sparse dictionary matrix of local reconstruction weights in LLE. Then, SDI is obtained both in the spectral and spatial domains, where spatial SDI is computed after DR by LLE. Finally, joint SDI, combining spectral SDI and spatial SDI, is computed, and the optimal SDI is performed for AD. Experimental results demonstrate that the proposed algorithm significantly improves the performance, when compared with its counterparts.

Hyperspectral Imagery (HSI) data are inherently more expensive and difficult to acquire than Multispectral Imagery (MSI) data. Most HSI acquisitions are currently limited to relatively low spatial resolution with the inherent limitation that they are not able to spatially resolve many important targets. Higher spatial resolution MSI data are available, however, they typically don’t provide sufficient spectral resolution to allow unambiguous identification of specific targets. We describe a new method using predictive modeling of combined HSI/MSI data to generate simulated high-spatial- resolution, high-spectral-resolution image data. Targets observed by the low spatial resolution HSI sensor are spectrally resolved through linear spectral unmixing, but not spatially resolved. Targets observed by higher spatial resolution MSI sensor are partially spatially resolved but lower spectral resolution doesn’t allow target identification. The fused data are used to predict the subpixel locations of HSI-identified subpixel targets within the superior MSI spatial resolution. Comparison of individual HSI endmember spectra to MSI signatures at multiple MSI pixels allows mapping the spatial locations of specific HSI endmembers creating a simulated HSI dataset that allows location of spectrally resolved targets. The combined data incorporate the best features of HSI/MSI to allow improved target detection and characterization.

Although hyperspectral remotely sensed data are believed to be nonlinear, they are often modeled and processed by algorithms assuming that the data are realizations of some linear stochastic processes. This is likely due to the reason that either the nonlinearity of the data may not be strong enough, and the algorithms based on linear data assumption may still do the job, or the effective algorithms that are capable of dealing with nonlinear data are not widely available. The simplification on data characteristics, however, may compromise the effectiveness and accuracy of information extraction from hyperspectral imagery. In this paper, we are investigating the existence of non- linearity in hyperspectral data represented by a 4-m Airborne Visible/Infrared Imaging Spectrometer image acquired over an area of coastal forests on Vancouver Island. The method employed for the investigation is based on the statistical test using surrogate data, an approach often used in nonlinear time series analysis. In addition to the high-order autocorrelation, spectral angle is utilized as the discriminating statistic to evaluate the differences between the hyperspectral data and their surrogates. To facilitate the statistical test, simulated data sets are created under linear stochastic constraints. Both simulated and real hyperspectral data are rearranged into a set of spectral series where the spectral and spatial adjacency of the original data is maintained as much as possible. This paper reveals that the differences are statistically significant between the values of discriminating statistics derived from the hyperspectral data and their surrogates. This indicates that the selected hyperspectral data are nonlinear in the spectral domain. Algorithms that are capable of explicitly addressing the nonlinearity are needed for processing hyperspectral remotely sensed data.

Our objective is to identify and map individuals of nine tree species in a Hawaiian lowland tropical forest by comparing the performance of a variety of semi-supervised classifiers. A method was adapted to process hyperspectral imagery, LiDAR intensity variables, and LiDAR-derived canopy height and use them to assess the identification accuracy. We found that semi-supervised Support Vector Machine classification using tensor summation kernel was superior to supervised classification, with demonstrable accuracy for at least eight out of nine species, and for all combinations of data types tested. We also found that the combination of hyperspectral imagery and LiDAR data usually improved species classification. Both LiDAR intensity and LiDAR canopy height proved useful for classification of certain species, but the improvements varied depending upon the species in question. Our results pave the way for target-species identification in tropical forests and other ecosystems.

Remote sensing of individual tree species has many applications in resource management, biodiversity assessment, and conservation. Airborne remote sensing using light detection and ranging (LiDAR) and hyperspectral sensors has been used extensively to extract biophysical traits of vegetation and to detect species. However, its application for individual tree mapping remains limited due to the technical challenges of precise coalignment of images acquired from different sensors and accurately delineating individual tree crowns (ITCs). In this study, we developed a generic workflow to map tree species at ITC level from hyperspectral imagery and LiDAR data using a combination of well established and recently developed techniques. The workflow uses a nonparametric image registration approach to coalign images, a multiclass normalized graph cut method for ITC delineation, robust principal component analysis for feature extraction, and support vector machine for species classification. This workflow allows us to automatically map tree species at both pixel- and ITC-level. Experimental tests of the technique were conducted using ground data collected from a fully mapped temperate woodland in the UK. The overall accuracy of pixel-level classification was 91%, while that of ITC-level classification was 61%. The test results demonstrate the effectiveness of the approach, and in particular the use of robust principal component analysis to prune the hyperspectral dataset and reveal subtle difference among species.

Due to the data-hungry nature of deep neural networks and the high dimensionality of hyperspectral imagery (HSI) data, the scarcity of training samples remains a significant challenge for HSI classification based on deep learning. Recently, the pairing or recombining of samples has proven to be an efficient approach to increase the amount of input training data, and accordingly, the architecture of a network trained by these samples has to be redesigned, since the input is no longer a single sample. Following this strategy, in this article, we propose a spatial–spectral relation network (SS-RN) for HSI classification with limited training samples. The SS-RN takes advantage of the relation network architecture to precisely capture the profound similarity between samples. Additionally, to make SS-RN more suitable for HSI classification, an entire three-dimensional (3-D) neighborhood instead of the isolated pixel is considered to explore both spectral and spatial information thoroughly. Multiple support samples are also selected for each class to make sure that the extracted features are stable enough to avoid the intraclass similarity and interclass dissimilarity problem of HSI. The network is composed on the basis of 3-D convolutional neural network blocks with the aim of extracting spatial–spectral features. Extensive experiments on three widely used HSI datasets demonstrate that the proposed method can achieve better classification accuracy than conventional deep learning methods with limited labeled training samples.

Hyperspectral image fusion is a key technique of hyperspectral data processing. In recent years, many fusion methods have been proposed, but there is little work concerning evaluation of the performances of different image fusion methods. In this paper, a method called quantitative correlation analysis (QCA) is proposed, which provides a quantitative measure of the information transferred by an image fusion technique into the output image. Using the proposed method, the performances of different image fusion methods can be compared and analyzed directly based on the images of before and after performing the fusion. The correlation information entropy, based on the developed QCA, is also proposed and testified by numerical simulations. Typical hyperspectral data are applied to the proposed method. The results show that the method is effective, and its conclusions agree with the classification results in applications.

In this paper, we evaluate the contribution of the classification constrained dimensionality reduction (CCDR) algorithm to the performance of several classifiers. We present an extension to previously introduced CCDR algorithm to multiple hypotheses. We investigate classification performance using the CCDR algorithm on hyperspectral satellite imagery data. We demonstrate the performance gain for both local and global classifiers and demonstrate a 10% improvement of the k-nearest neighbors algorithm performance. We present a connection between intrinsic dimension estimation and the optimal embedding dimension obtained using the CCDR algorithm.