Chapter 5 - Hyperspectral imaging applications

Abstract. Hyperspectral image enhancement has been a concern for the remote sensing society for detailed end member detection. Hyperspectral remote sensor collects images in hundreds of narrow, continuous spectral channels, whereas multispectral remote sensor collects images in relatively broader wavelength bands. However, the spatial resolution of the hyperspectral sensor image is comparatively lower than that of the multispectral. As a result, spectral signatures from different end members originate within a pixel, known as mixed pixels. This paper presents an approach for obtaining an image which has the spatial resolution of the multispectral image and spectral resolution of the hyperspectral image, by fusion of hyperspectral and multispectral image. The proposed methodology also addresses the band remapping problem, which arises due to different regions of spectral coverage by multispectral and hyperspectral images. Therefore we apply algorithms to restore the spatial information of the hyperspectral image by fusing hyperspectral bands with only those bands which come under each multispectral band range. The proposed methodology is applied over Henry Island, of the Sunderban eco-geographic province. The data is collected by the Hyperion hyperspectral sensor and LISS IV multispectral sensor.

For many remote sensing applications it is preferable to have images with both high spectral and spatial resolutions. On this regards, hyperspectral and multispectral images have complementary characteristics in terms of spectral and spatial resolutions. In this paper we propose an approach for the fusion of low spatial resolution hyperspectral images with high spatial resolution multispectral images in order to obtain superresolution (spatial and spectral) hyperspectral images. The proposed approach is based on the assumption that, since both hyperspectral and multispectral images acquired on the same scene, the corresponding endmembers should be the same. On a first step the hyperspectral image is spectrally down-sampled in order to match the multispectral one. Then an endmember extraction algorithm is performed on the down-sampled hyperspectral image and the successive abundance estimation is performed on the multispectral one. Finally, the extracted endmembers are up-sampled back to the original hyperspectral space and then used to reconstruct the superresolution hyperspectral image according to the abundances obtained from the multispectral image.

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral-spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

This paper studies a new Bayesian optimization algorithm for fusing hyperspectral and multispectral images. The hyperspectral image is supposed to be obtained by blurring and subsampling a high spatial and high spectral target image. The multispectral image is modeled as a spectral mixing version of the target image. By introducing appropriate priors for parameters and hyperparameters, the fusion problem is formulated within a Bayesian estimation framework, which is very convenient to model the noise and the target image. The high spatial resolution hyperspectral image is then inferred from its posterior distribution. To compute the Bayesian maximum a posteriori estimator associated with this posterior, an alternating direction method of multipliers within block coordinate descent algorithm is proposed. Simulation results demonstrate the efficiency of the proposed fusion method when compared with several state-of-the-art fusion techniques.

In this research, hyperspectral and multispectral images were utilized for vegetation studies in the greenbelt corridor near Denton. EO-1 Hyperion was the hyperspectral image and Landsat Thematic Mapper (TM) was the multispectral image used for this research. In the first part of the research, both the images were classified for land cover mapping (after necessary atmospheric correction and geometric registration) using supervised classification method with maximum likelihood algorithm and accuracy of the classification was also assessed for comparison. Hyperspectral image was preprocessed for classification through principal component analysis (PCA), segmented principal component analysis and minimum noise fraction (MNF) transform. Three different images were achieved after these pre-processing of the hyperspectral image. Therefore, a total of four images were classified and assessed the accuracy. In the second part, a more precise and improved land cover study was done on hyperspectral image using linear spectral unmixing method. Finally, several vegetation constituents like chlorophyll a, chlorophyll b, caroteoids were distinguished from the hyperspectral image using feature-oriented principal component analysis (FOPCA) method and which component dominates which type of land cover particularly vegetation were correlated.

Recent advances and new guidelines on hyperspectral and multispectral image fusion

Recently, many methods have been proposed to generate a high spatial resolution (HR) hyperspectral image (HSI) by fusing HSI and multispectral image (MSI). Most methods need a precondition that HSI and MSI are well registered. However, in practice, it is hard to acquire registered HSI and MSI. In this paper, a synchronous nonnegative matrix factorization (SNMF) is proposed to directly fuse unregistered HSI and MSI. The proposed SNMF does not require the registration operation by modeling the abundances of unregistered HSI and MSI independently. Moreover, to exploit both HSI and MSI in the endmember optimization of the desired HR HSI, the unregistered HSI and MSI fusion is formulated as a bound-constrained optimization problem. A synchronous projected gradient method is proposed to solve this bound-constrained optimization problem. Experiments on both simulated and real data demonstrate that the proposed SNMF outperforms the state-of-the-art methods.

Monitoring multi-water quality of internationally important karst wetland through deep learning, multi-sensor and multi-platform remote sensing images: A case study of Guilin, China

Spectral imaging including machine vision and multispectral imaging can provide a rapid, nondestructive means to assess agricultural commodities for their quality and safety for human consumption. These nondestructive techniques generally rely on reflectance measurements; the most commonly used spectral regions range through the visible (Vis) to the near infrared (NIR). Another optical sensing technique is based on fluorescence that is generally regarded as more sensitive optical sensing tools compared to reflectance techniques. Recently, we developed two imaging systems for food safety research; a laboratory-based hyperspectral imaging system capable of both reflectance and fluorescence measurements and a multispectral laser induced fluorescence imaging system. Because many contiguous spectral bands are acquired with each hyperspectral image pixel, the resulting data quantity makes uses of hyperspectral imaging spectrometer for on-line food safety inspection impractical. Instead, the spectral and spatial information is used to determine several optimal wavebands that can be implemented to a multispectral imaging system. We present a systematic method for using hyperspectral image data to identify few wavebands to be used in multispectral detection systems. Also presented is a two-band ratio as a multispectral fusion method along with automated detection for animal feces contamination on apples. The fluorescence imaging techniques demonstrated that even minute animal feces, not clearly visible to human eye, could be detected. Practical considerations for development of multispectral fluorescence imaging systems for on-line uses are also discussed.

Fusing a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI), aiming to produce a super-resolution hyperspectral image, has recently attracted increasing research interest. In this paper, a novel approach based on coupled non-negative tensor decomposition is proposed. The proposed method performs a tucker tensor factorization of a low resolution hyperspectral image and a high resolution multispectral image under the constraint of non-negative tensor decomposition (NTD). The conventional matrix factorization methods essentially lose spatio-spectral structure information when stacking the 3D data structure of a hyperspectral image into a matrix form. Moreover, the spectral, spatial, or their joint structural features have to be imposed from the outside as a constraint to well pose the matrix factorization problem. The proposed method has the advantage of preserving the spatio-spectral structure of hyperspectral images. In this paper, the NTD is directly imposed on the coupled tensors of the HSI and MSI. Hence, the intrinsic spatio-spectral structure of the HSI is represented without loss, and spatial and spectral information can be interdependently exploited. Furthermore, multilinear interactions of different modes of the HSIs can be exactly modeled with the core tensor of the Tucker tensor decomposition. The proposed method is straightforward and easy to implement. Unlike other state-of-the-art approaches, the complexity of the proposed approach is linear with the size of the HSI cube. Experiments on two well-known datasets give promising results when compared with some recent methods from the literature.

Hyperspectral (HS) remote sensing image with finer spectral information has great advantages in feature identification and classification. However, the spatial resolution of HS image is usually low due to practical limitations. In this paper, the low-spatial-resolution HS image is fused with the high-spatial-resolution multispectral (MS) image of the same observation scene to improve its spatial resolution. A novel spectral unmixing based HS and MS image fusion approach (VSC-CNMF) is proposed, in which CNMF with minimum endmember simplex volume and abundance sparsity constraints is employed for coupled unmixing of HS and MS images. Simulative experiments are employed for verification and comparison. The experimental results illustrate that the newly proposed VSC-CNMF based HS and MS fusion algorithm outperforms several state-of-the-art unmixing based fusion approaches in cases with moderate number of endmembers.

Hyperspectral images (HSI) have high spectral and low spatial resolutions. However, multispectral images (MSI) usually have low spectral and high spatial resolutions. In various applications HSI with high spectral and spatial resolutions are required. In this paper, a new method for spatial resolution enhancement of HSI using high resolution MSI based on sparse coding and linear spectral unmixing (SCLSU) is introduced. In the proposed method (SCLSU), high spectral resolution features of HSI and high spatial resolution features of MSI are fused. In this case, the sparse representation of some high resolution MSI and linear spectral unmixing (LSU) model of HSI and MSI is simultaneously used in order to construct high resolution HSI (HRHSI). The fusion process of HSI and MSI is formulated as an ill-posed inverse problem. It is solved by the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) and an orthogonal matching pursuit (OMP) algorithm. Finally, the proposed algorithm is applied to the Hyperion and ALI datasets. Compared with the other state-of-the-art algorithms such as Coupled Nonnegative Matrix Factorization (CNMF) and local spectral unmixing, the SCLSU has significantly increased the spatial resolution and in addition the spectral content of HSI is well maintained.

Remote sensing technology has undeniable importance in various industrial applications, such as mineral exploration, plant detection, defect detection in aerospace and shipbuilding, and optical gas imaging, to name a few. Remote sensing technology has been continuously evolving, offering a range of image modalities that can facilitate the aforementioned applications. One such modality is Hyperspectral Imaging (HSI). Unlike Multispectral Images (MSI) and natural images, HSI consist of hundreds of bands. Despite their high spectral resolution, HSI suffer from low spatial resolution in comparison to their MSI counterpart, which hinders the utilization of their full potential. Therefore, spatial enhancement, or Super Resolution (SR), of HSI is a classical problem that has been gaining rapid attention over the past two decades. The literature is rich with various SR algorithms that enhance the spatial resolution of HSI while preserving their spectral fidelity. This paper reviews and discusses the most important algorithms relevant to this area of research between 2002-2022, along with the most frequently used datasets, HSI sensors, and quality metrics. Meta-analysis are drawn based on the aforementioned information, which is used as a foundation that summarizes the state of the field in a way that bridges the past and the present, identifies the current gap in it, and recommends possible future directions.

The intent of this study was to develop a technique for weight and ripeness estimation of oil palm (Elaeis guieensis Jacq. var. tenera) bunches from hyperspectral and RGB color images. In the experiments, color and hyperspectral images of the bunch were acquired from four different angles, each differing by 90 degrees. Acquired RGB color images were converted to HSI and L*a*b color space. Gray-scale thresholds were used to identify the area of the bunch and the area of space between the fruits. The total number of pixels in the bunch and the space were counted, respectively. In the hyperspectral images, the total number of pixels in the bunch was also counted from an image composed of three wavelengths (560 nm, 680 nm, and 740 nm), while the total number of pixels of space between fruits was obtained at a wavelength of 910 nm. From these sets of data, weight-estimation equations were determined by linear regression (LR) or multiple linear regression (MLR). As a result, the coefficient of determination (R2) of actual weight and estimated weight were at a level of 0.989 and 0.992 for color and hyperspectral images, respectively. Estimation of oil palm bunch ripeness was also tested. Bunches belonging to 4 classes of ripeness (overripe, ripe, underripe, and unripe) were used for this study. Since ripeness estimation from overall data from a bunch was quite difficult, we focused on the difference in colors or reflectivity of the portion concealed and not-concealed with fronds. Euclidean distances between the test sample and the standard 4 classes of ripeness were calculated, and the test sample was classified into the ripeness class that had the shortest distance from the sample. In the classification based on color image, average RGB values of concealed and not-concealed areas were used, while in hyperspectral images the average intensity values of fruits pixels from the concealed area were used. The results of validation experiments with the developed estimation methods indicated acceptable estimation accuracy, and a possibility for practical use to estimate the ripeness of oil palm bunches.

In this work, a technique is presented for the fusion of multi-spectral (MS) and hyperspectral (HS) images to enhance the spatial resolution of the latter. The technique works in the wavelet domain, and is based on a Bayesian estimation of the HS image, assuming a joint normal model for the images, and an additive noise imaging model for the HS image. An appropriate estimation strategy is also proposed. The technique is compared to its counterpart in the spatial domain, and validated for noisy conditions. Further, its performance is compared to several state-of-the-art pansharpening techniques, in the case where the MS image becomes a panchromatic image, and to some MS and HS image fusion techniques from the literature.