Morphology-based fusion method of hyperspectral image

In recent years, there has been a notable surge in the development of methods dedicated to classifying hyper spectral image samples. However, the inherent high dimensionality of hyper spectral data presents significant challenges for traditional supervised classification algorithms. Prior studies have attempted to address these challenges by incorporating active learning methods for hyper spectral data segmentation. Nevertheless, a crucial drawback of active learning lies in its dependence on labeled data samples, leading to less improved classification results for unlabeled data samples. In response to this limitation, our work introduces Supervised Active Learning (SAL) techniques that harness both spectral and spatial information. The key innovation of our research lies in the introduction of a novel hyper spectral image segmentation approach based on the k-dependence Bayesian (KDB) model. We estimate the maximizer of the posterior marginal using Symmetric Iterative Proportional Fitting (SIPF) within this KDB model. By implementing the KDB model for hyper spectral image segmentation, our goal is to enhance the classification accuracy of our SAL classifiers. The SAL framework integrates shape prior probability estimation, class density results from Improved Multi-Level Logistic (IMLL) methods, and the maximization of the posterior marginal estimated through Symmetric Iterative Proportional Fitting (SIPF). We evaluate the accuracy of our proposed MPM-SIPF-SAL system using the real ROSIS Hyper Spectral Dataset, demonstrating that our system outperforms existing methods in terms of classification accuracy.

Analysis of hyperspectral images is of great importance in many scientific disciplines. Obtaining the spectral and spatial information simultaneously from time-varying hyperspectral images is a challenging task due to their high dimensionality. In this paper, we design an interface that allows users to study hyperspectral images interactively and obtain spectral features and enhanced images at the same time. The image fusion results change dynamically with the regions of interest selected by users and convey both the spatial and spectral information. We show the usefulness of our approach using time-varying hyperspectral plant images. We compare our method with existing hyperspectral image analysis techniques. Our evaluation indicates that our interface can help users determine important bands, identify regions of interest, and generate image fusion results for time-varying hyperspectral plant images.

The hyperspectral and multispectral image fusion process, commonly known as the multisharpening process (also known as the hypersharpening process), results in the creation of unobservable high spectral and spatial resolution hyperspectral remote sensing data, by combining (or fusing) a hyperspectral image, characterized by a high spectral resolution, with, especially, a high spatial resolution multispectral one. There are several approaches in the literature performing such a process, in particular, those based on the spectral unmixing concept. The latter fusion methods aim to extract pure-material spectra from the high spectral resolution hyperspectral image and combine them with their corresponding proportions extracted from the high spatial resolution multispectral image. However, most of these methods, which mainly consider the standard linear mixing model, do not address the spectral variability phenomenon, precisely due to the considered data mixing model, which does not handle variations in illumination and atmospheric conditions or material composition inducing spectral variability. Neglecting such a phenomenon may lead to errors in the fusion process, thus decreasing the spatial and spectral qualities of sharpened products. In this paper, a new hyperspectral and multispectral data fusion method, based on the linear spectral unmixing concept that deals with spectral variability, is proposed. This approach, based on a multiplicative Nonnegative Matrix Factorization (NMF) technique, substantially extends the multiplicative Coupled NMF (CNMF) algorithm by considering a linear data mixing model that addresses the considered phenomenon. Indeed, the introduced approach minimizes new cost functions involving matrices with specific structures in order to deal with the spectral variability phenomenon. The proposed method is applied to synthetic and real data, and its effectiveness is evaluated with established performance criteria. Experimental results show that the designed algorithm globally yields sharpened hyperspectral data with fine spectral and spatial fidelities. Also, these results demonstrate that the proposed technique globally outperforms the tested literature approaches.

In order to quickly distinguish infertile eggs from fertile eggs, the hyperspectral imaging technology consisting of imaging and spectral information was used for detecting the fertile information of eggs. Before hatching eggs were incubated, a hyperspectral imaging system (wavelength between 400 to 1 000 nm) was used to acquire the images one-by-one manually. The characteristic information of ratios of length to short axis, elongation, roundness and the ratios of the yolk area to the whole area was extracted based on the images. The normalization method was used as the spectral data preprocessing, and then 155 spectral characteristic variables were extracted from 520 nm waveband through the correlation coefficient method. Principal component analysis (PCA) method was adopted to reduce the dimensions of image-spectrum fusion information; the top six principal components were extracted. Support vector machine (SVM) method was used to establish classification of fertile and infertile eggs models, which are based on image, spectrum and image-spectrum fusion information respectively. The accuracy rates of the SVM models were 84.00%, 90.00% and 93.00% respectively. The experimental results show that the model based on image-spectrum fusion information technology is superior to the single information model. Hyperspectral transmission imaging technology is effective and feasible to detect the fertile hatching eggs before incubation. Keywords: hyperspectral image, hatching eggs, information fusion, nondestructive detection DOI: 10.3965/j.ijabe.20150804.1672 Citation: Zhu Z H, Liu T, Xie D J, Wang Q H, Ma M H. Nondestructive detection of infertile hatching eggs based on spectral and imaging information. Int J Agric & Biol Eng, 2015; 8(4): 69－76.

Hyperspectral imaging provides complex image data with spectral information from many fluorescent species contained within the sample such as the fluorescent labels and cellular or pigment autofluorescence. To maximize the utility of this spectral imaging technique it is necessary to couple hyperspectral imaging with sophisticated multivariate analysis methods to extract meaningful relationships from the overlapped spectra. Many commonly employed multivariate analysis techniques require the identity of the emission spectra of each component to be known or pure component pixels within the image, a condition rarely met in biological samples. Multivariate curve resolution (MCR) has proven extremely useful for analyzing hyperspectral and multispectral images of biological specimens because it can operate with little or no a priori information about the emitting species, making it appropriate for interrogating samples containing autofluorescence and unanticipated contaminating fluorescence. To demonstrate the unique ability of our hyperspectral imaging system coupled with MCR analysis techniques we will analyze hyperspectral images of four-color in-situ hybridized rat brain tissue containing 455 spectral pixels from 550 - 850 nm. Even though there were only four colors imparted onto the tissue in this case, analysis revealed seven fluorescent species, including contributions from cellular autofluorescence and the tissue mounting media. Spectral image analysis will be presented along with a detailed discussion of the origin of the fluorescence and specific illustrations of the adverse effects of ignoring these additional fluorescent species in a traditional microscopy experiment and a hyperspectral imaging system.

Hyperspectral imaging is concerned with the measurement, analysis, and interpretation of spectra acquired from a given scene (or specific object) at a short, medium or long distance by an airbone or satellite sensor. Over the last few years, hyperspectral image data sets have been collected for a great amount of locations over the world, using a variety of instruments for Earth observation. Despite the increasing importance of hyperspectral images in remote sensing applications, there is no common repository of hyperspectral data intended to distribute and share hyperspectral data sets in the community. Quite opposite, the hyperspectral data sets which are available for public use are spread among different storage locations and present significant heterogeneity regarding the storage format, associated meta-data (if any), or ground-truth availability. As a result, the development of a standardized hyperspectral data repository is a highly desired goal in the remote sensing community. In this paper, we take a necessary first step towards the development of a digital repository for remotely sensed hyperspectral data. The proposed system allows uploading new hyperspectral data sets along with meta-data, ground-truth and analysis results, with the ultimate goal of sharing publicly available hyperspectral images within the remote sensing community. The database has been designed in order to allow storing relevant information for the hyperspectral data available through the system, including basic image characteristics (width, height, number of bands, format) and more advanced meta-data (ground-truth information, publications in which the data has been used). The current implementation consists of a front-end to ease the management of images through a web interface, thus containing both synthetic and real hyperspectral images from two highly representative instruments, such as NASAs Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite Mining District in Nevada. Most importantly, the developed system includes a spectral unmixing-based content based image retrieval (CBIR) functionality which allows searching for images on the spectral unmixing information (spectrally pure components or endmembers and their associated abundances in the scene). This information is stored as meta-data associated to each hyperspectral image instance, and then used to search and retrieve images based on information content. This paper presents the design of the system and a preliminary validation of the unmixing-based retrieval functionality using both synthetic and real hyperspectral images stored in the database.

Rooibos (Aspalathus linearis) and Honeybush (Cyclopia species) are teas indigenous to South Africa that are popularly consumed locally as well as internationally for its purported health benefits. Studies have confirmed anti-oxidant, antimutagenic and anticancer activity mainly due to the presence of polyphenols. Hyperspectral imaging (HSI) integrates conventional spectroscopy and imaging to obtain spectral and spatial information from a sample. The potential of short wave infrared (SWIR) hyperspectral imaging in combination with chemometric data analysis as a rapid quality control method for commercially important herbal teas was investigated. Images were acquired with the sisuChema short wave infrared (SWIR) hyperspectral pushbroom imaging system at a spectral range of 920 – 2514nm. Principal Component Analysis (PCA) was performed to remove background pixels and to visualize the data. Classes (i.e. Rooibos or Honeybush) were interactively assigned using the score images and plots. Partial Least Squares Discriminant Analysis (PLS-DA) classification models developed were subsequently used to accurately predict the species identity of commercially available herbal tea samples introduced as an external dataset, producing a visually interpretable result. The results showed that the labeling information is correct but that Honeybush tea is in most cases present in very low amounts. It is evident that hyperspectral imaging is an objective and non-destructive quality control method that can be successfully used to determine the botanical species included in herbal teas in addition to giving an indication of the abundance of each species in polyherbal tea mixtures.

Recent advances and new guidelines on hyperspectral and multispectral image fusion

Fusion of hyperspectral and panchromatic images using structure tensor and matting model

Encoding of multispectral and hyperspectral image data using wavelet transform and gain shape vector quantization

Most of the existing hyperspectral image change detection methods based on deep learning utilize convolutional neural networks (CNN), recurrent neural networks (RNN), and long-short-term memory (LSTM). However, CNNs primarily focus on extracting spatial information from hyperspectral images, thereby failing to fully utilize channel and temporal information. In this paper, we propose a network called STSTNet that can jointly extract spatial, spectral, and temporal feature information. To achieve a richer feature representation space, we employ a spatial attention module to extract spatial feature information from feature maps, and a spectral transformer encoder structure to extract abundant spectral sequence information. Specifically, we design two temporal information extraction structures after each spatial information extraction structure or spectral information extraction structure to capture correlations and interactions within the hyperspectral images. Extensive experiments have been conducted on three hyperspectral datasets, and the experimental results demonstrate the network’s excellent performance.

Hyperspectral (HS) images obtained from space are useful for monitoring different natural phenomena on regional to global scales. The Environmental Mapping and Analysis Program (EnMAP) is a satellite recently launched by Germany to monitor the environment and explore the capabilities of hyperspectral sensors in the 420 and 2450 nm range of the spectrum. However, the data captured by the EnMAP mission have a ground sampling distance (GSD) of 30 m. This limits the use of the data for some applications that require higher spatial resolution (<10 m). This study examines the potential for improving the resolution of hyperspectral data using high resolution multispectral (MS) data obtained by Cubesats. Specifically, this work uses the data captured by the PlanetScope constellation, which has more than 150 CubeSats in low Earth orbit, with a high spatial and temporal resolution. The approach adopted leverages (1) the spectral capability of the hyperspectral EnMAP sensor, with a bandwidth of 6.5 nm in the visible and near infrared (VNIR) range (420–1000 nm) and 10 nm in the SWIR range (900–2450 nm), and (2) the spatial capability of the multispectral PlanetScope data, with a GSD of 3 meters, to enable significant spatial improvements due to its high spatial resolution. The main components of this work include: (i) area of interest clipping (ii) data co-registration, (iii) HS-MS data fusion, and (iv) quality assessments using the Jointly Spectral and Spatial Quality Index (QNR). In this study, a 2 km x 2 km area of interest was selected in the Malaucene region of France, where six state-of-the-art HS-MS fusion methods were evaluated: (1) fast multi-band image fusion algorithm (FUSE), (2) coupled nonnegative matrix factorization (CNMF), (3) smoothing filtered-based intensity modulation (SFIMHS), (4) maximum a posteriori stochastic mixing model (MAPSMM), (5) Hyperspectral Superresolution (HySure), and (6) generalized laplacian pyramid hypersharpening (GLPHS). Quality assessments of the enhanced data showed that high spectral and spatial fidelity are maintained, with the best performing fusion method being FUSE with a QNR of 0.625 followed by the MAPSMM method with a QNR of 0.604. Overall, this study advocates the benefits associated with the fusion of hyperspectral and multispectral data to obtain enhanced EnMAP data at 3 m GSD. 

Paddy seed viability prediction based on feature fusion of color and hyperspectral image with multivariate analysis

Fusing a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI), aiming to produce a super-resolution hyperspectral image, has recently attracted increasing research interest. In this paper, a novel approach based on coupled non-negative tensor decomposition is proposed. The proposed method performs a tucker tensor factorization of a low resolution hyperspectral image and a high resolution multispectral image under the constraint of non-negative tensor decomposition (NTD). The conventional matrix factorization methods essentially lose spatio-spectral structure information when stacking the 3D data structure of a hyperspectral image into a matrix form. Moreover, the spectral, spatial, or their joint structural features have to be imposed from the outside as a constraint to well pose the matrix factorization problem. The proposed method has the advantage of preserving the spatio-spectral structure of hyperspectral images. In this paper, the NTD is directly imposed on the coupled tensors of the HSI and MSI. Hence, the intrinsic spatio-spectral structure of the HSI is represented without loss, and spatial and spectral information can be interdependently exploited. Furthermore, multilinear interactions of different modes of the HSIs can be exactly modeled with the core tensor of the Tucker tensor decomposition. The proposed method is straightforward and easy to implement. Unlike other state-of-the-art approaches, the complexity of the proposed approach is linear with the size of the HSI cube. Experiments on two well-known datasets give promising results when compared with some recent methods from the literature.

A spatial-spectral SIFT for hyperspectral image matching and classification