Hyperspectral Image Acquisition Methods and Processing Techniques Based on Traditional and Deep Learning Methodologies-A Study

Feature extraction (FE) is a crucial research area in hyperspectral image (HSI) processing. Recently, due to the powerful ability of deep learning (DL) to extract spatial and spectral features, DL-based FE methods have shown great potentials for HSI processing. However, most of the DL-based FE methods are supervised, and the training of them suffers from the absence of labeled samples in HSIs severely. The training issue of supervised DL-based FE methods limits their application on HSI processing. To address this issue, in this paper, a novel modified generative adversarial network (GAN) is proposed to train a DL-based feature extractor without supervision. The designed GAN consists of two components, which are a generator and a discriminator. The generator can focus on the learning of real probability distributions of data sets and the discriminator can extract spatial-spectral features with superior invariance effectively. In order to learn upsampling and downsampling strategies adaptively during FE, the proposed generator and discriminator are designed based on a fully deconvolutional subnetwork and a fully convolutional subnetwork, respectively. Moreover, a novel min-max cost function is designed for training the proposed GAN in an end-to-end fashion without supervision, by utilizing the zero-sum game relationship between the generator and discriminator. Besides, the proposed modified GAN replaces the original Jensen-Shannon divergence with the Wasserstein distance, aiming to mitigate the unstability and difficulty of the training of GAN frameworks. Experimental results on three real data sets validate the effectiveness of the proposed method.

Dimensionality reduction is a key step in hyperspectral image processing. Recent investigations are looking into nonlinear manifold learning for dimensionality reduction in hyperspectral imagery. Nonlinear manifold learning methods such as Isomap, and Local Linear Embedding are use to recover the low dimensional representation of an unknown nonlinear manifold for high dimensional data where it is important to retain the neighborhood structure of the manifold. Although these algorithms use different philosophies for recovering the nonlinear manifold, they all incorporate neighborhood information from each data point to construct a weighted graph having the data points as vertices. Thus the performance of these methods highly depends on how well these neighborhoods are selected, since all subsequent steps rely on it. The k -NN algorithm is the most widely used technique for neighbor selection in manifold learning. However, it can result in a disconnected graph and it does not fully exploit spatial neighborhood information from the image in selecting points to form the neighborhoods. In this paper, recently proposed methods for constructing the weighted graph in manifold learning are studied: k -VC, k -EC and k -MST, which have the advantage of creating connected graphs and have performed well in artificial data sets. Spatial information of the hyperspectral images is included in the manifold learning process by using spatial coherence. Experiments are conducted with artificial data and hyperspectral images. For hyperspectral images, classification accuracy was used as an indirect measure to understand how the low dimensional embedding of the data reduces dimensionality while still maintaining good discrimination performance.

Target detection is a key issue in processing hyperspectral images (HSIs). Spectral-identification-based algorithms are sensitive to spectral variability and noise in acquisition. In most cases, both the target spatial distributions and the spectral signatures are unknown, so each pixel is separately tested and appears as a target when it significantly differs from the background. In this paper, we propose two algorithms to improve the signal-to-noise ratio (SNR) of hyperspectral data, leading to detectors that are robust to noise. These algorithms consist in integrating adaptive spatial/spectral filtering into the adaptive matched filter and adaptive coherence estimator. Considering the HSIs as tensor data, our approach introduces a data representation involving multidimensional processing. It combines the advantages of spatial and spectral information using an alternating least square algorithm. To estimate the signal subspace dimension in each spatial mode, we extend the Akaike information criterion, and we develop an iterative algorithm for spectral-mode rank estimation. We demonstrate the interest of integrating the quadtree decomposition to perform an adaptive 3-D filtering and thereby preserve the local image characteristics. This leads to a significant improvement in terms of denoised tensor SNR and, consequently, in terms of detection probability. The performance of our method is exemplified using simulated and real-world HYperspectral Digital Imagery Collection Experiment images.

Anomaly detection in hyperspectral images involves identifying deviations or outliers within the high-dimensional spectral data captured across numerous contiguous wavelength bands. Hyperspectral imaging provides detailed spectral information, making it a powerful tool for detecting subtle variations in materials or objects that are not visible in traditional imaging techniques. The proposed system employs advanced machine learning techniques, including convolutional neural networks (CNNs) and autoencoders, to analyse hyperspectral images for anomalies. By training the models on a dataset of normal hyperspectral images, the system learns the inherent spectral characteristics and identifies patterns of typical data. New hyperspectral data is then analysed to detect deviations that may indicate potential anomalies. This approach is particularly effective in applications such as remote sensing, environmental monitoring, precision agriculture, mineral exploration, and quality control, where detecting anomalies like land degradation, crop stress, or material defects is crucial.

Brain cancer surgery has the goal of performing an accurate resection of the tumor and preserving as much as possible the quality of life of the patient. There is a clinical need to develop non-invasive techniques that can provide reliable assistance for tumor resection in real-time during surgical procedures. Hyperspectral imaging (HSI) arises as a new, noninvasive and non-ionizing technique that can assist neurosurgeons during this difficult task. In this paper, we explore the use of deep learning (DL) techniques for processing hyperspectral (HS) images of in-vivo human brain tissue. We developed a surgical aid visualization system capable of offering guidance to the operating surgeon to achieve a successful and accurate tumor resection. The employed HS database is composed of 26 in-vivo hypercubes from 16 different human patients, among which 258,810 labelled pixels were used for evaluation. The proposed DL methods achieve an overall accuracy of 95% and 85% for binary and multiclass classifications, respectively. The proposed visualization system is able to generate a classification map that is formed by the combination of the DL map and an unsupervised clustering via a majority voting algorithm. This map can be adjusted by the operating surgeon to find the suitable configuration for the current situation during the surgical procedure.

The primary role of the oily secretion (meibum) is to ensure tear film stability and retard evaporation. In addition to these is providing ocular surface lubrication, which is necessary for smooth eyelid movements. When a gland is blocked, it is described as a Meibomian gland cyst (MGC), which can be a meibomian cyst, usually referred to as chalazion (eye bump), or in the case of inflammation, it is considered to be a hordeolum (sty or stye). Topical ophthalmic ointments and eyelid heat massages can treat early diagnosed MGC; otherwise, surgical operation is required. The current techniques of diagnosing MGC are usually uncomfortable or invasive, such as examining the tarsal plate after everting the eyelid or by biopsy procedures. The purpose of this work is to propose a non-invasive MGC evaluation and classification technique using hyperspectral imaging and image processing. The proposed technique was carried out on a single patient (i.e., case study) for a period of 4 months to monitor the MGC evolution until postsurgery-recovery and was compared with a normal eyelid patient. The collected hypercube data were processed using Multivariate Curve Resolution (MCR) and image analysis to classify the MGC severity levels. The proposed work built the threshold of the complete system, where early diagnosis of an MGC is possible before it becomes visible to the eye, hence reducing the treatment duration and avoiding any further complications that require clinical lancing. Also, the system can be used as a postoperative check postsurgery to make sure the eyelid went back to normal.

This paper reviews progress in hyperspectral remote sensing (HRS) in China, focusing on the past three decades. China has made great achievements since starting in this promising field in the early 1980s. A series of advanced hyperspectral imaging systems ranging from ground to airborne and satellite platforms have been designed, built, and operated. These include the field imaging spectrometer system (FISS), the Modular Airborne Imaging Spectrometer (MAIS), and the Chang'E-I Interferometer Spectrometer (IIM). In addition to developing sensors, Chinese scientists have proposed various novel image processing techniques. Applications of hyperspectral imaging in China have been also performed including mineral exploration in the Qilian Mountains and oil exploration in Xinjiang province. To promote the development of HRS, many generic and professional software tools have been developed. These tools such as the Hyperspectral Image Processing and Analysis System (HIPAS) incorporate a number of special algorithms and features designed to take advantage of the wealth of information contained in HRS data, allowing them to meet the demands of both common users and researchers in the scientific community.

Hyperspectral imagery has emerged as a popular sensing modality for a variety of applications, and sparsity-based methods were shown to be very effective to deal with challenges coming from high dimensionality in most hyperspectral classification problems. In this paper, we challenge the conventional approach to hyperspectral classification that typically builds sparsity-based classifiers directly on spectral reflectance features or features derived directly from the data. We assert that hyperspectral image (HSI) processing can benefit very significantly by decoupling data into geometrically distinct components since the resulting decoupled components are much more suitable for sparse representation-based classifiers. Specifically, we apply morphological separation to decouple data into texture and cartoon-like components, which are sparsely represented using local discrete cosine bases and multiscale shearlets, respectively. In addition to providing a structured sparse representation, this approach allows us to build classifiers with invariance properties specific to each geometrically distinct component of the data. The experimental results using real-world HSI data sets demonstrate the efficacy of the proposed framework for classifying multichannel imagery under a variety of adverse conditions-in particular, small training sample size, additive noise, and rotational variabilities between training and test samples.

We present a framework for hyperspectral image (HSI) analysis validation, specifically abundance fraction estimation based on HSI measurements of water soluble dye mixtures printed on microarray chips. In our work we focus on the performance of two algorithms, the Least Absolute Shrinkage and Selection Operator (LASSO) and the Spatial LASSO (SPLASSO). The LASSO is a well known statistical method for simultaneously performing model estimation and variable selection. In the context of estimating abundance fractions in a HSI scene, the “sparse” representations provided by the LASSO are appropriate as not every pixel will be expected to contain every endmember. The SPLASSO is a novel approach we introduce here for HSI analysis which takes the framework of the LASSO algorithm a step further and incorporates the rich spatial information which is available in HSI to further improve the estimates of abundance. In our work here we introduce the dye mixture platform as a new benchmark data set for hyperspectral biomedical image processing and show our algorithm’s improvement over the standard LASSO.

The use of hyperspectral imaging in the VNIR (400–1000 nm) and SWIR range (1000–2500 nm) for detecting counterfeit drugs with identical API composition

Dimensionality reduction is an essential task in hyperspectral image processing. How to preserve the original intrinsic structure information and enhance the discriminant ability is still a challenge in this area. Recently, with the advantage of preserving global intrinsic structure information, low-rank representation has been applied to dimensionality reduction and achieved promising performance. By exploiting the submanifold information of the original data set, multimanifold learning is effective in enhancing the discriminant ability of the processed data set. In addition, due to the ability of preserving the spatial neighborhood structure information, the tensor analysis has become a popular technique for hyperspectral image processing. Motivated by the above-mentioned analysis, a novel tensor-based low-rank graph with multimanifold regularization (T-LGMR) for dimensionality reduction of hyperspectral images is proposed in this paper. In the T-LGMR, a low-rank constraint is employed to preserve the global data structure while multimanifold information is utilized to enhance the discriminant ability, and tensor representation is used to preserve the spatial neighborhood information. Finally, dimensionality reduction is achieved in the graph embedding framework. Experimental results on three real hyperspectral data sets demonstrate the superiority of the proposed method over several state-of-the-art approaches.

Recent advances in remote sensing have shown great potential for different kinds of applications such as vegetation and crop supervision. Using hyperspectral image processing, this process can be used to assess large areas of land, allowing for a fast and non-invasive analysis of variables such as water stress, diseases, and type of crops. In this context, representative spatial-spectral features are crucial for developing the data classification process of hyperspectral remote sensing images. In this way, extended attribute profiles (EAPs) are a powerful tool for efficiently capturing spatial and spectral properties. In addition, deep learning networks have shown a remarkable feature extraction performance in many applications, such as in the processing of hyperspectral images. In this paper, a data classification model is proposed that uses EAPs and an inception network to generate deep spatial-spectral features, which are labelled using a Riemannian classifier as a fine-tuning stage. This model was validated using three different datasets. Results obtained shows that the classification strategy proposed in this paper has better performance regarding the results generated by a traditional model of stacked convolutional neural networks (CNNs). Quantitatively speaking, the overall accuracy for the experiments with the proposed model and the tested data sets is greater than 99%.

Compressed hyperspectral image reconstruction plays an important role in hyperspectral image processing, which aims to reduce the cost of transmission and recover the hyperspectral image from a small amount of data. Compressed sensing based methods have provided a new way for hyperspectral image reconstruction; however, the reconstruction problem is an ill-posed linear inverse problem, which needs some prior knowledge as constraint conditions. To make the most use of the prior knowledge, we introduce a block low-rank and three-dimension weighted total variation reconstruction algorithm. First, considering hyperspectral images have a piecewise smooth structure in both spatial and spectral domain, we use the total variation constraint in all three dimensions, and weight is utilized to trade off the contribution of the total variation in the different dimensions. Second, low-rank regularization is adopted in the reconstruction algorithm. To overcome the problem that the spectrum curves of different materials become similar which caused by the low-rank regular term, we divide the hyperspectral image into some non-overlapping blocks in the spatial domain and use low-rank regularization individually. Finally, taking into account the phenomenon that the rank of a block which has many elements in a given scene is usually higher than a normal block, we use one percent pixels in the original data to decide whether to perform low-rank constraint on the block or not. We use the fast iterative shrinkage/thresholding algorithm to solve the reconstruction problem. Extensive experiments demonstrate the superiority of the proposed model compared to other state-of-the-art reconstruction algorithms.

Hyperspectral sensors provide an opportunity to capture the intensity of high spatial/spectral information and enable applications for high-level earth observation missions, such as accurate land cover mapping and target/object detection. Currently, convolutional neural networks (CNNs) are good at coping with hyperspectral image processing tasks because of the strong spatial and spectral feature extraction ability brought by hierarchical structures, but the convolution operation in CNNs is limited to local feature extraction in both dimensions. In the meanwhile, the introduction of the Transformer structure has provided an opportunity to capture long-distance dependencies between tokens from a global perspective; however, Transformer-based methods have a restricted ability to extract local information because they have no inductive bias, as CNNs do. To make full use of these two methods’ advantages in hyperspectral image processing, a dual-flow architecture named Hyper-LGNet to couple local and global features is firstly proposed by integrating CNN and Transformer branches to deal with HSI spatial-spectral information. In particular, a spatial-spectral feature fusion module (SSFFM) is designed to maximally integrate spectral and spatial information. Three mainstream hyperspectral datasets (Indian Pines, Pavia University and Houston 2013) are utilized to evaluate the proposed method’s performance. Comparative results show that the proposed Hyper-LGNet achieves state-of-the-art performance in comparison with the other nine approaches concerning overall accuracy (OA), average accuracy (AA) and kappa index. Consequently, it is anticipated that, by coupling CNN and Transformer structures, this study can provide novel insights into hyperspectral image analysis.

Hyperspectral imaging (HSI) can extract information from scenes on the earth surface acquired by airborne or spaceborne sensors. On-board processing of HSI is characterized by large datasets on one side and limited processing time and communication links on the other. The CCSDS-123 algorithm is a compression standard assembled for space-related application that allows compacted data transmission via a transmission link. In this paper, a low-complexity high-throughput field-programmable gate array (FPGA) implementation of CCSDS-123 compression algorithm with band interleaved by pixel ordering is presented. Hardware accelerators implemented in FPGAs are increasingly used for custom tasks due to their efficiency, low power, and reconfigurability. The proposed implementation of CCSDS-123 compression standard has been tested on ZedBoard development board containing a Zynq-7020 FPGA. The results are verified against an existing software implementation. The synthesized design can perform on-the-fly processing of hyperspectral images with maximum operating frequency of 147 MHz. The achieved throughput of 147 Msamples/s (2.35 Gb/s) is higher when compared with the throughput reported in recent state-of-the-art FPGA implementations.