Wavelet-Based Classification of Hyperspectral Images Using Extended Morphological Profiles on Graphics Processing Units

Compressive sensing (CS) has been employed to compress and store hyperspectral images (HSI) to transfer extensive data efficiently, and obtaining high-quality reconstructed images is crucial for subsequent applications. Model-based methods usually capture the priors of HSI, including sparsity, global and nonlocal low-rankness, nonlocal self-similarity and global correlation. However, it is challenging to reasonably integrate different regularizations in a united framework and exploit various priors mentioned to reconstruct high-quality images. In this paper, a tensor optimization model based on the low-rank coefficient tensor and global prior (LRCTGP) is proposed for HSI reconstruction. Complementary regularization terms are integrated into a united framework and effectively promote the reconstruction results. First, we apply the mode- tensor-matrix product to decompose the original HSI rather than employing the regularization to capture the spectral low-rankness and simultaneously constrain the original and feature HSI. Then, tensor ring decomposition is employed to constrain the coefficient tensor with fewer bands, which is more efficient in the low-rank approximation and has smaller calculation costs than applying it to the original HSI. Moreover, integrating BM3D as a regularizer is more efficient than other patch-based models. Finally, considering the smoothness and global correlation, spatial-spectral total variation (SSTV) is applied to compensate for the shortcomings after decomposing the original and feature HSI, which complements the overall structure and details of reconstructed images and improves the reconstruction quality. Alternating direction method of multipliers (ADMM) is used to optimize the proposed model. Experimental results of the LRCTGP model on different HSI datasets are better than existing state-of-the-art approaches, which proves the superiority.

Powdery mildew (PM, Blumeria graminis f. sp. tritici) is a devastating disease for wheat growth and production. It is highly meaningful that the disease severities can be objectively and accurately identified by image visualization technology. In this study, an integral method was proposed based on a hyperspectral imaging dataset and machine learning algorithms. The disease severities of wheat leaves infected with PM were quantitatively identified based on hyperspectral images and image segmentation techniques. A technical procedure was proposed to perform the identification and evaluation of leaf-scale wheat PM, specifically including three primary steps of the acquisition and preprocessing of hyperspectral images, the selection of characteristic bands, and model construction. Firstly, three-dimensional reduction algorithms, namely principal component analysis (PCA), random forest (RF), and the successive projections algorithm (SPA), were comparatively used to select the bands that were most sensitive to PM. Then, three diagnosis models were constructed by a support vector machine (SVM), RF, and a probabilistic neural network (PNN). Finally, the best model was selected by comparing the overall accuracies. The results show that the SVM model constructed by PCA dimensionality reduction had the best result, and the classification accuracy reached 93.33% by a cross-validation method. There was an obvious improvement of the identification accuracy with the model, which achieved an 88.00% accuracy derived from the original hyperspectral images. This study can provide a reference for accurately estimating the disease severity of leaf-scale wheat PM and other plant diseases by non-contact measurement technology.

Hyperspectral Image (HSI) unmixing is a critical problem in remote sensing image processing. It aims to estimate the pure spectral signatures and their fractional abundances in each HSI pixel. Generally, most spectral-spatial unmixing methods do not exploit the contextual structure of HSI data cube. Therefore, in order to fully utilize the spatial information, we propose a new approach for spectral-spatial unmixing based on deep learning and Extended Morphological Profiles (EMPs). First, a latent representation from the original HSI data (spectral features) and the morphological profiles (spatial features) is obtained through the AutoEncoder (AE) network. Then, the estimation of both endmembers and abundance maps is achieved by the Convolutional AE (CAE). Experiments are performed using two HSI benchmarks which are Jasper Ridge and Samson. Compared to a spectral-spatial unmixing method, our proposed method shows significantly higher performance.

The massive parallel architecture enables graphics processing units (GPUs) to boost performance for a wide range of applications. Initially, GPUs only employ scratchpad memory as on-chip memory. Recently, to broaden the scope of applications that can be accelerated by GPUs, GPU vendors have used caches in conjunction with scratchpad memory as on-chip memory in the new generations of GPUs. Unfortunately, GPU caches face many performance challenges that arise due to excessive thread contention for cache resource. Cache bypassing, where memory requests can selectively bypass the cache, is one solution that can help to mitigate the cache resource contention problem. In this paper, we propose coordinated static and dynamic cache bypassing to improve application performance. At compile-time, we identify the global loads that indicate strong preferences for caching or bypassing through profiling. For the rest global loads, our dynamic cache bypassing has the flexibility to cache only a fraction of threads. In CUDA programming model, the threads are divided into work units called thread blocks. Our dynamic bypassing technique modulates the ratio of thread blocks that cache or bypass at run-time. We choose to modulate at thread block level in order to avoid the memory divergence problems. Our approach combines compile-time analysis that determines the cache or bypass preferences for global loads with run-time management that adjusts the ratio of thread blocks that cache or bypass. Our coordinated static and dynamic cache bypassing technique achieves up to 2.28X (average I.32X) performance speedup for a variety of GPU applications.

In order to quickly distinguish infertile eggs from fertile eggs, the hyperspectral imaging technology consisting of imaging and spectral information was used for detecting the fertile information of eggs. Before hatching eggs were incubated, a hyperspectral imaging system (wavelength between 400 to 1 000 nm) was used to acquire the images one-by-one manually. The characteristic information of ratios of length to short axis, elongation, roundness and the ratios of the yolk area to the whole area was extracted based on the images. The normalization method was used as the spectral data preprocessing, and then 155 spectral characteristic variables were extracted from 520 nm waveband through the correlation coefficient method. Principal component analysis (PCA) method was adopted to reduce the dimensions of image-spectrum fusion information; the top six principal components were extracted. Support vector machine (SVM) method was used to establish classification of fertile and infertile eggs models, which are based on image, spectrum and image-spectrum fusion information respectively. The accuracy rates of the SVM models were 84.00%, 90.00% and 93.00% respectively. The experimental results show that the model based on image-spectrum fusion information technology is superior to the single information model. Hyperspectral transmission imaging technology is effective and feasible to detect the fertile hatching eggs before incubation. Keywords: hyperspectral image, hatching eggs, information fusion, nondestructive detection DOI: 10.3965/j.ijabe.20150804.1672 Citation: Zhu Z H, Liu T, Xie D J, Wang Q H, Ma M H. Nondestructive detection of infertile hatching eggs based on spectral and imaging information. Int J Agric & Biol Eng, 2015; 8(4): 69－76.

In this paper, we propose a novel multiple kernel learning (MKL) framework to incorporate both spectral and spatial features for hyperspectral image classification, which is called multiple-structure-element nonlinear MKL (MultiSE-NMKL). In the proposed framework, multiple structure elements (MultiSEs) are employed to generate extended morphological profiles (EMPs) to present spatial-spectral information. In order to better mine interscale and interstructure similarity among EMPs, a nonlinear MKL (NMKL) is introduced to learn an optimal combined kernel from the predefined linear base kernels. We integrate this NMKL with support vector machines (SVMs) and reduce the min-max problem to a simple minimization problem. The optimal weight for each kernel matrix is then solved by a projection-based gradient descent algorithm. The advantages of using nonlinear combination of base kernels and multiSE-based EMP are that similarity information generated from the nonlinear interaction of different kernels is fully exploited, and the discriminability of the classes of interest is deeply enhanced. Experiments are conducted on three real hyperspectral data sets. The experimental results show that the proposed method achieves better performance for hyperspectral image classification, compared with several state-of-the-art algorithms. The MultiSE EMPs can provide much higher classification accuracy than using a single-SE EMP.

Imaging spectroscopy in the remote sensing is an ever emerging platform that has offered the hyperspectral imaging (HSI) which delivers the Earth’s object information in hundreds of bands. HSI integrates conventional imaging with spectroscopy to get rich spectral and spatial features of the object. However, the challenges associated with HSI are its huge dimensionality and data redundancy that requests huge space, complex computations, and lengthier processing time. Therefore, this study aims to find the optimal bands to characterize the roof surfaces using supervised classifiers. To deal with high dimensionality of hyperspectral data, this study assesses the band selection method over data transformation methods. This study provides the comparison between data reduction methods and used classifiers. The height information from LiDAR was used to characterize urban roofs above the height of 2.5 meters. The optimal bands were investigated using supervised classifiers such as artificial neural network (ANN), support vector machine (SVM), and spectral angle mapper (SAM) by comparing accuracies. The classification result shows that ANN and SVM classifiers outperform whereas SAM performed poorly in roof characterization. The band selection method worked efficiently than the transformation methods. The classification algorithm successfully identifies the optimum bands with significant accuracy.

Hyperspectral images are used in different applications in Earth and space science, and many of these applications exhibit real- or near real-time constraints. A problem when analyzing hyperspectral images is that their spatial resolution is generally not enough to separate different spectrally pure constituents (endmembers); as a result, several of them can be found in the same pixel. Spectral unmixing is an important technique for hyperspectral data exploitation, aimed at finding the spectral signatures of the endmembers and their associated abundance fractions. The development of techniques able to provide unmixing results in real-time is a long desired goal in the hyperspectral imaging community. In this paper, we describe a real-time hyperspectral unmixing chain based on three main steps: 1) estimation of the number of endmembers using the hyperspectral subspace identification with minimum error (HySime); 2) estimation of the spectral signatures of the endmembers using the vertex component analysis (VCA); and 3) unconstrained abundance estimation. We have developed new parallel implementations of the aforementioned algorithms and assembled them in a fully operative real-time unmixing chain using graphics processing units (GPUs), exploiting NVIDIA’s compute unified device architecture (CUDA) and its basic linear algebra subroutines (CuBLAS) library, as well as OpenMP and BLAS for multicore parallelization. As a result, our real-time chain exploits both CPU (multicore) and GPU paradigms in the optimization. Our experiments reveal that this hybrid GPU–CPU parallel implementation fully meets real-time constraints in hyperspectral imaging applications.

The goal of this study was to explore the potential of using near-infrared (NIR) hyperspectral imaging in combination with multivariate analysis for the identification of kiwifruits treated with exogenous plant growth regulator (EPGR). One hundred twenty kiwifruits, variety ‘Xixuan,’ treated with EPGR and another 120 ‘Xixuan’ kiwifruits without EPGR (named as normal ones) were used in the study. Hyperspectral images were acquired using a near-infrared hyperspectral imaging system in the spectral range of 865.11–1,711.71 nm. Based on the Kennard-Stone method, the samples were divided into two sets: 180 samples in calibration set and 60 samples in validation set. Standard normal variate transformation was used to preprocess obtained spectra. Principal component analysis (PCA) and successive projections algorithm (SPA) were used to select principal components and characteristic wavelengths. Partial least square (PLS) regression and support vector machine (SVM) modeling methods were used to establish models for identifying EPGR-treated kiwifruits and normal ones. The results indicated that average correct identification rates of all models were higher than 98.9 and 96.7 % for the calibration set and validation set, respectively. The identification performance of PLS was better than that of SVM, and the best model was PLS-SPA, whose average accuracy rate reached 100 % for the calibration set and 98.4 % for the validation set. The results demonstrated that NIR hyperspectral imaging technique can be used as a noninvasive method for distinguishing kiwifruits treated with EPGR from normal ones.

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial–spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial–spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial–spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3 approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training.

Recently, deep learning models based on convolutional neural networks (CNN) remain dominant in hyperspectral image (HSI) classification. However, there are some problems in CNN models, such as not good at modeling the long-distance dependencies and obtaining global context information. Different from the existing CNN-based models, an innovative classification method based on the transformer model is proposed to further improve the classification accuracy of HSI. Specifically, the proposed method first extracts the extended morphological profile (EMP) features of HSI to make full use of the spatial and spectral information while effectively reducing the number of bands. Next, a deep network model is constructed by introducing the transformer-iN-transformer (TNT) modules to carry out end-to-end classification. The outer and inner transformer models in the TNT module can extract the patch-level and pixel-level features, respectively, to make full use of the global and local information in the input EMP cubes. Experimental results on three public HSI data sets show that the proposed method can achieve better classification performance than the existing CNN-based models. In addition, using the transformer-based deep model without convolution to classify HSI provides a new idea for related research.

To overcome the difficulty of automating and intelligently classifying the ground features in remote-sensing hyperspectral images, machine learning methods are gradually introduced into the process of remote-sensing imaging. First, the PaviaU, Botswana, and Cuprite hyperspectral datasets are selected as research subjects in this study, and the objective is to process remote-sensing hyperspectral images via machine learning to realize the automatic and intelligent classification of features. Then, the basic principles of the support vector machine (SVM) and extreme learning machine (ELM) classification algorithms are introduced, and they are applied to the datasets. Next, by adjusting the parameter estimates using a restricted Boltzmann machine (RBM), a new terrain classification model of hyperspectral images that is based on a deep belief network (DBN) is constructed. Next, the SVM, ELM, and DBN classification algorithms for hyperspectral image terrain classification are analysed and compared in terms of accuracy and consistency. The results demonstrate that the average detection accuracies of ELM on the three datasets are 89.54%, 96.14%, and 96.28%, and the Kappa coefficient values are 0.832, 0.963, and 0.924; the average detection accuracies of SVM are 88.90%, 92.11%, and 91.68%, and the Kappa coefficient values are 0.768, 0.913, and 0.944; the average detection accuracies of the DBN classification model are 92.36%, 97.31%, and 98.84%, and the Kappa coefficient values are 0.883, 0.944, and 0.972. The results also demonstrate that the classification accuracy of the DBN algorithm exceeds those of the previous two methods because it fully utilizes the spatial and spectral information of hyperspectral remote-sensing images. In summary, the DBN algorithm that is proposed in this study has high application value in object classification for remote-sensing hyperspectral images.

Hyperspectral and multispectral imagery allows remote-sensing applications such as the land-cover mapping, which is a significant baseline to understand and to monitor the Earth. Furthermore, it is a relevant process for socio-economic activities. For that reason, high land-classification accuracies are imperative, and minor image processing time is essential. In addition, the process of gathering classes’ documented samples is complicated. This implies that the classification system is required to perform with a limited number of training observations. Another point worth mentioning is that there are hardly any methods that can be used analogously for hyperspectral or multispectral images. This paper aims to propose a novel classification system that can be used for both types of images. The designed classification system is composed of a novel parallel feature extraction algorithm, which utilises a cluster of two graphics processing units in combination with a multicore central processing unit (CPU), and an artificial neural network (ANN) particularly devised for the classification of the features ensued by the implemented feature extraction method. To prove the performance of the proposed classification system, it is compared with non-parallel and CPU-only-parallel implementations employing multispectral and hyperspectral databases. Moreover, experiments with different number of samples for training the classifier are performed. Finally, the proposed ANN is compared with a state-of-the-art support vector machine in classification and processing time results.

In very high resolution (VHR) remote sensing (RS) classification tasks, conventional pixel-based contextual information extraction methods such as morphological profiles (MPs), extended MPs (EMPs) and MPs with partial reconstruction (MPPR) with limited numbers, sizes and shapes of structural elements (SEs) cannot perfectly match all sizes and shapes of the objects in an image. To overcome such limitation, we introduce novel spatial feature extractors, namely, the superpixel-guided morphological profiles (SPMPs), where the superpixels are used as SEs in opening by reconstruction and closing by reconstruction operations. Moreover, to avoid possible side effects from unusual maximum and minimum values within superpixels, the mean pixel value of superpixels is adopted (SPMPsM). Additionally, new decision forest based on penalizing the attributes in previous trees, the ForestPA is introduced and evaluated through a comparative investigation on three VHR multi-/hyperspectral RS image classification tasks. Support vector machine and benchmark ensemble classifiers, including bagging, AdaBoost, MultiBoost, ExtraTrees, Random Forest and Rotation Forest, are adopted. The experimental results confirm the effectiveness and superior performances of the proposed SPMPs and SPMPsM relative to those of the MPs and MPPR. Moreover, ForestPA outperforms only bagging and is not suitable for learning from large numbers of samples with high dimensionality from the computational efficiency and classification accuracy perspective.

The superpixel-based multiple kernels model uses the average value of all pixels within superpixel as the spatial feature, which results in inaccurate extraction of edge pixels. To solve this problem, a local binary patterns and superpixel-based multiple kernels method is proposed for hyperspectral image (HSI) classification. First, the original HSI is segmented into multiple superpixels by using the entropy rate superpixel segmentation algorithm. On the HSI with superpixel index, the spectral kernel is second obtained by combining the spectral feature map with the radial basis kernel (RBF). By introducing local binary pattern (LBP) and weighted average filtering into RBF, the spatial kernels are obtained within and among superpixels. Finally, the combined kernel containing the abovementioned three kernels is inputted into the support vector machine classifier to generate a classification map. The experimental procedure in this article uses LBP to extract the information in superpixels, which effectively prevents the loss of edge features in superpixels. The experimental results show that the proposed method is superior to the state-of-the-art classifiers for HSI classification.