A novel spatial recurrent neural network for hyperspectral imagery classification

Convolutional neural network (CNN) is well known for its capability of feature learning and has made revolutionary achievements in many applications, such as scene recognition and target detection. In this paper, its capability of feature learning in hyperspectral images is explored by constructing a five-layer CNN for classification (C-CNN). The proposed C-CNN is constructed by including recent advances in deep learning area, such as batch normalization, dropout, and parametric rectified linear unit (PReLU) activation function. In addition, both spatial context and spectral information are elegantly integrated into the C-CNN such that spatial-spectral features are learned for hyperspectral images. A companion feature-learning CNN (FL-CNN) is constructed by extracting fully connected feature layers in this C-CNN. Both supervised and unsupervised modes are designed for the proposed FL-CNN to learn sensor-specific spatial-spectral features. Extensive experimental results on four benchmark data sets from two well-known hyperspectral sensors, namely airborne visible/infrared imaging spectrometer (AVIRIS) and reflective optics system imaging spectrometer (ROSIS) sensors, demonstrate that our proposed C-CNN outperforms the state-of-the-art CNN-based classification methods, and its corresponding FL-CNN is very effective to extract sensor-specific spatial-spectral features for hyperspectral applications under both supervised and unsupervised modes.

Hyperspectral images have the characteristics of high spectral resolution and low spatial resolution, which will make the extracted features insufficient and lack detailed information about ground objects, thus affecting the accuracy of classification. The numerous spectral bands of hyperspectral images contain rich spectral features but also bring issues of noise and redundancy. To improve the spatial resolution and fully extract spatial and spectral features, this article proposes an improved feature enhancement and extraction model (IFEE) using spatial feature enhancement and attention-guided bidirectional sequential spectral feature extraction for hyperspectral image classification. The adaptive guided filtering is introduced to highlight details and edge features in hyperspectral images. Then, an image enhancement module composed of two-dimensional convolutional neural networks is used to improve the resolution of the image after adaptive guidance filtering and provide a high-resolution image with key features emphasized for the subsequent feature extraction module. The proposed spectral attention mechanism helps to extract more representative spectral features, emphasizing useful information while suppressing the interference of noise. Experimental results show that our method outperforms other comparative methods even with very few training samples.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

In hyperspectral image processing, classification is one of the most popular research topics. In recent years, research progress made in deep-learning-based hierarchical feature extraction and classification has shown a great power in many applications. In this paper, we propose a novel local spatial sequential (LSS) method, which is used in a recurrent neural network (RNN). Using this model, we can extract local and semantic information for hyperspectral image classification. First, we extract low-level features from hyperspectral images, including texture and differential morphological profiles. Second, we combine the low-level features together and propose a method to construct the LSS features. Afterwards, we build an RNN and use the LSS features as the input to train the network for optimizing the system parameters. Finally, the high-level semantic features generated by the RNN is fed into a softmax layer for the final classification. In addition, a nonlocal spatial sequential method is presented for the recurrent neural network model (NLSS-RNN) to further enhance the classification performance. NLSS-RNN finds nonlocal similar structures to a given pixel and extracts corresponding LSS features, which not only preserve the local spatial information, but also integrate the information of nonlocal similar samples. The experimental results on three publicly accessible datasets show that our proposed method can obtain competitive performance compared with several state-of-the-art classifiers.

ABSTRACTGraph-based methods are developed to efficiently extract data information. In particular, these methods are adopted for high-dimensional data classification by exploiting information residing on weighted graphs. In this paper, we propose a new hyperspectral texture classifier based on graph-based wavelet transform. This recent graph transform allows extracting textural features from a constructed weighted graph using sparse representative pixels of hyperspectral image. Different measurements of spectral similarity between representative pixels are tested to decorrelate close pixels and improve the classification precision. To achieve the hyperspectral texture classification, Support Vector Machine is applied on spectral graph wavelet coefficients. Experimental results obtained by applying the proposed approach on Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and Reflective Optics System Imaging Spectrometer (ROSIS) datasets provide good accuracy which could exceed 98.7%. Compared to other famous classification methods as conventional deep learning-based methods, the proposed method achieves better classification performance. Results have shown the effectiveness of the method in terms of robustness and accuracy.

Hyperspectral remote sensing offers a powerful tool in many different application contexts. The imbalance between the high dimensionality of the data and the limited availability of training samples calls for the need to perform dimensionality reduction in practice. Among traditional dimensionality reduction techniques, feature extraction is one of the most widely used approaches due to its flexibility to transform the original spectral information into a subspace. In turn, band selection is important when the application requires preserving the original spectral information (especially the physically meaningful information) for the interpretation of the hyperspectral scene. In the case of hyperspectral image classification, both techniques need to discard most of the original features/bands in order to perform the classification using a feature set with much lower dimensionality. However, the discriminative information that allows a classifier to provide good performance is usually classdependent and the relevant information may live in weak features/bands that are usually discarded or lost through subspace transformation or band selection. As a result, in practice, it is challenging to use either feature extraction or band selection for classification purposes. Relevant lines of attack to address this problem have focused on multiple feature selection aiming at a suitable fusion of diverse features in order to provide relevant information to the classifier. In this paper, we present a new dimensionality reduction technique, called multiple criteria-based spectral partitioning, which is embedded in an ensemble learning framework to perform advanced hyperspectral image classification. Driven by the use of a multiple band priority criteria that is derived from classic band selection techniques, we obtain multiple spectral partitions from the original hyperspectral data that correspond to several band subgroups with much lower spectral dimensionality as compared with the original band set. An ensemble learning technique is then used to fuse the information from multiple features, taking advantage of the relevant information provided by each classifier. Our experimental results with two real hyperspectral images, collected by the reflective optics system imaging spectrometer (ROSIS) over the University of Pavia in Italy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) over the Salinas scene, reveal that our presented method, driven by multiple band priority criteria, is able to obtain better classification results compared with classic band selection techniques. This paper also discusses several possibilities for computationally efficient implementation of the proposed technique using various high-performance computing architectures.

Recurrent neural networks, especially the convolutional long short-term memory (ConvLSTM), have attracted plenty of attention and shown promising results due to their ability in modeling long-term dependencies in many research fields. In this paper, a lightweight tensor attention-driven ConvLSTM neural network (TACLNN) is proposed for hyperspectral image (HSI) classification. Firstly, to reduce the trainable parameters and memory requirements of ConvLSTM (specifically, the 2-D version of LSTM, i.e., ConvLSTM2D), a lightweight ConvLSTM2D cell is developed by utilizing tensor-train decomposition, resulting in a TT-ConvLSTM2D cell, with which a spatial-spectral TT-ConvLSTM 2-D neural network (SSTTCL2DNN) is built. However, it is inevitable for SSTTCL2DNN to obtain lower accuracies for HSI classification. To recover the accuracy loss caused by the TT-ConvLSTM2D cell in SSTTCL2DNN, a learnable tensor attention residual block (TARB) module is built to further enhance its geometrical structure. When applied to three widely used HSI benchmarks, the proposed TACLNN model outperforms several state-of-the-art methods for HSI classification. In addition, the proposed TACLNN can effectively reduce the number of parameters and storage requirements achieving higher classification accuracies as compared to other competitive baselines.

Advancements in smart grid technology and the extensive applications of electric power big data have made in-depth exploration of the behavioral characteristics of power consumers highly necessary for further development of the electricity market. This paper proposes an effective and interpretable load curve classification method that based on sequential trajectory feature learning and a random forest algorithm. Firstly, the unsupervised K-medoid clustering algorithm is used to obtain and filter precise category labels. Next, the fused lasso generalized eigenvector (FLAG) technique is used to search for interpretable sub-sequences from the labeled data in order to properly account for the sequential trajectory feature of load curves and increase the speed of the computation process. Following that, shapelet transformation is used to extract the sequential trajectory features from original data. Finally, in order to inherit the interpretability of shapelet, the random forest is trained on the sequential trajectory features. The simulated examples based on the real load curves of the specific city in China were investigated in order to assess the performance of the proposed load curve categorization approach. The results of the simulation demonstrate that the proposed approach has considerable advantages in terms of effectiveness, accuracy, and interpretability of load classification.

Recurrent neural networks (RNNs) have been widely used for hyperspectral image (HSI) classification via sequence modeling. However, most of the RNN methods focus on modeling long-range dependencies along the spectral direction, without fully exploring multi-directional dependencies in the joint spectral-spatial domain. To tackle this issue, we propose MSLAN, a two-branch multi-directional spectral-spatial long short-term memory (LSTM) attention network, for HSI classification. In particular, we employ LSTMs to extract six-directional spatial-spectral features which simultaneously capture the spectral-spatial dependencies along different directions. We then design an attention-based feature fuse module to integrate these directional features, followed by a fully connected layer with cross-entropy loss for classification. Additionally, we incorporate an auxiliary branch into our model to enhance the generalization capability. In this branch, random spatial shuffle and a cosine loss are explored for feature consistency learning by taking into account the varying spatial distributions. The resulting two branch networks, sharing the same network structure and weights, are incorporated into a unified deep learning architecture for training. Experiments show the superiority of MSLAN to the state-of-the-art methods for HSI classification with limited training samples.

Hyperspectral (HS) image is a three dimensional data image where the 3rd dimension carries the wealth of spectrum information. HS image compression is one of the areas that has attracted increasing attention for big data processing and analysis. HS data has its own distinguishing feature which differs with video because without motion, also different with a still image because of redundancy along the wavelength axis. The prediction based method is playing an important role in the compression and research area. Reflectance distribution of HS based on our analysis indicates that there is some nonlinear relationship in intra-band. The Multilayer Propagation Neural Networks (MLPNN) with backpropagation training are particularly well suited for addressing the approximation function. In this paper, an MLPNN based predictive image compression method is presented. We propose a hybrid Adaptive Prediction Mechanism (APM) with MLPNN model (APM-MLPNN). MLPNN is trained to predict the succeeding bands by using current band information. The purpose is to explore whether MLPNN can provide better image compression results in HS images. Besides, it uses less computation cost than a deep learning model so we can easily validate the model. We encoded the weights vector and the bias vector of MLPNN as well as the residuals. That is the only few bytes it then sends to the decoder side. The decoder will reconstruct a band by using the same structure of the network. We call it an MLPNN decoder. The MLPNN decoder does not need to be trained as the weights and biases have already been transmitted. We can easily reconstruct the succeeding bands by the MLPNN decoder. APM constrained the correction offset between the succeeding band and the current spectral band in order to prevent HS image being affected by large predictive biases. The performance of the proposed algorithm is verified by several HS images from Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) reflectance dataset. MLPNN simulation results can improve prediction accuracy; reduce residual of intra-band with high compression ratio and relatively lower bitrates.

In this paper, a hyperspectral (HS) image super-resolution (SR) approach based on attention convolutional bi-long short-term memory (ConvBiLSTM) network is proposed, aiming to explore the collaborative spatial and spectral attention characteristics, thereby enhancing the spatial resolution of HS image. ConvBiLSTM network combines the spatial feature mining and sequential predicting abilities of convolutional neural network and recurrent neural network, respectively. We adapt the ConvBiLSTM network for our super-resolution purpose by regarding each band as a single frame of sequential data, and propose a band-sharing spatial-channel attention-combined ConvBiLSTM SR method to intensify the saliency features. Moreover, a spatial-regularized loss function is presented to further promote the fidelity of the super-resolved HS image. Experiments on four HS data sets show that the proposed approach outperforms some state-of-the-art HS image SR techniques, from the aspect of spectral fidelity.

On the use of small training sets for neural network-based characterization of mixed pixels in remotely sensed hyperspectral images

Hyperspectral imaging (HSI) collects hundreds of images over large spatial observation areas on the Earth’s surface, recording scenes at different wavelength channels and providing a vast amount of information. Recurrent neural networks (RNNs) have been widely used for the classification of HSI datasets, understood as a single sequence of pixel vectors with high dimensionality. However, the RNN model scales poorly when dealing with HSI scenes with large dimensionality. In order to mitigate this problem, this paper presents a new RNN classifier based on simple recurrent units that performs HSI classification in a highly scalable and efficient way. Our experimental results (conducted on four real HSI datasets) reveal very good performance, not only in terms of classification accuracy (in line with existing methods), but also in terms of computational performance when dealing with large datasets.

In this paper, a novel method based on wavelet transformation of functional data for accurate classification of hyperspectral images is proposed. The motivation of the proposed method is that in the hyperspectral images, the spectral curve of each pixel can be viewed as a function, mathematically. And it can make the best of the abundant spectral information of the original hyperspectral images. Based on this, we have proposed an effective hyperspectral image classification method which mainly used functional principal component analysis (FPCA) before. However, FPCA method focuses on the global features analysis of hyperspectral image data and may not well extract local structural features. In this paper, in order to better reflect the local structural features of hyperspectral images, we make use of the time-frequency localization characteristics of wavelet transformation to decompose hyperspectral images. Then, the sparse wavelet decomposition coefficients are classified by using extreme learning machine (ELM). Experimental results on two popular hyperspectral images show that our method outperforms some state-of-the-art hyperspectral image classification methods.

Exploitation of the spectral capabilities of modern hyperspectral image demands efficient preprocessing and analyses methods. Analysts' choice of classifier and dimensionality reduction (DR) method and the harmony between them determine the accuracy of image classification. Multiple classifier system (MCS) has the potential to combine the relative advantages of several classifiers into a single image classification exercise for the hyperspectral image classification. In this paper, we propose an algorithmic extension of the MCS, named as dynamic classifier system (DCS), which exploits the context-based image and information class characteristics represented by multiple DR methods for hyperspectral image classification for land cover mapping. The proposed DCS algorithm pairs up optimal combinations of classifiers and DR methods specific to the hyperspectral image and performs image classifications based only on the identified combinations. Further, the impact of various trainable and nontrainable combination functions on the performance of the proposed DCS has been assessed. Image classifications were carried out on five multi-site airborne hyperspectral images using the proposed DCS and were compared with the MCS and SVM based supervised image classifications with and without DR. The results indicate the potential of the proposed DCS algorithm to increase the classification accuracy considerably over that of MCS or SVM supervised image classifications.