A deep learning framework for hyperspectral image classification using spatial pyramid pooling

For hyperspectral image (HSI) classification, two branch networks generally use the convolution neural networks (CNNs) to extract the spatial features and the long short-term memory (LSTM) to learn the spectral features. However, CNN with a local kernel neglects the global properties of the whole HSI. LSTM doesn't consider the macroscopic and detailed information of spectra. In this paper, we propose a dual-view spectral and global spatial feature fusion network (DSGSF) to extract the spatial-spectral features for HSI classification, including a spatial subnetwork and a spectral subnetwork. In the spatial subnetwork, we propose a global spatial feature representation model based on the encoder-decoder structure with channel attention and spatial attention to learn the global spatial features. In the spectral subnetwork, we design a dual-view spectral feature aggregation model with view attention to learn the diversity of spectral features. By fusing the two subnetworks, we construct DSGSF to extract the spatial-spectral features of HSI with strong discriminating performance. Experimental results on three public datasets illustrate that the proposed method can achieve competitive results compared with the state-of-the-art methods. Code: https://github.com/RZWang-WH/DSGSF.

Development in the field of computer-aided learning and testing have stimulated the progress of novel and efficient knowledge-based expert systems that have shown hopeful outcomes in a broad variety of practical applications. In particular, deep learning techniques have been extensively carried out to identify remote sensed data obtained by the instruments of Earth observation. Hyperspectral imaging (HSI) is an evolving area in the study of remotely sensed data due to the huge volume of information found in these images, which enables better classification and processing of the Earth’s surface by integrating ample of spatial and spectral features. Nevertheless, because of the high-dimensional data and restricted training samples available, HSI presents some crucial challenges for classification of supervised methods. In particular, it addresses the problems of spectral and spatial resolution, volume of data, and model conversion from multimedia images to hyperspectral data. Various deep learning-based architectures are currently being established to solve these limitations, showing significant results in the analysis of hyperspectral data. In this paper, we deal primarily with the hyperspectral datasets, the dimensionality curse problem, and methods for classifying those datasets using some deep neural networks (DNN), especially convolutional neural networks (CNN). We provide a comparative analysis of various dimensionality reduction (DR) and classification techniques used for finding accuracies based on the datasets used. We also explore certain hyperspectral imaging applications along with some of the research axes.

The accurate identification of melon seed varieties is essential for improving seed purity and the overall quality of melon production. In this study, hyperspectral imaging was used to identify six varieties of melon seeds. Both hyperspectral images and RGB images were generated during hyperspectral image acquisition. The spectral features of seeds were extracted from the hyperspectral images. The image features of the corresponding seeds were manually extracted from the RGB images. Five different datasets were formed using the spectral features and RGB images of the seeds, including seed spectral features, manually extracted seed image features, seed images, the fusion of seed spectral features with manually extracted features, and the fusion of seed spectral features with seed images. Logistic Regression (LR), Support Vector Classification (SVC), and Extreme Gradient Boosting (XGBoost) were used to establish classification models using spectral features and the manually extracted image features. Convolutional Neural Network (CNN) models were established using the five datasets. The results indicated that the CNN models achieved good performance in all five datasets, with classification accuracies exceeding 90% for the training, validation, and test sets. Also, CNN using the fused datasets obtained optimal performance, achieving classification accuracies exceeding 97% for the training, validation, and test sets. The results indicated that both spectral features and image features can be used to identify the six varieties of melon seeds, and their fusion of spectral features and image features can improve classification performance. These findings provide an alternative approach for melon seed variety identification, which can also be extended to other seed types.

Hyperspectral data consists of continuous narrow spectral bands. Due to this, it has less spatial and high spectral information. Convolutional neural networks (CNNs) emerge as a highly contextual information model for remote sensing applications. Unfortunately, CNNs have constraints in their underlying network architecture in regards to the global correlation of spatial and spectral features, making them less reliable for mining and representing the sequential properties of spectral signatures. In this article, limpid size attention network (LSANet) is proposed, which contains 3D and 2D convolution blocks for enhancement of spatial-spectral features of the hyperspectral image (HSI). In addition, limpid attention block (LAB) is designed to provide a global correlation of the spectral and spatial features through LS attention. Furthermore, the computational costs of LS-attention are less compared to the multi-head self-attention (MHSA) of the classical vision transformer (ViT). In the ViT encoder a conditional position encoding (CPE) module is utilized that dynamically generates tokens from the feature maps to capture a richer contextual representation. The LSANet obtained overall accuracy (OA) of 98.78%, 98.67%, 97.52% and 89.45%, respectively, on the Indian Pines (IP), Pavia University (PU), Salina Valley (SV) and Botswana datasets. Our model’s quantitative and qualitative results are considerably better than the classical CNN and transformer-based methods.

This paper studies the classification problem of hyperspectral image (HSI). Inspired by the great success of deep neural networks in Artificial Intelligence (AI), researchers have proposed different deep learning based algorithms to improve the performance of hyperspectral classification. However, deep learning based algorithms always require a large-scale annotated dataset to provide sufficient training. To address this problem, we propose a semi-supervised deep learning framework based on the residual networks (ResNets), which use very limited labeled data supplemented by abundant unlabeled data. The core of our framework is a novel dual-strategy sample selection co-training algorithm, which can successfully guide ResNets to learn from the unlabeled data by making full use of the complementary cues of the spectral and spatial features in HSI classification. Experiments on the benchmark HSI dataset and real HSI dataset demonstrate that, with a small number of training data, our approach achieves competitive performance with maximum improvement of 41% (compare with traditional convolutional neural network (CNN) with 5 initial training samples per class on Indian Pines dataset) for HSI classification as compared with the results from those state-of-the-art supervised and semi-supervised methods.

Convolutional neural networks (CNNs) have been widely used for hyperspectral image classification. As a common process, small cubes are first cropped from the hyperspectral image and then fed into CNNs to extract spectral and spatial features. It is well known that different spectral bands and spatial positions in the cubes have different discriminative abilities. If fully explored, this prior information will help improve the learning capacity of CNNs. Along this direction, we propose an attention-aided CNN model for spectral-spatial classification of hyperspectral images. Specifically, a spectral attention subnetwork and a spatial attention subnetwork are proposed for spectral and spatial classifications, respectively. Both of them are based on the traditional CNN model and incorporate attention modules to aid networks that focus on more discriminative channels or positions. In the final classification phase, the spectral classification result and the spatial classification result are combined together via an adaptively weighted summation method. To evaluate the effectiveness of the proposed model, we conduct experiments on three standard hyperspectral data sets. The experimental results show that the proposed model can achieve superior performance compared with several state-of-the-art CNN-related models.

The advancement of unique and efficient knowledge-based expert systems has been significantly influenced by the progress in the field of computer-aided learning and testing, which have demonstrated promising results in a wide range of practical applications. Deep learning techniques, in particular, have been used extensively to detect, predict, and classify patterns in remotely sensed urban land cover areas. Machine learning is already lending a hand in monitoring and tracking changes undergoing on the Earth’s surface. However, deep learning enables a higher level of abstraction and provides better extraction of spectral and spatial features in satellite-based imaging analysis. This chapter primarily deals with the application of Internet of Things and deep learning in hyperspectral imaging analysis. For this purpose, some well-known publicly available datasets are used for the classification of different classes present in these datasets. Four main deep neural networks, especially convolution neural network, recurrent neural network (long short-term memory and gated recurrent unit), auto-encoders, and generative adversarial network have been used for the experimentation purpose. A comparative analysis of these classification techniques used for finding the accuracy is made. In the end, certain challenges in deep learning are analyzed, along with some of the emerging future research axes.

The Hyperspectral Images (HSI) are now being widely popular due to the evolution of satellite imagery and camera technology. Remote sensing has also gained popularity and it is also closely related to HSI. HSI possesses a wide variety of spatial and spectral features. However, HSI also has a consider-able amount of useless or redundant data. This redundant data causes a lot of trouble during classifications as it possesses a huge range in contrast to RGB. Traditional classification techniques do not apply efficiently to HSI. Even if somehow the traditional techniques are applied to it, the results produced are inefficient and undesirable. The Convolutional Neural Network (CNN), which are widely famous for the classification of images, have their fair share of trouble when dealing with HSI. 2D CNNs is not very efficient and 3D CNNs increases the computational complexity. To overcome these issues a new hybrid CNN approach is used which uses sigmoid activation function at the output layer, using a 2D CNN with 3D CNN to generate the desired output. Here, we are using HSI classification using hybrid CNN i.e., 2D and 3D. The dataset used is the Indian pines dataset sigmoid classifier for classification. And we gain the Overall accuracy 99.34 %, average accuracy 99.27%, kappa 99.25%.

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral-spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

The classification of hyperspectral images (HSIs) is one of the most popular topics in the remote sensing community. Numerous feature extraction methods have been proposed to improve the classification accuracy of HSIs. Recently, deep features extracted by convolution neural network (CNN) have been introduced into the classification process of HSIs. Due to the nonlinear and invariant advantages of the features, CNN methods provide a powerful tool for representing geographic objects and classifying HSIs. However, traditional deep features only extracted at pixel-level and often neglect multiscale characteristics of geographic objects. In this study, a new deep feature extraction method is proposed, which takes advantage of multi-scale object analysis and the CNN model. Firstly, multiscale image objects are obtained by the multiscale segmentation algorithm and multiscale low-level features of objects are extracted. Secondly, the CNN is devoted to obtain deep features from low-level object features at each scale, respectively. Thirdly, the obtained deep features at all scales are stacked and fed to one fully connected layer to extract the multiscale deep learning features for classification. Finally, the logistic regression classifier is applied to hyperspectral image (HSI) classification based on object-oriented multiscale deep features. The proposed method was carried out on three widely used hyperspectral data sets: University of Pavia, Salinas, and Washington DC. The results reveal that the proposed method provides better results than other state-of-the-art methods.

Hyperspectral Image (HSI) classification is one of the fundamental tasks in the field of remote sensing data analysis. CNN (Convolutional Neural Network) has been proven to be an effective deep learning model, which can extract high-level features directly from the raw data and thereby utilize rich information contained in HSI data. However, labor cost to label enough HIS data for training model is usually expensive, so that it is a strong demand of utilizing limited training data to get a satisfied classification accuracy. In this paper, we put forward a deep cube CNN model – DCNR, which is composed of a cube neighbor HSI pixels strategy, a deep CNN and a random forest classifier. In DCNR model, cubic samples, containing spectral-spatial information, are generated by putting each target pixel and its neighbors together. Then features with high representative ability, extracted by applying a specially designed cube CNN model on each cubic sample, are fed into the random forest classifier for the classification of the target pixel. Results show that DCNR model can achieve classification accuracy of 96.78%, 96.08% and 94.85% on KSC, IP and SA datasets respectively with 20% samples as training set, and 85.03%, 83.45 and 62.17% on KSC, IP and SA datasets respectively with only 1% samples as training set, significantly outperforming random forest and cube CNN models.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

Hyperspectral imaging is a rising imaging modality in the field of medical applications, and the combination of both spectral and spatial information provides wealth information for cell classification. In this paper, deep convolutional neural network (CNN) is employed to achieve blood cell discrimination in medical hyperspectral images (MHSI). As a deep learning architecture, CNNs are expected to get more discriminative and semantic features, which effect classification accuracy to a certain extent. Experimental results based on two real medical hyperspectral image data sets demonstrate that cell classification using CNNs is effective. In addition, compared to traditional support vector machine (SVM), the proposed method, which jointly exploits spatial and spectral features, can achieve better classification performance, showcasing the CNN-based methods' tremendous potential for accurate medical hyperspectral data classification.

Hyperspectral image (HSI) classification remains a challenging problem due to unique characteristics of HSI data (such as numerous bands and strong correlations in the spectral and spatial domains) and small sample size. To address such concerns, we propose a novel spectral–spatial feature extraction method for HSI classification by employing graph embedding and deep learning (DL) models. Since the conventional graph cannot capture the complex manifold relationship of HSI data, and there exist the observations of within-class variation as well as the similarity between different classes in the spectral domain, we construct the supervised within-class/between-class hypergraph (SWBH) to extract the spectral features of HSI. Since it is difficult for DL models to learn representative features for HSI data when the labeled training samples are limited, we propose the random zero settings to newly generate a large amount of labeled HSI samples for the training of convolutional neural network (CNN). The designed sample expanded CNN (SECNN) is used to extract the HSI spatial features. Thus, the spectral–spatial features of HSI can be learned by integrating the features extracted from SWBH and SECNN, respectively. Experiments on three real HSI datasets demonstrate higher classification accuracy of the proposed SWBH–SECNN method.

Cervical cancer is the fourth most prevalent disease among women. Prompt diagnosis and its management can significantly improve patients’ survival rates. Therefore, routine screening for cervical cancer is of paramount importance. Herein, we explore the potential of a deep learning model to automatically distinguish abnormal cells from normal cells. The ThinPrep cytologic test dataset was collected from the fourth central hospital of Baoding city, China. Based on the dataset, four classification models were developed. The first model was a 10-layer convolutional neural network (CNN). The second model was an advancement of the first model equipped with a spatial pyramid pooling (SPP) layer (CNN + SPP) to treat cell images based on their sizes. Based on the first model, the third model replaced the CNN layers with the inception module (CNN + Inception). However, the fourth model incorporated both the SPP layer and the inception module into the first model (CNN + inception + SPP). The performances of the four models are estimated and compared by using the same testing data and evaluation index. The testing results demonstrated that the fourth model yields the best performance. Moreover, the area under the curve (AUC) for module four was 0.997.