Spectral–Spatial Land Cover Mapping Using a Deep 3D Convolutional Neural Network on Hyperspectral Data

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

Hyperspectral Image (HSI) classification is one of the fundamental tasks in the field of remote sensing data analysis. CNN (Convolutional Neural Network) has been proven to be an effective deep learning model, which can extract high-level features directly from the raw data and thereby utilize rich information contained in HSI data. However, labor cost to label enough HIS data for training model is usually expensive, so that it is a strong demand of utilizing limited training data to get a satisfied classification accuracy. In this paper, we put forward a deep cube CNN model – DCNR, which is composed of a cube neighbor HSI pixels strategy, a deep CNN and a random forest classifier. In DCNR model, cubic samples, containing spectral-spatial information, are generated by putting each target pixel and its neighbors together. Then features with high representative ability, extracted by applying a specially designed cube CNN model on each cubic sample, are fed into the random forest classifier for the classification of the target pixel. Results show that DCNR model can achieve classification accuracy of 96.78%, 96.08% and 94.85% on KSC, IP and SA datasets respectively with 20% samples as training set, and 85.03%, 83.45 and 62.17% on KSC, IP and SA datasets respectively with only 1% samples as training set, significantly outperforming random forest and cube CNN models.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral-spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral-spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton's capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral-spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral-spatial capsules, which are able to learn HSI spectral-spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral-spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Background: Hyperspectral Image (HSI) classification involves analyzing images captured across numerous spectral bands to identify and categorize materials or objects. By exploiting spectral information, HSI enables precise detection of materials based on their unique spectral signatures. This extensive spectral information allows for accurate identification of materials and land cover types, making HSI classification essential for various applications. This research developed an optimization-based Deep Learning (DL) framework for effective classification of HSI. The methodology begins with a preprocessing step using Minimum Noise Fraction (MNF), a technique that leverages the noise covariance structure of the dataset to reduce dimensionality while preserving the most informative spectral features. To enhance classification performance, a novel model is proposed, known as Archimedes Optimization Algorithm-3D Convolutional Neural Network (AOA based 3D CNN). Archimedes Optimization Algorithm (AOA) is applied to fine-tune learning rate of 3D Convolutional Neural Network (3D CNN), thereby optimizing training convergence and enhancing accuracy. Result: The proposed AOA based 3D CNN approach exhibits outstanding performance, achieving 95.6% accuracy, 95.9% TPR, 94.1% TNR, 94.9% PPV, a Kappa coefficient of 95.4%, and an F1-score of 95.5%.Keywords: These results highlight the model's effectiveness, adaptability and robustness for HSI classification.

Convolutional neural networks (CNNs), a kind of feedforward neural network with a deep structure, are one of the representative methods in hyperspectral image (HSI) classification. However, redundant information and interclass interference are common and challenging problems in HSI classification. In addition, if the spectral and spatial information is not properly extracted and analyzed, it will affect the classification performance of the network to a great extent. Aiming at these issues, this article proposes an HSI classification method based on an adaptive hash attention mechanism and a lower triangular network (AHA-LT). First, the attention mechanism is introduced in the preprocessing stage, which is composed of the spectral attention module and the adaptive hash spatial attention module in series. Then, the data processed by the attention mechanism are introduced into the lower triangular network (LTNet) to obtain the fused high-dimensional semantic features. Finally, we compress the features and obtain the output classification results through several fully connected layers. Among them, LTNet is composed of 2-D–3-D CNN and multiscale features. The network integrates the characteristics of multibranch, feature fusion, feature compression, and skip connections. Extensive experiments on four widely used HSI data sets show that the proposed method can obtain a great improvement in performance compared with the existing methods.

Hyperspectral image (HSI) classification remains a challenging problem due to unique characteristics of HSI data (such as numerous bands and strong correlations in the spectral and spatial domains) and small sample size. To address such concerns, we propose a novel spectral–spatial feature extraction method for HSI classification by employing graph embedding and deep learning (DL) models. Since the conventional graph cannot capture the complex manifold relationship of HSI data, and there exist the observations of within-class variation as well as the similarity between different classes in the spectral domain, we construct the supervised within-class/between-class hypergraph (SWBH) to extract the spectral features of HSI. Since it is difficult for DL models to learn representative features for HSI data when the labeled training samples are limited, we propose the random zero settings to newly generate a large amount of labeled HSI samples for the training of convolutional neural network (CNN). The designed sample expanded CNN (SECNN) is used to extract the HSI spatial features. Thus, the spectral–spatial features of HSI can be learned by integrating the features extracted from SWBH and SECNN, respectively. Experiments on three real HSI datasets demonstrate higher classification accuracy of the proposed SWBH–SECNN method.

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

In recent years, deep-learning-based hyperspectral image (HSI) classification networks have become one of the most dominant implementations in HSI classification tasks. Among these networks, convolutional neural networks (CNNs) and attention-based networks have prevailed over other HSI classification networks. While convolutional neural networks with perceptual fields can effectively extract local features in the spatial dimension of HSI, they are poor at capturing the global and sequential features of spectral–spatial information; networks based on attention mechanisms, for example, Transformer, usually have better ability to capture global features, but are relatively weak in discriminating local features. This paper proposes a fusion network of convolution and Transformer for HSI classification, known as FusionNet, in which convolution and Transformer are fused in both serial and parallel mechanisms to achieve the full utilization of HSI features. Experimental results demonstrate that the proposed network has superior classification results compared to previous similar networks, and performs relatively well even on a small amount of training data.

Recently, deep learning-based methods have made great progress in hyperspectral image (HSI) classification (HSIC). Different from ordinary images, the intrinsic complexity of HSIs data still limits the performance of many common convolutional neural network (CNN) models. Thus, the network architecture becomes more and more complex to extract discriminative spectral-spatial features. For instance, 3-D CNN usually has a large number of trainable parameters, thus increasing the computational complexity of the HSIC. In this letter, we designed a cooperative spectral-spatial attention dense network (CS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ADN) that takes raw 3-D HSI data as input data. Specifically, the attention module consists of spectral and spatial axes, by which the salient spectral-spatial features will be emphasized. Furthermore, we combined these attention modules with the dense connection, which is termed as the lightweight dense block; it has a lower computation cost and achieves better classification performance. At the same time, we introduced the center loss, by jointly using the supervision of the center loss and the softmax loss, where the discriminative features could be clearly observed, particularly for small data sets. Experimental results on the biased and unbiased HSI data show that our method outperforms several state-of-the-art methods in HSIC with small training samples.

Hyperspectral image classification via nonlocal joint kernel sparse representation based on local covariance

Hyperspectral images (HSI) offer detailed spectral reflectance information about sensed objects through provision of information on hundreds of narrow spectral bands. HSI have a leading role in a broad range of applications, such as in forestry, agriculture, geology, and environmental sciences. The monitoring and management of agricultural lands is of great importance for meeting the nutritional and other needs of a rapidly and continuously increasing world population. In relation to this, classification of HSI is an effective way for creating land use and land cover maps quickly and accurately. In recent years, classification of HSI using convolutional neural networks (CNN), which is a sub-field of deep learning, has become a very popular research topic and several CNN architectures have been developed by researchers. The aim of this study was to investigate the classification performance of CNN model on agricultural HSI scenes. For this purpose, a 3D-2D CNN framework and a well-known support vector machine (SVM) model were compared using the Indian Pines and Salinas Scene datasets that contain crop and mixed vegetation classes. As a result of this study, it was confirmed that use of 3D-2D CNN offers superior performance for classifying agricultural HSI datasets.

Most hyperspectral image (HSI) classification methods assume that all classes in the test set are present during training. However, in real-world applications, acquiring labeled training samples is challenging. As a result, it is difficult for the training dataset to cover all possible land cover types, leading to the generalized zero-shot learning (GZSL) problem. Recently, vision-language models (VLMs) have provided rich semantic priors for land cover classes, offering promising potential for GZSL. However, two fundamental gaps hinder their application to HSI classification: the task paradigm gap, arising from the difference between image-level VLMs and the pixel-level HSI classification task; and the knowledge gap, due to the inconsistency between VLM features and HSI spectral–spatial representations. To bridge both gaps, a novel framework leveraging VLM semantic priors for GZSL in HSI classification is proposed, primarily using pseudo-labeling technique to provide knowledge for unseen classes. Specifically, a pseudo-label generation and enhancement module enables a paradigm transition from image-level understanding to pixel-level classification by incorporating HSI’s spatial information. A pseudo-label correction module then refines noisy labels using spectral cues to address the knowledge gap. Finally, a global learning strategy integrates pseudo-label distillation, supervised learning, and feature regularization to classify seen classes while enabling generalization to unseen ones. Experiments on benchmark HSI datasets demonstrate the proposed method’s superiority in generalized zero-shot classification. This work highlights the potential of VLMs in advancing HSI classification in practical applications.

Abstract. Hyperspectral images (HSIs) contain hundreds of spectral bands, providing high-resolution spectral information pertaining to the Earth’s surface. Additionally, abundant spatial contextual information can also be obtained simultaneously from a HSI. To characterize the properties of ground objects, classification is the most widely-used technology in the field of remote sensing, where each pixel in a HSI is assigned to a pre-defined class. Over the past decade, deep learning has attracted increasing attention in the machine-learning and computer-vision domains, due to its favourable performances for various types of tasks, and it has been successfully introduced to the remote-sensing community. Instead of utilizing the shallow features within in a given image, which is the approach that is generally adopted in other conventional classification methods, deep-learning algorithms can extract hierarchical features from raw HSI data. Within the deep-learning framework, recurrent neural networks (RNNs), which are able to encode sequential features, have exhibited promising capabilities and have achieved encouraging performances, especially for the natural-language processing and speech-recognition communities. As multi-temporal remote-sensing images can be readily obtained from increasing numbers of satellite and unmanned aircraft systems, and since analysis of such multi-temporal data comprises a critical issue within numerous research subfields, including land-cover and land-change analyses, and land-resource management, RNNs have been applied in recent studies in order to extract temporal sequential features from multi-temporal remote-sensing images for the purpose of image classification. Apart from using multi-temporal image datasets, RNNs can also be utilized on a single image, where the spectral feature/band of each individual pixel can be taken as a sequential feature for the input layer of RNNs. However, the application of such sequential feature extraction that relies on a single image still needs to be further investigated since applying RNNs to spectral bands will directly introduce more parameters that need to be optimized, consequently increasing the total training time.In this study, we propose a novel RNN-based HSI classification framework. In this framework, unlabelled pixels obtained from a single image are considered when constructing sequential features. Two spatial similarity measurements, referred to as pixel-matching and block-matching, respectively, are employed to extract pixels that are “similar” to the target pixel. Then, the sequential feature of the target pixel is constructed by exploiting several of the most “similar” pixels and ordering them based on their similarities to the target pixel. The aforementioned two schemes are advantageous, as unlabelled pixels within the given HSI are taken into consideration for similarity measurement and sequential feature construction for the RNN model. Moreover, the block-matching scheme also takes advantage of spatial contextual information, which has been widely utilized in spatial-spectral-based HSI classification methods. To evaluate the proposed methods, two benchmark HSIs are used, including a HSI collected over Pavia University, Italy by the airborne Reflective Optics System Imaging Spectrometer (ROSIS) sensor, and an image acquired over the Salinas Valley, California, USA via the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Spatio-temporally coincident ground-reference data accompanies each of these respective HSIs. In addition, the proposed methods are compared with three state-of-the-art algorithms, including support vector machine (SVM), the 1-dimensional convolutional neural network (1DCNN), and the 1-dimensional RNN (1DRNN).Experimental results indicate that our proposed methods achieve markedly better classification performance compared with the baseline algorithms on both datasets. For example, for the Pavia University image, the block-matching based RNN achieves the highest overall classification accuracy, with 94.32% accuracy, which is 9.87% higher than the next most accurate algorithm of the aforementioned three baseline methods, which in this case is the 1DCNN, with 84.45% overall accuracy. More specifically, the block-matching method performs better than the pixel-matching method in terms of both quantitative and qualitative assessments. Based on visual assessment/interpretation of the classification maps, it is apparent that “salt-and-pepper” noise is markedly alleviated; with block-matching, smoother classified images are generated compared with pixel-matching-based methods and the three baseline algorithms. Such results demonstrate the effectiveness of utilizing spatial contextual information in the similarity measurement.

Feature extraction (FE) methods based on low-rank representation (LRR) have become important topics in hyperspectral images (HSIs) data analysis. In this paper, a supervised FE method for HSIs data based on LRR with the ability to preserve the local pairwise constraints information (LRLPC) is proposed. LRLPC does not change the data dimensionality and only employs a technique to enrich the original feature space (OFS) and to obtain enriched feature space, which results in features richer than OFS. To overcome the problem of LRR in lacking the local structure information (LSI) of data, a local discriminative regularization term is imposed on the fitness function of LRR to keep the LSI of data. For nonlinear structure of data, LRLPC is extended to kernel LRLPC (KLRLPC) using kernel trick. Utilization of existing information in the pairwise constraints is useful for limited labeled samples situations as a common problem in HSI data analysis. The obtained experimental results using two well-known HSI data sets confirm the effectiveness of LRLPC and KLRLPC for dimension reduction and classification of HSIs.