HZSCM: Hyperspectral Image Zero-Shot Classification via Vision-Language Models

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

Hyperspectral image (HSI) classification has attracted much attention and extensive research efforts over the past decade. Due to few labeled samples versus high dimensional features, it is a challenging problem in practice. Recently, combining the pixel spectral information and the spatial (neighborhood) information has been verified to be effective for HSI classification. In this paper, we introduce a novel method for HSI classification using set-to-set distance (SSD). Based on the assumption that neighbor pixels tend to belong to the same class with high probability, we model a test pixel and its neighbor pixels as a testing set (or a neighbor set) inspired by bilateral filtering. Meanwhile, the training pixels belong to the same class are modeled as a training set. Therefore, the classification is based on comparisons of sets distances. Experiments on a real HSI dataset show that our proposed method outperforms a number of existing state-of-the-art approaches.

Hyperspectral image (HSI) classification has become one of the most significant tasks in the field of hyperspectral analysis. However, classifying each pixel in HSI accurately is challenging due to the curse of dimensionality and limited training samples. In this paper, we present an HSI classification architecture called camera spectral response network (CSR-Net), which can learn the optimal camera spectral response (CSR) function for HSI classification problems and effectively reduce the spectral dimensions of HSI. Specifically, we design a convolutional layer to simulate the capturing process of cameras, which learns the optimal CSR function for HSI classification. Then, spectral and spatial features are further extracted by spectral and spatial attention modules. On one hand, the learned CSR can be implemented physically and directly used to capture scenes, which makes the image acquisition process more convenient. On the other hand, compared with ordinary HSIs, we only need images with far fewer bands, without sacrificing the classification precision and avoiding the curse of dimensionality. The experimental results of four popular public hyperspectral datasets show that our method, with only a few image bands, outperforms state-of-the-art HSI classification methods which utilize the full spectral bands of images.

Hyperspectral image classification is a challenging problem for machine learning methods due to the small number of labelled samples and high spectral variability. In this paper, to solve this problem, a novel superpixel and multi-classifier fusion (SMCF)-based classification method for hyperspectral images is proposed. This method takes full advantage of the spectral information of superpixels and the spatial information of hyperspectral images and includes the following three steps. First, superpixels are used to increase the number of training samples and their spectral diversity. Second, label propagation (LP) is used to classify the hyperspectral images. Although LP is an efficient semi-supervised classification method, the corresponding performance is poor for certain land cover types with dispersed spatial distributions. Thus, a support vector machine (SVM) classifier is introduced to classify the hyperspectral images. Finally, the results of the SVM and LP classifiers are combined using our new class-specific weighted fusion algorithm. In the experiments, we selected three widely used and real hyperspectral data sets for evaluation. The final classification performance was evaluated based on two common metrics: the overall accuracy (OA) and the Kappa coefficient. The experimental results show that the proposed SMCF method is superior to six well-known classification methods, even when only 1% or less of the labelled samples are used.

Recently, hyperspectral image (HSI) classification methods based on deep learning have demonstrated excellent performance. However, these deep learning methods still face two major challenges. One is that they require a large number of labeled samples, and the other is that training parameters takes a lot of time. In this paper, we propose a dual-branch deep stochastic adaptive Fourier decomposition (SAFD) network (DSAFDNet) to alleviate the aforementioned two issues in HSI classification applications. SAFD is a newly developed signal processing tool with solid mathematical foundation. It can be used to find common filters (i.e. convolution kernels) of a set of random signals or multi-signals. Since the convolution kernels obtained by SAFD decomposition are complex numbers, few deep learning methods directly deal with such complex convolution kernels. To this end, we propose a dual-branch network to extract deep features from hyperspectral images using both real and imaginary parts of convolutional kernels. After deep feature extraction using DSAFDNet, we further investigate the classification performance of different classifiers on the extracted features. Experimental results show that the proposed method outperforms some HSI classification methods with similar principles. Moreover, compared with other state-of-the-art deep learning methods, the proposed method can achieve better classification performance.

Spectral features cannot effectively reflect the differences among the ground objects and distinguish their boundaries in hyperspectral image (HSI) classification. Multi-scale feature extraction can solve this problem and improve the accuracy of HSI classification. The Gaussian pyramid can effectively decompose HSI into multi-scale structures, and efficiently extract features of different scales by stepwise filtering and downsampling. Therefore, this paper proposed a Gaussian pyramid based multi-scale feature extraction (MSFE) classification method for HSI. First, the HSI is decomposed into several Gaussian pyramids to extract multi-scale features. Second, we construct probability maps in each layer of the Gaussian pyramid and employ edge-preserving filtering (EPF) algorithms to further optimize the details. Finally, the final classification map is acquired by a majority voting method. Compared with other spectral-spatial classification methods, the proposed method can not only extract the characteristics of different scales, but also can better preserve detailed structures and the edge regions of the image. Experiments performed on three real hyperspectral datasets show that the proposed method can achieve competitive classification accuracy.

Convolutional neural networks (CNNs), a kind of feedforward neural network with a deep structure, are one of the representative methods in hyperspectral image (HSI) classification. However, redundant information and interclass interference are common and challenging problems in HSI classification. In addition, if the spectral and spatial information is not properly extracted and analyzed, it will affect the classification performance of the network to a great extent. Aiming at these issues, this article proposes an HSI classification method based on an adaptive hash attention mechanism and a lower triangular network (AHA-LT). First, the attention mechanism is introduced in the preprocessing stage, which is composed of the spectral attention module and the adaptive hash spatial attention module in series. Then, the data processed by the attention mechanism are introduced into the lower triangular network (LTNet) to obtain the fused high-dimensional semantic features. Finally, we compress the features and obtain the output classification results through several fully connected layers. Among them, LTNet is composed of 2-D–3-D CNN and multiscale features. The network integrates the characteristics of multibranch, feature fusion, feature compression, and skip connections. Extensive experiments on four widely used HSI data sets show that the proposed method can obtain a great improvement in performance compared with the existing methods.

Hyperspectral image classification using FPCA-based kernel extreme learning machine

In this paper, we propose a hyperspectral image (HSI) classification method using spectral-spatial long short term memory (LSTM) networks. Specifically, for each pixel, we feed its spectral values in different channels into Spectral LSTM one by one to learn the spectral feature. Meanwhile, we firstly use principle component analysis (PCA) to extract the first principle component from a HSI, and then select local image patches centered at each pixel from it. After that, we feed the row vectors of each image patch into Spatial LSTM one by one to learn the spatial feature for the center pixel. In the classification stage, the spectral and spatial features of each pixel are fed into softmax classifiers respectively to derive two different results, and a decision fusion strategy is further used to obtain a joint spectral-spatial results. Experiments are conducted on two widely used HSIs, and the results show that our method can achieve higher performance than other state-of-the-art methods.

Background: Hyperspectral Image (HSI) classification involves analyzing images captured across numerous spectral bands to identify and categorize materials or objects. By exploiting spectral information, HSI enables precise detection of materials based on their unique spectral signatures. This extensive spectral information allows for accurate identification of materials and land cover types, making HSI classification essential for various applications. This research developed an optimization-based Deep Learning (DL) framework for effective classification of HSI. The methodology begins with a preprocessing step using Minimum Noise Fraction (MNF), a technique that leverages the noise covariance structure of the dataset to reduce dimensionality while preserving the most informative spectral features. To enhance classification performance, a novel model is proposed, known as Archimedes Optimization Algorithm-3D Convolutional Neural Network (AOA based 3D CNN). Archimedes Optimization Algorithm (AOA) is applied to fine-tune learning rate of 3D Convolutional Neural Network (3D CNN), thereby optimizing training convergence and enhancing accuracy. Result: The proposed AOA based 3D CNN approach exhibits outstanding performance, achieving 95.6% accuracy, 95.9% TPR, 94.1% TNR, 94.9% PPV, a Kappa coefficient of 95.4%, and an F1-score of 95.5%.Keywords: These results highlight the model's effectiveness, adaptability and robustness for HSI classification.

One of the most important problems in classification of Hyperspectral images is Hughes phenomena. The main reason for this problem is high dimension feature space and deficiency of training samples. To solve this kind of problem, so many methods are applied which some of them emphasize feature reduction and the others point to classification method with low sensitivity to training samples or consider spatial information or semi supervised training samples. In feature reduction algorithms intensity vectors of pixels are used individually. Additionally, in most of such techniques each component of intensity vector is considered as an individual element without attention to inner relation among them. In this paper we consider members of intensity vector as sampling points of spectral reflecting curve (SRC) of corresponding land cover (LC) and use fractal features of SRC in order to classify the picture. Moreover, we will show using these features improve the classification results in multispectral images.

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

Hyperspectral image (HSI) classification is a classical task in remote sensing image analysis. With the development of deep learning, schemes based on deep learning have gradually become the mainstream of HSI classification. However, existing HSI classification schemes either lack the exploration of category-specific information in the spectral bands and the intrinsic value of information contained in features at different scales, or are unable to extract multiscale spatial information and global spectral properties simultaneously. To solve these problems, in this article, we propose a novel HSI classification framework named CL-MGNet, which can fully exploit the category-specific properties in spectral bands and obtain features with multiscale spatial information and global spectral properties. Specifically, we first propose a spectral weight learning (SWL) module with a category consistency loss to achieve the enhancement of information in important bands and the mining of category-specific properties. Then, a multiscale backbone is proposed to extract the spatial information at different scales and the cross-channel attention via multiscale convolution and a grouping attention module. Finally, we employ an attention multilayer perceptron (attention-MLP) block to exploit the global spectral properties of HSI, which is helpful for the final fully connected layer to obtain the classification result. The experimental results on five representative hyperspectral remote sensing datasets demonstrate the superiority of our method.

Recently, collaborative representation has received much attention in the hyperspectral image (HSI) classification due to its simplicity and effectiveness. However, the existing collaborative representation-based HSI classification methods ignore the correlation among different classes. To overcome this problem, we propose a discriminative kernel collaborative representation and Tikhonov regularization method (DKCRT) for HSI classification, which can make the kernel collaborative representation of different classes to be more discriminative. Specifically, the kernel trick is adopted to map the original HSI into a high space to improve the class separability. Besides, distance-weighted kernel Tikhonov regularization is adopted to enforce these training samples to have large representation coefficients, which are similar to the test sample in the high-dimensional feature space. Moreover, we add a discriminative regularization term to further enhance the separability of different classes, which can take the correlation among different classes into consideration. Furthermore, to take the spatial information of HSI into consideration, we extend the DKCRT to a joint version named JDKCRT. Experiments on real HSIs demonstrate the efficiency of the proposed DKCRT and JDKCRT.