A Novel Semi-Supervised Long-Tailed Learning Framework With Spatial Neighborhood Information for Hyperspectral Image Classification

Hyperspectral image (HSI) classification has attracted much attention and extensive research efforts over the past decade. Due to few labeled samples versus high dimensional features, it is a challenging problem in practice. Recently, combining the pixel spectral information and the spatial (neighborhood) information has been verified to be effective for HSI classification. In this paper, we introduce a novel method for HSI classification using set-to-set distance (SSD). Based on the assumption that neighbor pixels tend to belong to the same class with high probability, we model a test pixel and its neighbor pixels as a testing set (or a neighbor set) inspired by bilateral filtering. Meanwhile, the training pixels belong to the same class are modeled as a training set. Therefore, the classification is based on comparisons of sets distances. Experiments on a real HSI dataset show that our proposed method outperforms a number of existing state-of-the-art approaches.

<strong class="journal-contentHeaderColor">Abstract.</strong> Hyperspectral images contain abundant spectral and spatial information about the earth's surface, labeling data processing and analysis more difficult, as well as the problem of sample labeling. In this paper, local binary pattern (LBP), sparse representation and mixed logistic regression model are introduced, and a sample labeling method based on neighborhood information and priority classifier discrimination is presented. Then, a hyperspectral remote sensing image classification method based on texture features and semi-supervised learning is implemented. The LBP is employed to extract features of spatial texture information from remote sensing images and enrich the feature information of samples. Then the multivariate logistic regression model is used to select the unlabeled samples with the largest amount of information, and the unlabeled samples with neighborhood information and priority classifier tags are selected to obtain the pseudo-labeled samples after learning. By making full use of the advantages of sparse representation and mixed logistic regression model, a new hyperspectral remote sensing image classification model based on semi-supervised learning is constructed to effectively achieve accurate classification of hyperspectral images. The data of Indian Pines, Salinas scene and Pavia University are selected to verify the validity of the proposed method. The experiment results show that the proposed classification method can obtain higher classification accuracy and show stronger timeliness and generalization ability.

This paper studies the classification problem of hyperspectral image (HSI). Inspired by the great success of deep neural networks in Artificial Intelligence (AI), researchers have proposed different deep learning based algorithms to improve the performance of hyperspectral classification. However, deep learning based algorithms always require a large-scale annotated dataset to provide sufficient training. To address this problem, we propose a semi-supervised deep learning framework based on the residual networks (ResNets), which use very limited labeled data supplemented by abundant unlabeled data. The core of our framework is a novel dual-strategy sample selection co-training algorithm, which can successfully guide ResNets to learn from the unlabeled data by making full use of the complementary cues of the spectral and spatial features in HSI classification. Experiments on the benchmark HSI dataset and real HSI dataset demonstrate that, with a small number of training data, our approach achieves competitive performance with maximum improvement of 41% (compare with traditional convolutional neural network (CNN) with 5 initial training samples per class on Indian Pines dataset) for HSI classification as compared with the results from those state-of-the-art supervised and semi-supervised methods.

To overcome the difficulty of automating and intelligently classifying the ground features in remote-sensing hyperspectral images, machine learning methods are gradually introduced into the process of remote-sensing imaging. First, the PaviaU, Botswana, and Cuprite hyperspectral datasets are selected as research subjects in this study, and the objective is to process remote-sensing hyperspectral images via machine learning to realize the automatic and intelligent classification of features. Then, the basic principles of the support vector machine (SVM) and extreme learning machine (ELM) classification algorithms are introduced, and they are applied to the datasets. Next, by adjusting the parameter estimates using a restricted Boltzmann machine (RBM), a new terrain classification model of hyperspectral images that is based on a deep belief network (DBN) is constructed. Next, the SVM, ELM, and DBN classification algorithms for hyperspectral image terrain classification are analysed and compared in terms of accuracy and consistency. The results demonstrate that the average detection accuracies of ELM on the three datasets are 89.54%, 96.14%, and 96.28%, and the Kappa coefficient values are 0.832, 0.963, and 0.924; the average detection accuracies of SVM are 88.90%, 92.11%, and 91.68%, and the Kappa coefficient values are 0.768, 0.913, and 0.944; the average detection accuracies of the DBN classification model are 92.36%, 97.31%, and 98.84%, and the Kappa coefficient values are 0.883, 0.944, and 0.972. The results also demonstrate that the classification accuracy of the DBN algorithm exceeds those of the previous two methods because it fully utilizes the spatial and spectral information of hyperspectral remote-sensing images. In summary, the DBN algorithm that is proposed in this study has high application value in object classification for remote-sensing hyperspectral images.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

A decision tree network with semi-supervised entropy learning strategy for spectroscopy aided detection of blood hemoglobin

ABSTRACTIn the area of hyperspectral image (HSI) classification, graph-based semi-supervised learning (SSL) has been proved to be highly effective. Constructing a proper graph is critical for graph-based SSL tasks. In HSI, spectral distance is widely used to calculate the weight of graph edge, though it can be influenced by noise and outliers. Meanwhile, links among all the data points are incorporated in the graph, including those from different subspaces. Thus the constructed graph might contain incorrect information. In this letter, a novel semi-supervised HSI classification method using local low-rank representation (SL2R) is proposed. Edge weight calculation will not be affected by noise or outliers thanks to the robustness of low-rank representation (LRR). Since each graph is constructed at local level, where pixels are basically embedded in the same subspace, links among uncorrelated pixels can be removed. Moreover, spatial context is naturally characterized by low-rank constraint on adjacent pixels. Experimental results on two data sets (Indian Pines and Botswana) confirm the effectiveness of the proposed method.

Most hyperspectral image (HSI) classification methods assume that all classes in the test set are present during training. However, in real-world applications, acquiring labeled training samples is challenging. As a result, it is difficult for the training dataset to cover all possible land cover types, leading to the generalized zero-shot learning (GZSL) problem. Recently, vision-language models (VLMs) have provided rich semantic priors for land cover classes, offering promising potential for GZSL. However, two fundamental gaps hinder their application to HSI classification: the task paradigm gap, arising from the difference between image-level VLMs and the pixel-level HSI classification task; and the knowledge gap, due to the inconsistency between VLM features and HSI spectral–spatial representations. To bridge both gaps, a novel framework leveraging VLM semantic priors for GZSL in HSI classification is proposed, primarily using pseudo-labeling technique to provide knowledge for unseen classes. Specifically, a pseudo-label generation and enhancement module enables a paradigm transition from image-level understanding to pixel-level classification by incorporating HSI’s spatial information. A pseudo-label correction module then refines noisy labels using spectral cues to address the knowledge gap. Finally, a global learning strategy integrates pseudo-label distillation, supervised learning, and feature regularization to classify seen classes while enabling generalization to unseen ones. Experiments on benchmark HSI datasets demonstrate the proposed method’s superiority in generalized zero-shot classification. This work highlights the potential of VLMs in advancing HSI classification in practical applications.

A novel semi-supervised hyperspectral image classification approach based on spatial neighborhood information and classifier combination

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework that divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

Semantic segmentation is a crucial approach for remote sensing interpretation. High-precision semantic segmentation results are obtained at the cost of manually collecting massive pixelwise annotations. Remote sensing imagery contains complex and variable ground objects and obtaining abundant manual annotations is expensive and arduous. The semi-supervised learning (SSL) strategy can enhance the generalization capability of a model with a small number of labeled samples. In this study, a novel semi-supervised adversarial semantic segmentation network is developed for remote sensing information extraction. A multiscale input convolution module (MICM) is designed to extract sufficient local features, while a Transformer module (TM) is applied for long-range dependency modeling. These modules are integrated to construct a segmentation network with a double-branch encoder. Additionally, a double-branch discriminator network with different convolution kernel sizes is proposed. The segmentation network and discriminator network are jointly trained under the semi-supervised adversarial learning (SSAL) framework to improve its segmentation accuracy in cases with small amounts of labeled data. Taking building extraction as a case study, experiments on three datasets with different resolutions are conducted to validate the proposed network. Semi-supervised semantic segmentation models, in which DeepLabv2, the pyramid scene parsing network (PSPNet), UNet and TransUNet are taken as backbone networks, are utilized for performance comparisons. The results suggest that the approach effectively improves the accuracy of semantic segmentation. The F1 and mean intersection over union (mIoU) accuracy measures are improved by 0.82–11.83% and 0.74–7.5%, respectively, over those of other methods.

Classification of hyperspectral images has recently gained significant popularity due to both the development of remote sensing technologies and the advances in image analysis approaches. One crucial step to achieve accurate classification is to acquire sufficient high-quality training data, which is often a time-consuming and expensive process. To alleviate this burden, in this letter, we propose an active and semisupervised learning (SSL) approach that utilizes morphological component analysis (MCA) for classification of hyperspectral images. First, the original hyperspectral data are decomposed into its morphological components via MCA. In each feature domain, the active learning (AL) and SSL are combined to enlarge the training data set based on superpixels. Finally, decision fusion is carried out to integrate the predictions from the two components. The proposed method is tested on both benchmark and real world application hyperspectral data sets. Experimental results indicate that the proposed method can lead to a better classification with respect to the conventional AL approaches.

In the classification of hyperspectral images with supervised methods, acquisition of ground-truth information for a hyperspectral image is a challenging process in terms of time and cost. Besides, amount of the labeled data also affects the performance of classifiers. In this study, as a solution to this problem, a hyperspectral image classifier is proposed with semi-supervised learning, support vector machine classifier and deep learning. In the first phase to improve the classification performance, limited number of training data is increased by semisupervised learning methodology. Then, the classification process is performed with support vector machines and convolutional neural networks. According to the acquired classification results, a close classification performance is obtained by the system with small number of training data to the supervised classification. Furthermore, deep neural network has reached more successful results than support vector machines.

SemanticFormer: Hyperspectral image classification via semantic transformer