A multiple criteria-based spectral partitioning method for remotely sensed hyperspectral image classification

ABSTRACTGraph-based methods are developed to efficiently extract data information. In particular, these methods are adopted for high-dimensional data classification by exploiting information residing on weighted graphs. In this paper, we propose a new hyperspectral texture classifier based on graph-based wavelet transform. This recent graph transform allows extracting textural features from a constructed weighted graph using sparse representative pixels of hyperspectral image. Different measurements of spectral similarity between representative pixels are tested to decorrelate close pixels and improve the classification precision. To achieve the hyperspectral texture classification, Support Vector Machine is applied on spectral graph wavelet coefficients. Experimental results obtained by applying the proposed approach on Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and Reflective Optics System Imaging Spectrometer (ROSIS) datasets provide good accuracy which could exceed 98.7%. Compared to other famous classification methods as conventional deep learning-based methods, the proposed method achieves better classification performance. Results have shown the effectiveness of the method in terms of robustness and accuracy.

Abstract. Hyperspectral images (HSIs) contain hundreds of spectral bands, providing high-resolution spectral information pertaining to the Earth’s surface. Additionally, abundant spatial contextual information can also be obtained simultaneously from a HSI. To characterize the properties of ground objects, classification is the most widely-used technology in the field of remote sensing, where each pixel in a HSI is assigned to a pre-defined class. Over the past decade, deep learning has attracted increasing attention in the machine-learning and computer-vision domains, due to its favourable performances for various types of tasks, and it has been successfully introduced to the remote-sensing community. Instead of utilizing the shallow features within in a given image, which is the approach that is generally adopted in other conventional classification methods, deep-learning algorithms can extract hierarchical features from raw HSI data. Within the deep-learning framework, recurrent neural networks (RNNs), which are able to encode sequential features, have exhibited promising capabilities and have achieved encouraging performances, especially for the natural-language processing and speech-recognition communities. As multi-temporal remote-sensing images can be readily obtained from increasing numbers of satellite and unmanned aircraft systems, and since analysis of such multi-temporal data comprises a critical issue within numerous research subfields, including land-cover and land-change analyses, and land-resource management, RNNs have been applied in recent studies in order to extract temporal sequential features from multi-temporal remote-sensing images for the purpose of image classification. Apart from using multi-temporal image datasets, RNNs can also be utilized on a single image, where the spectral feature/band of each individual pixel can be taken as a sequential feature for the input layer of RNNs. However, the application of such sequential feature extraction that relies on a single image still needs to be further investigated since applying RNNs to spectral bands will directly introduce more parameters that need to be optimized, consequently increasing the total training time.In this study, we propose a novel RNN-based HSI classification framework. In this framework, unlabelled pixels obtained from a single image are considered when constructing sequential features. Two spatial similarity measurements, referred to as pixel-matching and block-matching, respectively, are employed to extract pixels that are “similar” to the target pixel. Then, the sequential feature of the target pixel is constructed by exploiting several of the most “similar” pixels and ordering them based on their similarities to the target pixel. The aforementioned two schemes are advantageous, as unlabelled pixels within the given HSI are taken into consideration for similarity measurement and sequential feature construction for the RNN model. Moreover, the block-matching scheme also takes advantage of spatial contextual information, which has been widely utilized in spatial-spectral-based HSI classification methods. To evaluate the proposed methods, two benchmark HSIs are used, including a HSI collected over Pavia University, Italy by the airborne Reflective Optics System Imaging Spectrometer (ROSIS) sensor, and an image acquired over the Salinas Valley, California, USA via the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) sensor. Spatio-temporally coincident ground-reference data accompanies each of these respective HSIs. In addition, the proposed methods are compared with three state-of-the-art algorithms, including support vector machine (SVM), the 1-dimensional convolutional neural network (1DCNN), and the 1-dimensional RNN (1DRNN).Experimental results indicate that our proposed methods achieve markedly better classification performance compared with the baseline algorithms on both datasets. For example, for the Pavia University image, the block-matching based RNN achieves the highest overall classification accuracy, with 94.32% accuracy, which is 9.87% higher than the next most accurate algorithm of the aforementioned three baseline methods, which in this case is the 1DCNN, with 84.45% overall accuracy. More specifically, the block-matching method performs better than the pixel-matching method in terms of both quantitative and qualitative assessments. Based on visual assessment/interpretation of the classification maps, it is apparent that “salt-and-pepper” noise is markedly alleviated; with block-matching, smoother classified images are generated compared with pixel-matching-based methods and the three baseline algorithms. Such results demonstrate the effectiveness of utilizing spatial contextual information in the similarity measurement.

Band selection (BS) aims to choose a salient subset implied sufficient information from the numerous bands, which supplies a significantly efficient way to alleviate the barrier of dimensionality disaster for hyperspectral image classification (HSIC). This letter develops a semisupervised BS approach based on dual-constrained low-rank representation BS (DCLRR-BS) with two regularizations for HSIC. To be specific, a low-rank representation model is first proposed with super-pixel and imbalanced class-wise constraints, which are explicitly integrated to improve the performance of the band description. Next, the clusters are built adaptively based on graph theory in an unsupervised manner to rapid selection efficiency. A selection criterion is last designed to highlight the prominent band of each subset cluster to fulfill the BS procedure. Experimental results conducted on four types of classifiers with two real hyperspectral image (HSI) data sets demonstrate that the proposed DCLRR-BS method performs well in the imbalanced HSIC area.

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

Remotely sensed hyperspectral imaging allows for the detailed analysis of the surface of the Earth using advanced imaging instruments which can produce high-dimensional images with hundreds of spectral bands. Supervised hyperspectral image classification is a difficult task due to the unbalance between the high dimensionality of the data and the limited availability of labeled training samples in real analysis scenarios. While the collection of labeled samples is generally difficult, expensive, and time-consuming, unlabeled samples can be generated in a much easier way. This observation has fostered the idea of adopting semisupervised learning techniques in hyperspectral image classification. The main assumption of such techniques is that the new (unlabeled) training samples can be obtained from a (limited) set of available labeled samples without significant effort/cost. In this paper, we develop a new approach for semisupervised learning which adapts available active learning methods (in which a trained expert actively selects unlabeled samples) to a self-learning framework in which the machine learning algorithm itself selects the most useful and informative unlabeled samples for classification purposes. In this way, the labels of the selected pixels are estimated by the classifier itself, with the advantage that no extra cost is required for labeling the selected pixels using this machine–machine framework when compared with traditional machine–human active learning. The proposed approach is illustrated with two different classifiers: multinomial logistic regression and a probabilistic pixelwise support vector machine. Our experimental results with real hyperspectral images collected by the National Aeronautics and Space Administration Jet Propulsion Laboratory's Airborne Visible–Infrared Imaging Spectrometer and the Reflective Optics Spectrographic Imaging System indicate that the use of self-learning represents an effective and promising strategy in the context of hyperspectral image classification.

Convolutional neural network (CNN) is well known for its capability of feature learning and has made revolutionary achievements in many applications, such as scene recognition and target detection. In this paper, its capability of feature learning in hyperspectral images is explored by constructing a five-layer CNN for classification (C-CNN). The proposed C-CNN is constructed by including recent advances in deep learning area, such as batch normalization, dropout, and parametric rectified linear unit (PReLU) activation function. In addition, both spatial context and spectral information are elegantly integrated into the C-CNN such that spatial-spectral features are learned for hyperspectral images. A companion feature-learning CNN (FL-CNN) is constructed by extracting fully connected feature layers in this C-CNN. Both supervised and unsupervised modes are designed for the proposed FL-CNN to learn sensor-specific spatial-spectral features. Extensive experimental results on four benchmark data sets from two well-known hyperspectral sensors, namely airborne visible/infrared imaging spectrometer (AVIRIS) and reflective optics system imaging spectrometer (ROSIS) sensors, demonstrate that our proposed C-CNN outperforms the state-of-the-art CNN-based classification methods, and its corresponding FL-CNN is very effective to extract sensor-specific spatial-spectral features for hyperspectral applications under both supervised and unsupervised modes.

This study is focused on time optimization for the classification problem presenting a comparison of five Artificial Neural Network Multilayer Perceptron (ANN-MLP) architectures. We use the Artificial Neural Network (ANN) because it allows to recognize patterns in data in a lower time rate. Time and classification accuracy are taken into account together for the comparison. According to time comparison, two paradigms in the computational field for each ANN-MLP architecture are analysed with three schemes. Firstly, sequential programming is applied by using a single CPU core. Secondly, parallel programming is employed over a multi-core CPU architecture. Finally, a programming model running on GPU architecture is implemented. Furthermore, the classification accuracy is compared between the proposed five ANN-MLP architectures and a state-of.the-art Support Vector Machine (SVM) with three classification frames: 50%,60% and 70% of the data set's observations are randomly selected to train the classifiers. Also, a visual comparison of the classified results is presented. The Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) criteria are also calculated to characterise visual perception. The images employed were acquired by the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS), the Reflective Optics System Imaging Spectrometer (ROSIS) and the Hyperion sensor.

Recently, there have been continuous advancements in hyperspectral image classification (HSIC) approaches based on deep learning mechanisms. Nonetheless, information redundancy, high complexity and ineffective descriptions remain primary obstacles to present HSIC networks. In order to overcome these issues, an optimization-enabled deep convolutional neural network (Deep CNN) for HSIC is proposed in this work. Here, band selection, feature extraction, as well as classification are important tasks in processing these images. The band selection is performed by utilizing the double exponential smoothing-artificial flora optimization (DES-AFO) algorithm, which is the combination of double exponential smoothing (DES) and artificial flora optimization (AFO) for HSIC. There is a requirement for extracting competent features from hyperspectral imaging (HSI) to enhance the classifier’s accuracy. Here, the feature extraction is done by empirical wavelet transform and a convolutional neural network (CNN). As the next process, HSIC takes place using the Deep CNN, and it is trained by DES-AFO. The wide experimental outcomes on HSI show that the proposed mechanism has noteworthy benefits estimated with the conventional mechanisms concerning accuracy and TPR. Here, analysis states that the proposed mechanism obtained better accuracy and TPR values of 0.930, as well as 0.965.

Hyperspectral image classification using FPCA-based kernel extreme learning machine

Band selection (BS) is an effective technique of dimensionality reduction in hyperspectral images (HSIs), which can solve the problems of high computational complexity and information redundancy and is helpful for the classification of HSIs. Since many hyperspectral scenes have limited label samples, semisupervised BS methods have attracted much attention. In this paper, an improved graph-based semisupervised BS method is proposed. The within-class and between-class graphs are firstly constructed by utilizing the information of the labeled and unlabeled samples, then Fisher's criteria combined with sparsity regularization (FCS) is designed to obtain the optimal projection matrix, which is used to select the representative bands. The proposed algorithm performs BS while preserving the local manifold structure of data. Experimental results on three hyperspectral data sets show that the proposed BS algorithm has good performance in selecting representative bands for classification.

Hyperspectral image (HSI) classification has become a hot topic in the field of remote sensing. In general, the complex characteristics of hyperspectral data make the accurate classification of such data challenging for traditional machine learning methods. In addition, hyperspectral imaging often deals with an inherently nonlinear relation between the captured spectral information and the corresponding materials. In recent years, deep learning has been recognized as a powerful feature-extraction tool to effectively address nonlinear problems and widely used in a number of image processing tasks. Motivated by those successful applications, deep learning has also been introduced to classify HSIs and demonstrated good performance. This survey paper presents a systematic review of deep learning-based HSI classification literatures and compares several strategies for this topic. Specifically, we first summarize the main challenges of HSI classification which cannot be effectively overcome by traditional machine learning methods, and also introduce the advantages of deep learning to handle these problems. Then, we build a framework that divides the corresponding works into spectral-feature networks, spatial-feature networks, and spectral-spatial-feature networks to systematically review the recent achievements in deep learning-based HSI classification. In addition, considering the fact that available training samples in the remote sensing field are usually very limited and training deep networks require a large number of samples, we include some strategies to improve classification performance, which can provide some guidelines for future studies on this topic. Finally, several representative deep learning-based classification methods are conducted on real HSIs in our experiments.

Hyperspectral image (HSI) provided with a substantial number of correlated bands causes calculation consumption and an undesirable "dimension disaster" problem for the classification. Band selection (BS) is an effective measure to reduce the information redundancy with the physics spectrum preserved for HSI. Although the existing BS methods have achieved noticeable progress, the correlation between neighbor bands still needs to be mined deeply for an effective selection criterion. This paper proposes a BS approach to collecting the discriminative band subset for hyperspectral image classification (HSIC), which adopts the self-supervised learning paradigm to implement the BS by auxiliary spectrum rebuilding task. In specific, we utilized a Convolutional neural network (CNN) and Graph Convolutional Network (GCN) for the spectral-spatial feature extraction. Next, GCN and CNN are developed for the refinement of the band correlation sequentially. Afterward, the selected bands in terms of the acquired correlation are fed into the presented self-supervised spectrum rebuilding network for spectral reconstruction. Simultaneously, the proposed architecture completed the selection with the optimization of the band reconstruction by a defined loss function. In this way, we supply substitution for selection criterion and path searching through the end-to-end framework. The extensive experimental results and analysis demonstrated that the proposed hybrid architecture provided a competitive band subset for the classification, and the accuracies with different types of classifiers are more effective than the compared BS methods.

Although graph convolutional network (GCN) has achieved remarkable success in hyperspectral image (HSI) classification, most existing GCN-based approaches have failed to realize a deep network structure due to the oversmoothing problem. This problem largely limits the expression ability and feature extraction ability of GCN and hampers GCN's capacity to model long-range relationships between samples in hyperspectral (HS) scenes. Moreover, there is a lack of theoretical analysis in those works that constructed deep GCN for HSI classification to illustrate how they overcome the oversmoothing problem. Aside from this, the characteristics and complexity of HSI are often neglected when constructing deep GCN models in HSI classification. To address these problems, a novel deep graph network based on first-order smoothing is proposed for HSI classification. Specifically, a local and global topologically consistent graph is constructed to thoroughly explore the union between fine pixel information and semantic superpixel information. Subsequently, a novel propagation procedure is proposed to address the oversmoothing problem. We creatively build a residual connection to the first layer to emphasize the feature information aggregated from the first-order neighborhood, which adds node features that have not yet become indistinguishable into deep layer, and at the same time, it can be considered as a correction to the original pixels affected by spectral variation in the input graph. Finally, we demonstrate how first-order smoothing-based deep graph network (FSDGN) can slow down the convergence rate of the oversmoothing problem by analyzing the propagation of FSDGN from the standpoint of the Laplacian spectral domain. In addition, the results of experiments performed on three benchmark datasets demonstrate its superiority over other state-of-the-art methods.

Band selection is a great challenging task in the classification of hyperspectral remotely sensed images HSI. This is resulting from its high spectral resolution, the many class outputs and the limited number of training samples. For this purpose, this paper introduces a new filter approach for dimension reduction and classification of hyperspectral images using information theoretic (normalized mutual information) and support vector machines SVM. This method consists to select a minimal subset of the most informative and relevant bands from the input datasets for better classification efficiency. We applied our proposed algorithm on two well-known benchmark datasets gathered by the NASA's AVIRIS sensor over Indiana and Salinas valley in USA. The experimental results were assessed based on different evaluation metrics widely used in this area. The comparison with the state of the art methods proves that our method could produce good performance with reduced number of selected bands in a good timing. Keywords: Dimension reduction, Hyperspectral images, Band selection, Normalized mutual information, Classification, Support vector machines