- Research Article
78
- 10.3390/rs10050711
Remote Sensing Big Data: Theory, Methods and Applications
- May 04, 2018
- Remote Sensing
- Peng Liu + 3 more +3
Nowadays, our ability to acquire remote sensing data has been improved to an unprecedented level.[...]
The rapid development of high spatial resolution (HSR) remote sensing imagery techniques not only provide a considerable amount of datasets for scene classification tasks but also request an appropriate scene classification choice when facing with finite labeled samples. AlexNet, as a relatively simple convolutional neural network (CNN) architecture, has obtained great success in scene classification tasks and has been proven to be an excellent foundational hierarchical and automatic scene classification technique. However, current HSR remote sensing imagery scene classification datasets always have the characteristics of small quantities and simple categories, where the limited annotated labeling samples easily cause non-convergence. For HSR remote sensing imagery, multi-scale information of the same scenes can represent the scene semantics to a certain extent but lacks an efficient fusion expression manner. Meanwhile, the current pre-trained AlexNet architecture lacks a kind of appropriate supervision for enhancing the performance of this model, which easily causes overfitting. In this paper, an improved pre-trained AlexNet architecture named pre-trained AlexNet-SPP-SS has been proposed, which incorporates the scale pooling—spatial pyramid pooling (SPP) and side supervision (SS) to improve the above two situations. Extensive experimental results conducted on the UC Merced dataset and the Google Image dataset of SIRI-WHU have demonstrated that the proposed pre-trained AlexNet-SPP-SS model is superior to the original AlexNet architecture as well as the traditional scene classification methods.
Loading PDF
Remote Sensing Big Data: Theory, Methods and Applications
Nowadays, our ability to acquire remote sensing data has been improved to an unprecedented level.[...]
Remote Scene Image Scene Classification Based on Adaptive Segmentation and Dynamic Graph Convolution
As an important research topic in the remote sensing (RS) community, RS image scene classification is a challenging task due to the complex contents of RS images. In general, RS image scene classification is a single-label problem. Nevertheless, it is known that the contents within RS are huge in volume and diverse in type. Only a single semantic label cannot describe an RS scene completely, especially when the resolution of RS images is increased recently. The various semantics hidden in the high-resolution RS (HRRS) images are also important to the scene classification task. Taking the issues mentioned above into account, we develop a new scene classifier named graph scene classifier (GSCer) for HRRS images with the help of the deep convolution neural network (DCNN) and dynamic graph convolution (DGCN). Not only the global semantic but also the diverse hidden local semantics within an HRRS image can be fully explored. The encouraging experimental results counted on two public HRRS data sets demonstrate that our GSCer is effective in HRRS scene classification tasks.
Read moreHigh Spatial Resolution Remote Sensing Image Scene Classification using CNN with Transfer Learning
Traditional HSR remote sensing imagery understanding is based on recognizing pixel-based or object-based ground elements, but this cannot describe the whole content of the scene images and cannot bridge the "semantic gap" between the low-level features and the high-level semantics. Deep learning algorithms attempt to learn hierarchical features, corresponding to different levels of abstraction. Current progress in deep-learning models, specifically d eep c onvolutional n eural network (CNN) architectures, have improved the state-of-the-art in many fields of study i ncluding r emote s ensing scene c lassification. The choice of a proper network architecture for making strong and correct assumptions about the nature of the input data is still a big challenge. This work presents an implementation of the Convolutional Neural Network (CNN) AlexNet trained on the well-known datasets UC Merced Land-Use and WHU- RS by using Transfer Learning for the High Spatial Resolution Image Scene Classification p roblem. A l ayer c orresponding to the Spatial Pyramid Pooling (SPP) is incorporated in order to consider different image sizes in the input of the network and multi-scale spatial information of the scenes to be classified. Results are comparable to literature, improving some published approaches.
Read moreAttention based Residual Network for High-Resolution Remote Sensing Imagery Scene Classification
Remote sensing image scene classification, which aims to identify the types of land cover, is a fundamental task in remote sensing image analysis. Remote sensing images contain a variety of land-cover objects. These land-cover objects form a complex and diverse scene through spatial combination and correlation, which makes remote sensing imagery scenes classification difficult. In addition, remote sensing images contain redundant information that has a negative impact on remote sensing imagery scene classification, which makes remote sensing imagery scenes classification rather challenging. Recently, there are many deep learning based methods, which have achieved remarkable performance through an end-to-end supervised training process. Existing advances in remote sensing imagery scene classification mainly focus on training multi-layer convolutional neural networks (CNNs). These CNNs do not explicitly distinguish between key information and redundant information of the image. Therefore, the ability to extract features is limited. How to focus on key information and ignore redundant information in remote sensing imagery scene classification is a valuable problem. Inspired by the attention mechanism, we propose a CNN-based network that combines residual units and attention mechanism. It automatically assigns large weights to key areas of the image and thus has the ability to adaptively ignore redundant information. We evaluated the proposed approach with some state-of-the-art methods on the UC Merced Land-Use dataset and the NWPU-RESISC45 dataset. Experimental results show that the proposed attention model has achieved the best classification performance.
Read moreScene Classification Based on a Deep Random-Scale Stretched Convolutional Neural Network
With the large number of high-resolution images now being acquired, high spatial resolution (HSR) remote sensing imagery scene classification has drawn great attention but is still a challenging task due to the complex arrangements of the ground objects in HSR imagery, which leads to the semantic gap between low-level features and high-level semantic concepts. As a feature representation method for automatically learning essential features from image data, convolutional neural networks (CNNs) have been introduced for HSR remote sensing image scene classification due to their excellent performance in natural image classification. However, some scene classes of remote sensing images are object-centered, i.e., the scene class of an image is decided by the objects it contains. Although previous methods based on CNNs have achieved comparatively high classification accuracies compared with the traditional methods with handcrafted features, they do not consider the scale variation of the objects in the scenes. This makes it difficult to directly utilize CNNs on those remote sensing images belonging to object-centered classes to extract features that are robust to scale variation, leading to wrongly classified scene images. To solve this problem, scene classification based on a deep random-scale stretched convolutional neural network (SRSCNN) for HSR remote sensing imagery is proposed in this paper. In the proposed method, patches with a random scale are cropped from the image and stretched to the specified scale as the input to train the CNN. This forces the CNN to extract features that are robust to the scale variation. Furthermore, to further improve the performance of the CNN, a robust scene classification strategy is adopted, i.e., multi-perspective fusion. The experimental results obtained using three datasets—the UC Merced dataset, the Google dataset of SIRI-WHU, and the Wuhan IKONOS dataset—confirm that the proposed method performs better than the traditional scene classification methods.
Read moreSAGN: Semantic-Aware Graph Network for Remote Sensing Scene Classification.
The scene classification of remote sensing (RS) images plays an essential role in the RS community, aiming to assign the semantics to different RS scenes. With the increase of spatial resolution of RS images, high-resolution RS (HRRS) image scene classification becomes a challenging task because the contents within HRRS images are diverse in type, various in scale, and massive in volume. Recently, deep convolution neural networks (DCNNs) provide the promising results of the HRRS scene classification. Most of them regard HRRS scene classification tasks as single-label problems. In this way, the semantics represented by the manual annotation decide the final classification results directly. Although it is feasible, the various semantics hidden in HRRS images are ignored, thus resulting in inaccurate decision. To overcome this limitation, we propose a semantic-aware graph network (SAGN) for HRRS images. SAGN consists of a dense feature pyramid network (DFPN), an adaptive semantic analysis module (ASAM), a dynamic graph feature update module, and a scene decision module (SDM). Their function is to extract the multi-scale information, mine the various semantics, exploit the unstructured relations between diverse semantics, and make the decision for HRRS scenes, respectively. Instead of transforming single-label problems into multi-label issues, our SAGN elaborates the proper methods to make full use of diverse semantics hidden in HRRS images to accomplish scene classification tasks. The extensive experiments are conducted on three popular HRRS scene data sets. Experimental results show the effectiveness of the proposed SAGN.
Read moreA Lightweight and Discriminative Model for Remote Sensing Scene Classification With Multidilation Pooling Module
With the growing spatial resolution of satellite images, high spatial resolution (HSR) remote sensing imagery scene classification has become a challenging task due to the highly complex geometrical structures and spatial patterns in HSR imagery. The key issue in scene classification is how to understand the semantic content of the images effectively, and researchers have been looking for ways to improve the process. Convolutional neural networks (CNNs), which have achieved amazing results in natural image classification, were introduced for remote sensing image scene classification. Most of the researches to date have improved the final classification accuracy by merging the features of CNNs. However, the entire models become relatively complex and cannot extract more effective features. To solve this problem, in this paper, we propose a lightweight and effective CNN which is capable of maintaining high accuracy. We use MobileNet V2 as a base network and introduce the dilated convolution and channel attention to extract discriminative features. To improve the performance of the CNN further, we also propose a multidilation pooling module to extract multiscale features. Experiments are performed on six datasets, and the results verify that our method can achieve higher accuracy compared to the current state-of-the-art methods.
Read moreA semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification
A semi-supervised generative framework with deep learning features for high-resolution remote sensing image scene classification
Read moreHigh-Resolution Remote Sensing Image Scene Classification via Key Filter Bank Based on Convolutional Neural Network
High-resolution remote sensing (HRRS) image scene classification has attracted an enormous amount of attention due to its wide application in a range of tasks. Due to the rapid development of deep learning (DL), models based on convolutional neural network (CNN) have made competitive achievements on HRRS image scene classification because of the excellent representation capacity of DL. The scene labels of HRRS images extremely depend on the combination of global information and information from key regions or locations. However, most existing models based on CNN tend only to represent the global features of images or overstate local information capturing from key regions or locations, which may confuse different categories. To address this issue, a key region or location capturing method called key filter bank (KFB) is proposed in this article, and KFB can retain global information at the same time. This method can combine with different CNN models to improve the performance of HRRS imagery scene classification. Moreover, for the convenience of practical tasks, an end-to-end model called KFBNet where KFB combined with DenseNet-121 is proposed to compare the performance with existing models. This model is evaluated on public benchmark data sets, and the proposed model makes better performance on benchmarks than the state-of-the-art methods.
Read moreAttack Selectivity of Adversarial Examples in Remote Sensing Image Scene Classification
Remote sensing image (RSI) scene classification is the foundation and important technology of ground object detection, land use management and geographic analysis. During recent years, convolutional neural networks (CNNs) have achieved significant success and are widely applied in RSI scene classification. However, crafted images that serve as adversarial examples can potentially fool CNNs with high confidence and are hard for human eyes to interpret. For the increasing security and robust requirements of RSI scene classification, the adversarial example problem poses a serious problem for the classification results derived from systems using CNN models, which has not been fully recognized by previous research. In this study, to explore the properties of adversarial examples of RSI scene classification, we create different scenarios by testing two major attack algorithms (i.e., the fast gradient sign method (FGSM) and basic iterative method (BIM)) trained on different RSI benchmark datasets to fool CNNs (i.e., InceptionV1, ResNet and a simple CNN). In the experiment, our results show that CNNs of RSI scene classification are also vulnerable to adversarial examples, and some of them have a fooling rate of over 80%. These adversarial examples are affected by the architecture of CNNs and the type of RSI dataset. InceptionV1 has a fooling rate of less than 5%, which is lower than the others. Adversarial examples generated on the UCM dataset are easier than other datasets. Importantly, we also find that the classes of adversarial examples have an attack selectivity property. Misclassifications of adversarial examples of RSIs are related to the similarity of the original classes in the CNN feature space. Attack selectivity reveals potential classes of adversarial examples and provides insights into the design of defensive algorithms in future research.
Read moreResidual Channel-attention (RCA) network for remote sensing image scene classification
High-resolution remote sensing (HRRS) image scene classification has gained increasing importance in recent years, with convolutional neural networks (CNNs) showing particular promise due to their proficiency in extracting spatial features. However, traditional CNNs face significant limitations. Specifically, they struggle to capture complex semantic relationships between objects at varying scales, and they lack the ability to effectively capture long-distance dependencies between features. This limitation is especially problematic in HRRS images, where spatial relationships and semantic content are deeply intertwined. Additionally, traditional CNNs are limited in handling substantial intra-class variation and inter-class similarity, which are common in remote sensing images. To overcome these challenges, we introduce a novel Residual Channel-attention (RCA) network for scene classification. The RCA network introduces a lightweight residual structure to better capture multi-scale spatial features and incorporates a channel attention mechanism that selectively emphasizes relevant feature channels while suppressing irrelevant ones. To further refine the focus on critical image features, we integrate a squeeze-and-excitation (SE) mechanism as a self-attention component, which helps the network prioritize the most informative features and ignore background noise. We evaluated the RCA network on three public datasets: RSSCN7, PatternNet, and EuroSAT, achieving classification accuracies of 97%, 99%, and 96%, respectively. The results demonstrate that superior of the RCA network compared to state-of-the-art strategies in remote sensing image classification. Furthermore, visualization using the Grad-CAM++ algorithm highlights the effectiveness of our channel attention mechanism and underscores the RCA network’s robust feature representation capabilities.
Read moreDynamic convolutional model based on distribution-collaboration strategy for remote sensing scene classification
Most existing remote sensing image scene classification (RSISC) models rely mainly on cloud or terminal convolutional neural networks (CNN). However, with the advent of the era of high-resolution remote sensing image (HRRSI) big data, the accuracy and efficiency of such models on remote sensing multi-platforms are poor, which cannot satisfy the real-time and efficient image interpretation. This article proposes a novel RSISC model based on a distribution-collaboration strategy to address this problem. Firstly, the strategy allows most HRRSI to provide classification results in advance within a certain expected confidence range, reducing cloud transmission, reducing latency, and improving inference efficiency. Secondly, an adaptive enhanced attention mechanism based on the Lie Group feature covariance matrix is designed to aggregate the essential attribute feature (EAF) of HRRSI, which can effectively deal with HRRSI without increasing the computational complexity and delay of the model with the increase of HRRSI resolution. Finally, two types of dynamic convolution kernel (class consensus kernel and adaptive kernel) are proposed for scene classification. We chose to conduct extensive experiments on three challenging datasets and compared them with the state-of-the-art models, our model has certain advantages in classification accuracy, inference efficiency, computational performance, and number of parameters.
Read moreLarge patch convolutional neural networks for the scene classification of high spatial resolution imagery
The increase of the spatial resolution of remote-sensing sensors helps to capture the abundant details related to the semantics of surface objects. However, it is difficult for the popular object-oriented classification approaches to acquire higher level semantics from the high spatial resolution remote-sensing (HSR-RS) images, which is often referred to as the "semantic gap." Instead of designing sophisticated operators, convolutional neural networks (CNNs), a typical deep learning method, can automatically discover intrinsic feature descriptors from a large number of input images to bridge the semantic gap. Due to the small data volume of the available HSR-RS scene datasets, which is far away from that of the natural scene datasets, there have been few reports of CNN approaches for HSR-RS image scene classifications. We propose a practical CNN architecture for HSR-RS scene classification, named the large patch convolutional neural network (LPCNN). The large patch sampling is used to generate hundreds of possible scene patches for the feature learning, and a global average pooling layer is used to replace the fully connected network as the classifier, which can greatly reduce the total parameters. The experiments confirm that the proposed LPCNN can learn effective local features to form an effective representation for different land-use scenes, and can achieve a performance that is comparable to the state-of-the-art on public HSR-RS scene datasets.
Read moreCross-Domain Scene Classification Based on a Spatial Generalized Neural Architecture Search for High Spatial Resolution Remote Sensing Images
By labelling high spatial resolution (HSR) images with specific semantic classes according to geographical properties, scene classification has been proven to be an effective method for HSR remote sensing image semantic interpretation. Deep learning is widely applied in HSR remote sensing scene classification. Most of the scene classification methods based on deep learning assume that the training datasets and the test datasets come from the same datasets or obey similar feature distributions. However, in practical application scenarios, it is difficult to guarantee this assumption. For new datasets, it is time-consuming and labor-intensive to repeat data annotation and network design. The neural architecture search (NAS) can automate the process of redesigning the baseline network. However, traditional NAS lacks the generalization ability to different settings and tasks. In this paper, a novel neural network search architecture framework—the spatial generalization neural architecture search (SGNAS) framework—is proposed. This model applies the NAS of spatial generalization to cross-domain scene classification of HSR images to bridge the domain gap. The proposed SGNAS can automatically search the architecture suitable for HSR image scene classification and possesses network design principles similar to the manually designed networks. To obtain a simple and low-dimensional search space, the traditional NAS search space was optimized and the human-the-loop method was used. To extend the optimized search space to different tasks, the search space was generalized. The experimental results demonstrate that the network searched by the SGNAS framework with good generalization ability displays its effectiveness for cross-domain scene classification of HSR images, both in accuracy and time efficiency.
Read moreDO WE STILL NEED IMAGENET PRE-TRAINING IN REMOTE SENSING SCENE CLASSIFICATION?
Abstract. Due to the scarcity of labeled data, using supervised models pre-trained on ImageNet is a de facto standard in remote sensing scene classification. Recently, the availability of larger high resolution remote sensing (HRRS) image datasets and progress in self-supervised learning have brought up the questions of whether supervised ImageNet pre-training is still necessary for remote sensing scene classification and would supervised pre-training on HRRS image datasets or self-supervised pre-training on ImageNet achieve better results on target remote sensing scene classification tasks. To answer these questions, in this paper we both train models from scratch and fine-tune supervised and self-supervised ImageNet models on several HRRS image datasets. We also evaluate the transferability of learned representations to HRRS scene classification tasks and show that self-supervised pre-training outperforms the supervised one, while the performance of HRRS pre-training is similar to self-supervised pre-training or slightly lower. Finally, we propose using an ImageNet pre-trained model combined with a second round of pre-training using in-domain HRRS images, i.e. domain-adaptive pre-training. The experimental results show that domain-adaptive pre-training results in models that achieve state-of-the-art results on HRRS scene classification benchmarks. The source code and pre-trained models are available at https://github.com/risojevicv/RSSC-transfer.
Read more