FishermaskFormer: Lightweight Remote Sensing Scene Classification With Masked Transformer

Remote sensing scene classification converts remote sensing images into classification information to support high-level applications, so it is a fundamental problem in the field of remote sensing. In recent years, many convolutional neural network (CNN)-based methods have achieved impressive results in remote sensing scene classification, but they have two problems in extracting remote sensing scene features: (1) fixed-shape convolutional kernels cannot effectively extract features from remote sensing scenes with complex shapes and diverse distributions; (2) the features extracted by CNN contain a large number of redundant and invalid information. To solve these problems, this paper constructs a deformable convolutional neural network to adapt the convolutional sampling positions to the shape of objects in the remote sensing scene. Meanwhile, the spatial and channel attention mechanisms are used to focus on the effective features while suppressing the invalid ones. The experimental results indicate that the proposed method is competitive to the state-of-the-art methods on three remote sensing scene classification datasets (UCM, NWPU, and AID).

Remote sensing scene classification is an important technology, which is widely used in military and civil applications. However, it is still a challenging problem due to the complexity of scene images. Recently, the development of remote sensing satellite and sensor devices has greatly improved the spatial resolution and semantic information of remote sensing images. Therefore, we propose a novel remote sensing scene classification approach to enhance the performance of scene classification. First, a spatial and channel-wise attention module is proposed to adequately utilize the spatial and feature information. Compare with other methods, channel-wise module works on the feature maps with diverse levels and pays more attention to semantic-level features. On the other hand, spatial attention module promotes correlation between foreground and classification result. Second, a novel classifier named broad-classifier is designed to enhance the discriminability. It greatly reduces the cost of computing in the meantime by broad learning system. The experimental results have show that our classification method can effectively improve the average accuracies on remote sensing scene classification data sets.

Despite deep neural networks (DNNs) have been widely applied in remote sensing (RS) scene classification and achieved satisfying performance, the vulnerability of DNNs towards adversarial examples significantly degrades their performance. Moreover, the relatively limited labeled samples of RS scene classification make DNNs more likely to overfit, leading to weak generalizability and noise sensitivity. This may result in DNNs being more vulnerable to adversarial examples. Consequently, the defense of adversarial examples is of crucial importance to improve both the generalizability and robustness of DNNs in the RS scene classification task. However, few studies have been conducted on defense for RS scene classification, especially ignoring the intrinsic characteristics of RS images. In this paper, an effective defense framework for RS scene classification, named reconstruction-assisted and distance-optimized adversarial training (RDAT), is proposed to defend adversarial examples. In order to solve the problems caused by high interclass similarity, a distance-optimized (DO) strategy is designed for adversarial training to strengthen the learning of underfitting content, increase the interclass distance, and improve the robustness of the networks. Furthermore, in order to generate high quality samples for adversarial training, a reconstruction-assisted (RA) block is proposed to eliminate adversarial perturbations in adversarial examples. Specifically, in this block, by swin transformer (SwinT) block and multi-scale convolution (MSC) block, SwinT-MSC-UNet (SMUNet) is constructed to fully extract global and multi-scale local features to adapt to the characteristics of RS images with large variance of ground object scales. Extensive experiments on the benchmark datasets, i.e., UC Merced (UCM) and Aerial Image Dataset (AID), have demonstrate that the proposed RDAT can effectively resist multiple adversarial attacks and yield superior results than other defense methods for RS scene classification.

Storing and processing Remote Sensing (RS) images require large amounts of memory space and computing resources. Consequently, RS images are compressed and stored in various compression formats, such as JPEG2000. However, the processing of RS images for machine interpretation and understanding still necessitates the deployment of an image decompression stage in its entirety, followed by a computationally demanding image analysis pipeline. The image analysis stage is commonly composed of machine learning techniques, such as Deep Convolutional Neural Network (DCNN) models. Classification of remote sensing images is among the most common image analysis tasks. In the scope of this paper, we propose a sub-band image based classification method for the Remote Sensing Scene Classification (RSSC) task in the JPEG2000 compressed domain. The proposed approach exploits the already available sub-band image coefficients to classify RS images without needing for full decompression. Our study shows that our method increases the high frequency information in the LL sub-band and allows the image to contain more detail, leading to improved classifier performance while taking advantage of the partial decompression method.

Accurate classification of remote sensing (RS) images is perennial topic of interest in the RS community. Recently, transfer learning, especially for fine-tuning pre-trained convolutional neural networks (CNNs), has been proposed as a feasible strategy for RS scene classification. However, because the target domain (i.e., the RS images) and the source domain (e.g., ImageNet) are quite different, simply using the model pre-trained on an ImageNet dataset presents some difficulties. The RS images and the pre-trained models need to be properly adjusted to build a better classification system. In this study, an adaptive learning strategy for transferring a CNN-based model is proposed. First, an adaptive transform is used to adjust the original size of the RS image to a certain size, which is tailored to the input of the subsequent pre-trained model. Then, an adaptive transferring model is proposed to automatically learn what knowledge from the pre-trained model should be transferred to the RS scene classification model. Finally, in combination with a label smoothing approach, adaptive label is presented to generate soft labels based on the statistics of the classification model predictions for each category, which is beneficial for learning the relationships between the target and non-target categories of scenes. In general, the proposed methods adaptively manage the input, model, and label simultaneously, which leads to better classification performance for RS scene classification. The proposed methods are tested on three widely-used data sets and the obtained results show that the proposed methods provide competitive classification accuracy compared to the state-of-the-art methods.

This study presents an improved data augmentation technique for the classification of remote sensing (RS) scenes. The method is called Quality-based Sample Selection (QSS) data augmentation technique. It is based on generating a large number of samples using geometric transformations and then selecting the best ones based on a quality criterion. Sample images are generated online, i.e. during training of the convolutional neural network (CNN) model. For each training sample image, a few images are generated randomly using a set of geometric transformations. Then, the generated images are passed to the CNN model being trained, and the predicted probabilities are used to evaluate the quality of the images. The motivation behind this research is that we aim to augment the training set with new images that will benefit the learning of the CNN model. Therefore, an objective method to evaluate the quality of the new images is the CNN model itself. The image with the best criteria score is used to augment the training set. Several quality criteria based on the prediction probabilities of the CNN model are considered including entropy, breaking-ties and our own proposed criteria called True Label Probability (TLP). QSS is tested on five common RS scene classification datasets: UCMerced, Optimal31, RSSCN7, AID and NWPU-RS45. It has outperformed all previous methods except for one case involving NWPU-RS45 dataset. For the 10%–90% train–test split, QSS achieved 94.51, 81.72, 89.89, 94.09 and 93.98, respectively, for the mentioned datasets. While for the 20%–80% train–test split, it achieved 97.09, 88.37, 94.01, 95.71 and 94.71, respectively, for the listed datasets. Thus, this work presents a novel data augmentation method that uses data with higher quality instead of randomly selected. The paper proves experimentally that the proposed method improves the classification accuracy of RS scene classification.

Because of the challenges associated with the difficulty of accurately labeling the remote sensing (RS) scene images and the need to identify new scene classes, few-shot learning has shown significant advantages in addressing the remote sensing scene classification (RSSC) tasks, leading to a growing interest. However, due to the scale variations of targets and irrelevant complex background in scene images, the current few-shot methods exist the following problems: the problem of the extraction capability of feature extractor in the few-shot mechanism; the problem of the separability of few-shot RS scene images classifier. To solve the above problems, an approach, called few-shot RSSC via subspace based on multiscale feature learning is introduced in this work. We first design a multiscale feature learning technique to address scale variations of the targets in the scene images. Concretely, different branches are utilized to learn scene features at various scales. The self-attention mechanism is embedded in each branch to incorporate the understanding of the global information in the different scale features. After that, a multiscale feature fusion operation, incorporating channel attention, will be devised to effectively merge the different scale features, so as to obtain a more precise feature representation of RS scene images. Furthermore, the subspace is utilized to capture the shared characteristics of each category, to reduce the impact of the complex irrelevant backgrounds in the scene images. The results of our experiments conducted on the public available RS scene datasets demonstrate the strong competitiveness of our approach.

Image scene classification in the remotely sensed (RS) society is an interesting subject that aims to allocate land use / cover semantic information. A big amount of Convolutional neural classification models for RS images have been proposed by the authors due to the massive behaviour of CNNs in features extracted. Despite their impressive results, there are still opportunities for advancement. To begin, local characteristics are just as important as global ones in identifying RS images. The CNNs' hierarchical organizational structure and multidimensional suitable capabilities make them good at trying to capture spatial information [1]. It's not uncommon for the feature maps to be overlooked, moreover. First and foremost, the ranges among RS image pairs must be minimised or maximised in order to achieve satisfying classifier performance. Despite this, the importance of these focuses in classification tasks is undervalued. We propose a new CNN called focus high quality infrastructure (ACNet) depending on the Siamese network in the this paper to improve the robustness discussed previous section. [3]To begin, the input data for ACNet are the picture groups achieved through spatial rotation as a result of the network's the double framework.

Remote sensing scene classification (RSSC) is a fundamental but challenging task in the discipline of remote sensing (RS). Significant recent advancements have been achieved in RSSC utilizing convolution neural network. The increasing amount of RS images has resulted in two tough problems: intra-class diversity and inter-class similarity. Existing approaches cannot effectively concentrate on both global and local key features simultaneously. Therefore, in this letter, we propose a dual-branch global-local attention network (DBGA-Net) to solve this problem. Specifically, effective complementary features are first extracted from two distinct models. Following that, it was designed that global-local attention module, which concurrently focuses on both the entire image global features and local pertinent parts within it, would be necessary to successfully extract valid information from RS images. In the end, the fusion module efficiently combines the extracted complementary features to yield more comprehensive scene information. To verify the effectiveness of DBGA-Net, experiments are conducted on three scene classification datasets, 99.86%, 97.60% and 96.06% experimental results are obtained, respectively, with significant classification performance of the proposed method compared with the state-of-the-art (SOTA) method. Our code and images are available at https://github.com/ZhouYao1020/DBGANet.

Learning powerful discriminative features is the key for remote sensing scene classification. Most existing approaches based on convolutional neural network (CNN) have achieved great results. However, they mainly focus on global-based visual features while ignoring object-based location features, which is important for large-scale scene classification. There are a large number of scene-related ground objects in remote sensing images, as well as Graph convolutional network (GCN) has the potential to capture the dependencies among objects. This article introduces a novel two-stream architecture that combines global-based visual features and object-based location features, so as to improve the feature representation capability. First, we extract appearance visual features from whole scene image based on CNN. Second, we detect ground objects and construct a graph to learn the spatial location features based on GCN. As a result, the network can jointly capture appearance visual information and spatial location information. To the best of authors' knowledge, we are the first to investigate the dependencies among objects in remote sensing scene classification task. Extensive experiments on two datasets show that our framework improves the discriminative ability of features and achieves competitive accuracy against other state-of-the-art approaches.

With the popular use of high-resolution satellite images, remote sensing scene classification has always been a hot research topic in its related areas. However, limited to the issues of remote sensing datasets including the small scale of scene classes, the lack of rich label information and so on, it is quite challenging for deep learning methods to learn powerful feature representation. To overcome this problem, we propose a rotation-invariant feature learning and joint decision-making method based on Siamese convolutional neural networks with the combination of identification and verification models. Firstly, a novel data augmentation strategy is proposed specially for the Siamese model to learning rotation-invariant features. Secondly, a joint decision mechanism is introduced in our method, which is realized by the identification and verification model to better improve the classification performance. The proposed method can not only suppress problems caused by lack of rich label samples but also improve the robustness of Siamese convolutional neural networks. Experimental results demonstrate that the proposed method is effective and efficient for remote sensing scene classification.

Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.

Convolutional Neural Networks have been widely used in remote sensing scene classification. Since this kind of model needs a large number of training samples containing data category information, a Generative Adversarial Network (GAN) is usually used to address the problem of lack of samples. However, GAN mainly generates scene data samples that do not contain category information. To address this problem, a novel supervised adversarial Lie Group feature learning network is proposed. In the case of limited data samples, the model can effectively generate data samples with category information. There are two main differences between our method and the traditional GAN. First, our model takes category information and data samples as the input of the model and optimizes the constraint of category information in the loss function, so that data samples containing category information can be generated. Secondly, the object scale sample generation strategy is introduced, which can generate data samples of different scales and ensure that the generated data samples contain richer feature information. After large-scale experiments on two publicly available and challenging datasets, it is found that our method can achieve better scene classification accuracy even with limited data samples.

The fusion of convolutional neural networks (CNNs) and vision transformers (ViTs) is a powerful strategy for remote sensing scene classification (RSSC), leveraging their complementary strengths in local texture and global context modeling. To fully exploit these complementary strengths for addressing the inter-class similarity and high intra-class variability caused by the complex spatial arrangements of RS scenes, this work introduces the hierarchically aligned bidirectional CNN-Transformer network (HAB-Net), which fosters deeper, two-way integration through two novel components. First, Hierarchical Attention Gating (HAG) distills multi-level attention from the ViT into spatial gates, compelling the CNN to focus on semantically critical regions. In turn, CNN-driven Feedback Attention (CFA) completes the loop by feeding these enhanced, detail-rich CNN features back into the ViT, refining its tokens via cross-attention. Ultimately, a dual-stream head merges these mutually reinforced features for scene classification. Extensive experiments on the UCM, AID, and NWPU-RESISC45 benchmarks demonstrate that HAB-Net achieves state-of-the-art performance. Ablation studies confirm the indispensable contributions of both HAG and CFA, validating the effectiveness of the proposed deep bidirectional integration strategy.

Recent advancements in deep learning have significantly improved the performance of remote sensing scene classification, a critical task in remote sensing applications. This study presents a new aerial scene classification model, the Multi-Scale Swin–CNN Aerial Classifier (MSCAC), which employs the Swin Transformer, an advanced architecture that has demonstrated exceptional performance in a range of computer vision applications. The Swin Transformer leverages shifted window mechanisms to efficiently model long-range dependencies and local features in images, making it particularly suitable for the complex and varied textures in aerial imagery. The model is designed to capture intricate spatial hierarchies and diverse scene characteristics at multiple scales. A framework is developed that integrates the Swin Transformer with a multi-scale strategy, enabling the extraction of robust features from aerial images of different resolutions and contexts. This approach allows the model to effectively learn from both global structures and fine-grained details, which is crucial for accurate scene classification. The model’s performance is evaluated on several benchmark datasets, including UC-Merced, WHU-RS19, RSSCN7, and AID, where it demonstrates a superior or comparable accuracy to state-of-the-art models. The MSCAC model’s adaptability to varying amounts of training data and its ability to improve with increased data make it a promising tool for real-world remote sensing applications. This study highlights the potential of integrating advanced deep-learning architectures like the Swin Transformer into aerial scene classification, paving the way for more sophisticated and accurate remote sensing systems. The findings suggest that the proposed model has significant potential for various remote sensing applications, including land cover mapping, urban planning, and environmental monitoring.