Reconstruction-Assisted and Distance-Optimized Adversarial Training: A Defense Framework for Remote Sensing Scene Classification

Deep neural networks, which can learn the representative and discriminative features from data in a hierarchical manner, have achieved state-of-the-art performance in the remote sensing scene classification task. Despite the great success that deep learning algorithms have obtained, their vulnerability toward adversarial examples deserves our special attention. In this article, we systematically analyze the threat of adversarial examples on deep neural networks for remote sensing scene classification. Both targeted and untargeted attacks are performed to generate subtle adversarial perturbations, which are imperceptible to a human observer but may easily fool the deep learning models. Simply adding these perturbations to the original high-resolution remote sensing (HRRS) images, adversarial examples can be generated, and there are only slight differences between the adversarial examples and the original ones. An intriguing discovery in our study shows that most of these adversarial examples may be misclassified into the wrong category by the state-of-the-art deep neural networks with very high confidence. This phenomenon, undoubtedly, may limit the practical deployment of these deep learning models in the safety-critical remote sensing field. To address this problem, the adversarial training strategy is further investigated in this article, which significantly increases the resistibility of deep models toward adversarial examples. Extensive experiments on three benchmark HRRS image data sets demonstrate that while most of the well-known deep neural networks are sensitive to adversarial perturbations, the adversarial training strategy helps to alleviate their vulnerability toward adversarial examples.

Remote sensing image (RSI) scene classification is the foundation and important technology of ground object detection, land use management and geographic analysis. During recent years, convolutional neural networks (CNNs) have achieved significant success and are widely applied in RSI scene classification. However, crafted images that serve as adversarial examples can potentially fool CNNs with high confidence and are hard for human eyes to interpret. For the increasing security and robust requirements of RSI scene classification, the adversarial example problem poses a serious problem for the classification results derived from systems using CNN models, which has not been fully recognized by previous research. In this study, to explore the properties of adversarial examples of RSI scene classification, we create different scenarios by testing two major attack algorithms (i.e., the fast gradient sign method (FGSM) and basic iterative method (BIM)) trained on different RSI benchmark datasets to fool CNNs (i.e., InceptionV1, ResNet and a simple CNN). In the experiment, our results show that CNNs of RSI scene classification are also vulnerable to adversarial examples, and some of them have a fooling rate of over 80%. These adversarial examples are affected by the architecture of CNNs and the type of RSI dataset. InceptionV1 has a fooling rate of less than 5%, which is lower than the others. Adversarial examples generated on the UCM dataset are easier than other datasets. Importantly, we also find that the classes of adversarial examples have an attack selectivity property. Misclassifications of adversarial examples of RSIs are related to the similarity of the original classes in the CNN feature space. Attack selectivity reveals potential classes of adversarial examples and provides insights into the design of defensive algorithms in future research.

Deep neural networks (DNNs), which learn a hierarchical representation of features, have shown remarkable performance in big data analytics of remote sensing. However, previous research indicates that DNNs are easily spoofed by adversarial examples, which are crafted images with artificial perturbations that fool DNN models toward wrong predictions. To comprehensively evaluate the impact of adversarial examples on the remote sensing image (RSI) scene classification, this study tests eight state-of-the-art classification DNNs on six RSI benchmarks. These data sets include both optical and synthetic-aperture radar (SAR) images of different spectral and spatial resolutions. In the experiment, we create 48 classification scenarios and use four cutting-edge attack algorithms to investigate the influence of the adversarial example on the classification of RSIs. The experimental result shows that the fooling rates of the attacks are all over 98% across the 48 scenarios. We also find that, for the optical data, the seriousness of the adversarial problem has a negative relationship with the richness of the feature information. Besides, adversarial examples generated from SAR images are used easily for fooling the models with an average fooling rate of 76.01%. By analyzing the class distribution of these adversarial examples, we find that the distribution of the misclassifications is not affected by the types of models and attack algorithms-adversarial examples of RSIs of the same class cluster on fixed several classes. The analysis of classes of adversarial examples not only helps us explore the relationships between data set classes but also provides insights for further designing defensive algorithms.

State-of-the-art Deep Neural Networks (DNNs) are promoting medical image processing. However, DNNs are susceptible to adversarial attacks, which could significantly deteriorate model performances and pose a threat to clinical diagnoses. One common method for prevention is through adversarial training, which is highly dependent on harnessing adversarial examples during the training stage. However, adversarial medical examples generated by many existing works are too perceptible to be adversarial examples. To improve the imperceptibility, we proposed a Regional Saliency Map Attack that generates an adversarial example by only perturbing a small number of pixels. Extensive experiments have shown that, on average, our method caused the same degradation in model performance by quantitatively less perceptible perturbations. Visualisations have also verified that the improvement in imperceptibility in an image is both global and regional.

With the rapid development of Deep Neural Networks (DNNs), remote sensing image analysis has achieved significant progress in scene classification, object detection, and semantic segmentation. However, the increasing deployment of DNNs in real-world remote sensing applications exposes them to adversarial risks. A critical challenge in this domain is the poor visual stealth of black-box adversarial examples, perturbations are often easily perceived by humans, limiting their practicality. This issue stems from the distribution mismatch between visually salient regions and model-sensitive regions in remote sensing images. The current approaches predominantly focus on improving attack success rates while neglecting such heterogeneity, resulting in redundant and conspicuous perturbations. To address this, we propose a Dual-Region Hierarchical Attack (DRHA) that improves stealth and attack performance by using the different roles of visually salient and model-sensitive regions. Shallow perturbations are applied to salient areas to preserve perceptual similarity, while directional perturbations guided by integrated gradients target high-contribution regions to enhance model deception. Our method leverages superpixel segmentation and attribution analysis to localize perturbation regions precisely. Experiments on the UCM and AID datasets show that DRHA improves average success rates by 10%–32% over baseline methods, while maintaining superior stealth and perturbation sparsity. These results demonstrate the effectiveness of region-aware attack design in constructing imperceptible and transferable adversarial examples for remote sensing tasks.

Deep neural networks (DNNs) have gained widespread adoption in computer vision. Unfortunately, state-of-the-art DNNs are vulnerable to adversarial example (AE) attacks, where an adversary introduces imperceptible perturbations to a test example for defrauding DNNs. The obstacles have urged intensive research on improving the DNN robustness via adversarial training, that is, the clean data set is blended with adversarial examples to carry out training. However, the adversarial example attack technologies are open-ended, and the adversarial training is insufficient to focus on improving robustness performance. To circumvent this limitation, we mitigate adversarial example attacks from another perspective, which aims at detecting adversarial examples. Feature autoencoder detector (FADetector), a novel defense framework that exploits feature knowledge is proposed. One of the hallmarks of FADetector is to not involve adversarial examples to train the detector. Our extensive evaluation on MNIST and CIFAR-10 data sets demonstrates that our defense outperforms the conventional autoencoder detectors in terms of detection accuracy.

The methods for remote sensing image (RSI) scene classification based on deep convolutional neural networks (DCNNs) have achieved prominent success. However, confronted with adversarial examples obtained by adding imperceptible perturbations to clean images, the great vulnerability of DCNNs makes it worth exploring effective defense methods. To date, numerous countermeasures for adversarial examples have been proposed, but how to improve the defensive ability for unknown attacks still to be answered. To address this issue, in this article, we propose an effective defense framework specified for RSI scene classification, named perturbation-seeking generative adversarial networks (PSGANs). In brief, a new training framework is designed to train the classifier by introducing the examples generated during the image reconstruction process, in addition to clean examples and adversarial ones. These generated examples can be random kinds of unknown attacks during training and thus are utilized to eliminate the blind spots of a classifier. To assist the proposed training framework, a reconstruction method is developed. First, instead of modeling the distribution of clean examples, we model the distributions of the perturbations added in adversarial examples. Second, to make a tradeoff between the diversity of the reconstructed examples and the optimization of PSGAN, a scale factor named seeking radius is introduced to scale the generated perturbations before they are subtracted by the given adversarial examples. Comprehensive and extensive experimental results on three widely used benchmarks for RSI scene classification demonstrate the great effectiveness of PSGAN when faced with both known and unknown attacks. Our source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xuxiangsun/PSGAN</uri> .

Remote sensing scene classification is an important technology, which is widely used in military and civil applications. However, it is still a challenging problem due to the complexity of scene images. Recently, the development of remote sensing satellite and sensor devices has greatly improved the spatial resolution and semantic information of remote sensing images. Therefore, we propose a novel remote sensing scene classification approach to enhance the performance of scene classification. First, a spatial and channel-wise attention module is proposed to adequately utilize the spatial and feature information. Compare with other methods, channel-wise module works on the feature maps with diverse levels and pays more attention to semantic-level features. On the other hand, spatial attention module promotes correlation between foreground and classification result. Second, a novel classifier named broad-classifier is designed to enhance the discriminability. It greatly reduces the cost of computing in the meantime by broad learning system. The experimental results have show that our classification method can effectively improve the average accuracies on remote sensing scene classification data sets.

Adversarial examples are crucial tools for assessing the robustness of deep neural networks (DNNs) and revealing potential security vulnerabilities. Adversarial example generation methods based on Generative Adversarial Networks (GANs) have made significant progress in generating image adversarial examples, but still suffer from insufficient sparsity and transferability. To address these issues, this study proposes a novel semi-white-box untargeted adversarial example generation method named Wavelet-AdvGAN, with an explicit threat model defined as follows. Specifically, the attack is strictly untargeted without predefined target categories, aiming solely to mislead DNNs into classifying adversarial examples into any category other than the original label. It adopts a semi-white-box setting where attackers are denied access to the target model’s private information. Regarding the generator’s information dependence, the training phase only utilizes public resources (i.e., the target model’s public architecture and CIFAR-10 public training data), while the test phase generates adversarial examples through one-step feedforward of clean images without interacting with the target model. The method incorporates a Frequency Sub-band Difference (FSD) module and a Wavelet Transform Local Feature (WTLF) extraction module, evaluating the differences between original and adversarial examples from the frequency domain perspective. This approach constrains the magnitude of perturbations, reinforces feature regions, and further enhances the attack effectiveness, thereby improving the sparsity and transferability of adversarial examples. Experimental results demonstrate that the Wavelet-AdvGAN method achieves an average increase of 1.26% in attack success rates under two defense strategies—data augmentation and adversarial training. Additionally, the adversarial transferability improves by an average of 2.7%. Moreover, the proposed method exhibits a lower l0 norm, indicating better perturbation sparsity. Consequently, it effectively evaluates the robustness of deep neural networks.

State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.

Many companies offer automatic speech recognition or Speech-to-Text APIs for use in diverse applications. However, audio classification algorithms trained with deep neural networks (DNNs) can sometimes misclassify adversarial examples, posing a significant threat to critical applications. In this paper, we present a novel way to create adversarial audio examples using a genetic algorithm. Our algorithm creates adversarial examples by iteratively adding perturbations to the original audio signal. Unlike most white-box adversarial example generations, our approach does not require knowledge about the target DNN's model and parameters (black-box) and heavy computational power of GPU resources. To show the feasibility of the proposed idea, we implement a tool called, Foolgle, using a genetic algorithm that performs untargeted attacks to create adversarial audio examples and evaluate those with the state-of-the-art Google Cloud Speech-to-Text API. Our preliminary experiment results show that Foolgle deceives the API with a success probability of 86%.

Deep neural networks have shown outstanding performance in various areas, but adversarial examples can easily fool them. Although strong adversarial attacks have defeated diverse adversarial defense methods, adversarial training, which augments training data with adversarial examples, remains an effective defense strategy. To further improve adversarial robustness, this paper exploits adversarial examples of adversarial examples. We observe that these doubly adversarial examples tend to return to the original prediction on the clean images but sometimes drift toward other classes. From this finding, we propose a regularization loss that prevents these drifts, which mitigates the vulnerability against multi-targeted attacks. Experimental results on the CIFAR-10 and CIFAR-100 datasets empirically show that the proposed loss improves adversarial robustness.

A survey of robust adversarial training in pattern recognition: Fundamental, theory, and methodologies

Storing and processing Remote Sensing (RS) images require large amounts of memory space and computing resources. Consequently, RS images are compressed and stored in various compression formats, such as JPEG2000. However, the processing of RS images for machine interpretation and understanding still necessitates the deployment of an image decompression stage in its entirety, followed by a computationally demanding image analysis pipeline. The image analysis stage is commonly composed of machine learning techniques, such as Deep Convolutional Neural Network (DCNN) models. Classification of remote sensing images is among the most common image analysis tasks. In the scope of this paper, we propose a sub-band image based classification method for the Remote Sensing Scene Classification (RSSC) task in the JPEG2000 compressed domain. The proposed approach exploits the already available sub-band image coefficients to classify RS images without needing for full decompression. Our study shows that our method increases the high frequency information in the LL sub-band and allows the image to contain more detail, leading to improved classifier performance while taking advantage of the partial decompression method.

Although Deep Neural Networks (DNNs) have achieved great success on various applications, investigations have increasingly shown DNNs to be highly vulnerable when adversarial examples are used as input. Here, we present a comprehensive defense framework to protect DNNs against adversarial examples. First, we present statistical and minor alteration detectors to filter out adversarial examples contaminated by noticeable and unnoticeable perturbations, respectively. Then, we ensemble the detectors, a deep Residual Generative Network (ResGN), and an adversarially trained targeted network, to construct a complete defense framework. In this framework, the ResGN is our previously proposed network which is used to remove adversarial perturbations, and the adversarially trained targeted network is a network that is learned through adversarial training. Specifically, once the detectors determine an input example to be adversarial, it is cleaned by ResGN and then classified by the adversarially trained targeted network; otherwise, it is directly classified by this network. We empirically evaluate the proposed complete defense on ImageNet dataset. The results confirm the robustness against current representative attacking methods including fast gradient sign method, randomized fast gradient sign method, basic iterative method, universal adversarial perturbations, DeepFool method, and Carlini &amp; Wagner method.