Detecting and Mitigating Adversarial Examples in Neural Networks: An Enhanced PGD Approach

Malware is constantly adapting in order to avoid detection. Model-based malware detectors, such as SVM and neural networks, are vulnerable to so-called adversarial examples which are modest changes to detectable malware that allows the resulting malware to evade detection. Continuous-valued methods that are robust to adversarial examples of images have been developed using saddle-point optimization formulations. We are inspired by them to develop similar methods for the discrete, e.g. binary, domain which characterizes the features of malware. A specific extra challenge of malware is that the adversarial examples must be generated in a way that preserves their malicious functionality. We introduce methods capable of generating functionally preserved adversarial malware examples in the binary domain. Using the saddle-point formulation, we incorporate the adversarial examples into the training of models that are robust to them. We evaluate the effectiveness of the methods and others in the literature on a set of Portable Execution (PE) files. Comparison prompts our introduction of an online measure computed during training to assess general expectation of robustness.

Many companies offer automatic speech recognition or Speech-to-Text APIs for use in diverse applications. However, audio classification algorithms trained with deep neural networks (DNNs) can sometimes misclassify adversarial examples, posing a significant threat to critical applications. In this paper, we present a novel way to create adversarial audio examples using a genetic algorithm. Our algorithm creates adversarial examples by iteratively adding perturbations to the original audio signal. Unlike most white-box adversarial example generations, our approach does not require knowledge about the target DNN's model and parameters (black-box) and heavy computational power of GPU resources. To show the feasibility of the proposed idea, we implement a tool called, Foolgle, using a genetic algorithm that performs untargeted attacks to create adversarial audio examples and evaluate those with the state-of-the-art Google Cloud Speech-to-Text API. Our preliminary experiment results show that Foolgle deceives the API with a success probability of 86%.

The automatic speech recognition (ASR) system based on deep neural network is easy to be attacked by an adversarial example due to the vulnerability of neural network, which is a hot topic in recent years. The adversarial example does harm to the ASR system, especially if the common-dependent ASR goes wrong, it will lead to serious consequences. To improve the robustness and security of the ASR system, the defense method against adversarial examples must be proposed. Based on this idea, we propose an algorithm of devastation and detection on adversarial examples which can attack the current advanced ASR system. We choose advanced text-dependent and command-dependent ASR system as our target system. Generating adversarial examples by the OPT on text-dependent ASR and the GA-based algorithm on command-dependent ASR. The main idea of our method is input transformation of the adversarial examples. Different random intensities and kinds of noise are added to the adversarial examples to devastate the perturbation previously added to the normal examples. From the experimental results, the method performs well. For the devastation of examples, the original speech similarity before and after adding noise can reach 99.68%, the similarity of the adversarial examples can reach 0%, and the detection rate of the adversarial examples can reach 94%.

Neural networks provide excellent service on recognition tasks such as image recognition and speech recognition as well as for pattern analysis and other tasks in fields related to artificial intelligence. However, neural networks are vulnerable to adversarial examples. An adversarial example is a sample that is designed to be misclassified by a target model, although it poses no problem for recognition by humans, that is created by applying a minimal perturbation to a legitimate sample. Because the perturbation applied to the legitimate sample to create an adversarial example is optimized, the classification score for the target class has the characteristic of being similar to that for the legitimate class. This regularity occurs because minimal perturbations are applied only until the classification score for the target class is slightly higher than that for the legitimate class. Given the existence of this regularity in the classification scores, it is easy to detect an optimized adversarial example by looking for this pattern. However, the existing methods for generating optimized adversarial examples do not consider their weakness of allowing detectability by recognizing the pattern in the classification scores. To address this weakness, we propose an optimized adversarial example generation method in which the weakness due to the classification score pattern is removed. In the proposed method, a minimal perturbation is applied to a legitimate sample such that the classification score for the legitimate class is less than that for some of the other classes, and an optimized adversarial example is created with the pattern vulnerability removed. The results show that using 500 iterations, the proposed method can generate an optimized adversarial example that has a 100% attack success rate, with distortions of 2.81 and 2.23 for MNIST and Fashion-MNIST, respectively.

As is known to all, deep neural networks have shown superior performance in many image classification tasks, but it is vulnerable to attack by adversarial examples, which are added tiny perturbations on the original examples to fool deep neural networks. Many previous defenses focused on mdifying model structure to enhance the robustness of the neural networks. In this paper, an image squeezing method proposed can help to classify images correctly, before inputing images to deep neural networks, image webp squeezng can destory the perturbation of adversarial examples. Firstly, it is necessary to generate corresponding adversarial examples of the neural networks, which can completely fool classifier, then apply webp squeezing to adversarial examples, and put them into the classifier to regain the correct image classification results. Our experiments on CIFAR-10 dataset demonstrated that webp image squeezing can help neural networsk classify adversarial examples correctly. And our method proposed in this paper can effectively resist FGSM, One-pixel attack and C&W attack, and its recognition accuracy is improved by more than 5% compared with the HGD method.

Feature-filter: Detecting adversarial examples by filtering out recessive features

State-of-the-art Deep Neural Networks (DNNs) are promoting medical image processing. However, DNNs are susceptible to adversarial attacks, which could significantly deteriorate model performances and pose a threat to clinical diagnoses. One common method for prevention is through adversarial training, which is highly dependent on harnessing adversarial examples during the training stage. However, adversarial medical examples generated by many existing works are too perceptible to be adversarial examples. To improve the imperceptibility, we proposed a Regional Saliency Map Attack that generates an adversarial example by only perturbing a small number of pixels. Extensive experiments have shown that, on average, our method caused the same degradation in model performance by quantitatively less perceptible perturbations. Visualisations have also verified that the improvement in imperceptibility in an image is both global and regional.

Natural language processing models based on neural networks are vulnerable to adversarial examples. These adversarial examples are imperceptible to human readers but can mislead models to make the wrong predictions. In a black-box setting, attacker can fool the model without knowing model’s parameters and architecture. Previous works on word-level attacks widely use single semantic space and greedy search as a search strategy. However, these methods fail to balance the attack success rate, quality of adversarial examples and time consumption. In this paper, we propose BeamAttack, a textual attack algorithm that makes use of mixed semantic spaces and improved beam search to craft high-quality adversarial examples. Extensive experiments demonstrate that BeamAttack can improve attack success rate while saving numerous queries and time, e.g., improving at most 7% attack success rate than greedy search when attacking the examples from MR dataset. Compared with heuristic search, BeamAttack can save at most 85% model queries and achieve a competitive attack success rate. The adversarial examples crafted by BeamAttack are highly transferable and can effectively improve model’s robustness during adversarial training. Code is available at https://github.com/zhuhai-ustc/beamattack/tree/master

The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs which mislead a classifier to incorrect outputs in high confidence. Limited by the extreme difficulty in examining a high-dimensional image space thoroughly, research on explaining and justifying the causes of adversarial examples falls behind studies on attacks and defenses. In this paper, we present a collection of potential causes of adversarial examples and verify (or partially verify) them through carefully-designed controlled experiments. The major causes of adversarial examples include model linearity, one-sum constraint, and geometry of the categories. To control the effect of those causes, multiple techniques are applied such as L2 normalization, replacement of loss functions, construction of reference datasets, and novel models using multi-layer perceptron probabilistic neural networks (MLP-PNN) and density estimation (DE). Our experiment results show that geometric factors tend to be more direct causes and statistical factors magnify the phenomenon, especially for assigning high prediction confidence. We believe this paper will inspire more studies to rigorously investigate the root causes of adversarial examples, which in turn provide useful guidance on designing more robust models.

Remote sensing image (RSI) scene classification is the foundation and important technology of ground object detection, land use management and geographic analysis. During recent years, convolutional neural networks (CNNs) have achieved significant success and are widely applied in RSI scene classification. However, crafted images that serve as adversarial examples can potentially fool CNNs with high confidence and are hard for human eyes to interpret. For the increasing security and robust requirements of RSI scene classification, the adversarial example problem poses a serious problem for the classification results derived from systems using CNN models, which has not been fully recognized by previous research. In this study, to explore the properties of adversarial examples of RSI scene classification, we create different scenarios by testing two major attack algorithms (i.e., the fast gradient sign method (FGSM) and basic iterative method (BIM)) trained on different RSI benchmark datasets to fool CNNs (i.e., InceptionV1, ResNet and a simple CNN). In the experiment, our results show that CNNs of RSI scene classification are also vulnerable to adversarial examples, and some of them have a fooling rate of over 80%. These adversarial examples are affected by the architecture of CNNs and the type of RSI dataset. InceptionV1 has a fooling rate of less than 5%, which is lower than the others. Adversarial examples generated on the UCM dataset are easier than other datasets. Importantly, we also find that the classes of adversarial examples have an attack selectivity property. Misclassifications of adversarial examples of RSIs are related to the similarity of the original classes in the CNN feature space. Attack selectivity reveals potential classes of adversarial examples and provides insights into the design of defensive algorithms in future research.

Generating adversarial examples via enhancing latent spatial features of benign traffic and preserving malicious functions

Convolutional neural networks (CNNs) have shown vulnerable to adversarial examples which contain imperceptible perturbations. In this paper, we propose an approach to defend against adversarial examples with soft decision trees embedding. Firstly, we extract the semantic features of adversarial examples with a feature extraction network. Then, a specific soft decision tree is trained and embedded to select the key semantic features for each feature map from convolutional layers and the selected features are fed to a light-weight classification network. To this end, we use the probability distributions of each tree node to quantify the semantic features. In this way, some small perturbations can be effectively removed and the selected features are more discriminative in identifying adversarial examples. Moreover, the influence of adversarial perturbations on classification can be reduced by migrating the interpretability of soft decision trees into the black-box neural networks. We conduct experiments to defend the state-of-the-art adversarial attacks. The experimental results demonstrate that our proposed approach can effectively defend against these attacks and improve the robustness of deep neural networks.

Deep Neural Networks (DNNs) prove to be susceptible to synthetically generated samples, so-called adversarial examples. Such adversarial examples aim at generating misclassifications by specifically optimizing input data for a matching perturbation. With the increasing use of deep learning on embedded devices and the resulting use of quantization techniques to compress deep neural networks, it is critical to investigate the adversarial vulnerability of quantized neural networks.In this paper, we perform an in-depth study of the adversarial robustness of quantized networks against direct attacks, where adversarial examples are both generated and applied on the same network. Our experiments show that quantization makes models resilient to the generation of adversarial examples, even for attacks that demonstrate a high success rate, indicating that it offers some degree of robustness against these attacks. Additionally, we open-source Adversarial Neural Network Toolkit (ANNT) to support the replication of our results.

The presence of adversarial examples in the physical world poses significant challenges for deploying Deep Neural Networks (DNNs) in safety-critical applications such as autonomous driving, where even minor misclassifications can lead to catastrophic consequences. Most existing methods for crafting physical-world adversarial examples are ad-hoc and deliberately designed, relying on temporary modifications like shadows, laser beams, or stickers that are tailored to specific scenarios. In this paper, we introduce a new class of physical-world adversarial examples, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i>, which draws inspiration from the naturally occurring phenomenon of ‘wear and tear’, an inherent property of physical objects. Unlike manually crafted perturbations, ‘wear and tear’ emerges organically over time due to environmental factors, as seen in the gradual deterioration of outdoor signboards. To achieve this, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i> follows a two-step approach. First, a GAN-based, unsupervised image-to-image translation network is employed to model these naturally occurring damages, particularly in the context of outdoor signboards. The translation network encodes the characteristics of damaged signs into a latent ‘damage style code’. In the second step, we introduce adversarial perturbations into the style code, strategically optimizing its transformation process. This manipulation subtly alters the damage style representation, guiding the network to generate adversarial images where the appearance of damages remains perceptually realistic, while simultaneously ensuring their effectiveness in misleading neural networks. Through comprehensive experiments on two traffic sign datasets, we show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i> effectively misleads DNNs in both digital and physical domains. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i> achieves an effective attack success rate, transferability, robustness, and a more natural appearance compared to existing physical-world adversarial examples. Additionally, integrating <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i> into training enhances a model’s generalizability to real-world damaged signs. We have released the code for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AdvWT</i> at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/samra-irshad/AdvWT</uri>.

Convolutional neural networks have been applied in image recognition widely, and acquired well-known results. However, the discovery of adversarial examples in recent years has attracted many researcher’s attention on the safety and reliability of convolutional neural networks. Adversarial examples are small perturbations added only to the clean image data, which could cause the trained model makes wrong identifications, while people could still recognize. In this paper, we propose a new convolutional neural network to defense the adversarial examples. It breaks into two branches after inputting an image. One branch likes a normal image recognition model, while another branch distills the low-frequency information of the image firstly, and then uses the low-frequency image to train. The two branches combine to make the output of the model. The presented model uses the image’s low-frequency information, through which people often recognizes an object. The experiments show that the presented model suppresses the success rate of the attack in the face of the adversarial examples, and improves the model’s recognition success rate, comparing to some other methods.KeywordsAdversarial examplesDefense methodImage low-frequency information