From Image to Code

With intentional feature perturbations to a deep learning model, the adversary generates an adversarial example to deceive the deep learning model. As an adversarial example has recently been considered in the most severe problem of deep learning technology, its defense methods have been actively studied. Such effective defense methods against adversarial examples are categorized into one of the three architectures: (1) model retraining architecture; (2) input transformation architecture; and (3) adversarial example detection architecture. Especially, defense methods using adversarial example detection architecture have been actively studied. This is because defense methods using adversarial example detection architecture do not make wrong decisions for the legitimate input data while others do. In this paper, we note that current defense methods using adversarial example detection architecture can classify the input data into only either a legitimate one or an adversarial one. That is, the current defense methods using adversarial example detection architecture can only detect the adversarial examples and cannot classify the input data into multiple classes of data, i.e., legitimate input data and various types of adversarial examples. To classify the input data into multiple classes of data while increasing the accuracy of the clustering model, we propose an advanced defense method using adversarial example detection architecture, which extracts the key features from the input data and feeds the extracted features into a clustering model. From the experimental results under various application datasets, we show that the proposed method can detect the adversarial examples while classifying the types of adversarial examples. We also show that the accuracy of the proposed method outperforms the accuracy of recent defense methods using adversarial example detection architecture.

Air transportation communication jamming recognition model based on deep learning (DL) can quickly and accurately identify and classify communication jamming, to improve the safety and reliability of air traffic. However, due to the vulnerability of deep learning, the jamming recognition model can be easily attacked by the attacker’s carefully designed adversarial examples. Although some defense methods have been proposed, they have strong pertinence to attacks. Thus, new attack methods are needed to improve the defense performance of the model. In this work, we improve the existing attack methods and propose a double level attack method. By constructing the dynamic iterative step size and analyzing the class characteristics of the signals, this method can use the adversarial losses of feature layer and decision layer to generate adversarial examples with stronger attack performance. In order to improve the robustness of the recognition model, we use adversarial examples to train the model, and transfer the knowledge learned from the model to the jamming recognition models in other wireless communication environments by transfer learning. Simulation results show that the proposed attack and defense methods have good performance.

State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.

Automatic diagnosis based on clinical notes has become a popular research field recently, and many proposed deep learning models have achieved competitive performance in diseases inference. However, previous research reveals that deep learning models are susceptible to negligibly perturbed inputs named adversarial examples, which contradicts with the safety and reliability requirements of the medical domain. To analyze the vulnerability and robustness of current automatic diagnosis models, we investigate in the generation of adversarial text examples. The main challenges for generating adversarial text examples are divided into three parts. First, the word embedding space is discrete, which makes it hard to perturb as small as adversarial image examples generation. Second, previous adversarial example generation methods focus mainly on multi-class classification models, while automatic diagnosis is a multi-label classification task. Third, the semantic and medical meaning of clinical notes are vital in disease inference, and even small perturbations can change them to a large extent. In this paper, we address the three main challenges and propose Clinical-Attacker, a general framework for both white-box and black-box adversarial text examples generation against automatic diagnosis models. Experimental results on MIMIC-III dataset demonstrate that our framework can easily alter the predictions of automatic diagnosis models with the semantic and medical meaning preserved.

Recent studies have shown that deep learning models such as image classification and object detection models are vulnerable to attacks from adversarial examples. These malicious examples may trigger potential security threats and cause damage in security-related fields. In this paper, in order to find model robustness weakness, we obtain the adversarial robustness metric against several adversarial attacks, including gradient-based attacks, optimization-based attacks, etc. The metric quantizes the adversarial robustness of deep learning models and can be used as a guide for extra model training. Meanwhile, we design a testing platform to achieve model adversarial robustness evaluation. Utilizing the front and back end separation strategy, the testing platform consist of 2 modules: one is multiple adversarial example attacks executor and the other is HCI module. In addition, the tool allows parallel comparison among different adversarial attack algorithms under given conditions and measures the quality of generated adversarial examples utilizing FID(Fréchet Inception Distance) metrics. Besides, we developed ERFGSM(edge-RFGSM), a new gradient-based attack using edge information through canny operator.

Deep learning models have shown their great capability for the hyperspectral image (HSI) classification task in recent years. Nevertheless, their vulnerability towards adversarial attacks could not be neglected. In this study, we systematically analyze the influence of adversarial attacks on the HSI classification task for the first time. While existing research of adversarial attacks focuses on the generation of adversarial examples in the RGB domain, the experiments in this study show such adversarial examples could also exist in the hyperspectral domain. Although the difference between the generated adversarial image and the original hyperspectral data is imperceptible to the human visual system, most of the existing state-of-the-art deep learning models could be fooled by the adversarial image to make wrong predictions. To address this challenge, a novel self-attention context network (SACNet) is further proposed. We discover that the global context information contained in HSI can significantly improve the robustness of deep neural networks when confronted with adversarial attacks. Extensive experiments on three benchmark HSI datasets demonstrate that the proposed SACNet possesses stronger resistibility towards adversarial examples compared with the existing state-of-the-art deep learning models.

Deep neural network-based synthetic aperture radar (SAR) automatic target recognition (ATR) systems are susceptible to attack by adversarial examples, which leads to misclassification by the SAR ATR system, resulting in theoretical model robustness problems and security problems in practice. Inspired by optical images, current SAR ATR adversarial example generation is performed in the image domain. However, the imaging principle of SAR images is based on the imaging of the echo signals interacting between the SAR and objects. Generating adversarial examples only in the image domain cannot change the physical world to achieve adversarial attacks. To solve these problems, this article proposes a framework for generating SAR adversarial examples in a 3D physical scene. First, adversarial attacks are implemented in the 2D image space, and the perturbation in the image space is converted into simulated rays that constitute SAR images through backpropagation optimization methods. The mapping between the simulated rays constituting SAR images and the 3D model is established through coordinate transformation, and point correspondence to triangular faces and intensity values to texture parameters are established. Thus, the simulated rays constituting SAR images are mapped to the 3D model, and the perturbation in the 2D image space is converted back to the 3D physical space to obtain the position and intensity of the perturbation in the 3D physical space, thereby achieving physical adversarial attacks. The experimental results show that our attack method can effectively perform SAR adversarial attacks in the physical world. In the digital world, we achieved an average fooling rate of up to 99.02% for three objects in six classification networks. In the physical world, we achieved an average fooling rate of up to 97.87% for these objects, with a certain degree of transferability across the six different network architectures. To the best of our knowledge, this is the first work to implement physical attacks in a full physical simulation condition. Our research establishes a theoretical foundation for the future concealment of SAR targets in practical settings and offers valuable insights for enhancing the attack and defense capabilities of subsequent DNNs in SAR ATR systems.

As machine and deep learning models are increasingly leveraged in predictive process monitoring, the focus has shifted towards making these models explainable. The successful adoption of a model is dependent on whether decision-makers can trust the predictions and explanations made. However, recent studies have shown that deep learning models are vulnerable to adversarial attacks -small perturbations to the inputs-which trick deep learning algorithms into making incorrect predictions. An additional crucial property is that the explanations are robust against these adversarial attacks when the model decision was not affected. Therefore, this paper introduces a robustness assessment framework by investigating the impact of adversarial attacks on the robustness of predictive accuracy and explanations used in the field of predictive process monitoring. First, adversarial examples of cases in the independent test set are generated to examine the robustness of the predictive model against intentionally manipulated data. Next, the predictive models are compared with similar models trained on data imputed with adversarial attacks. We monitor the impact on predictive performance in terms of AUC at different stages of the case execution. Finally, the robustness of the explanations is calculated as the distance between the original explanations and the explanations extracted from the model trained on attacked data. We test multiple machine and deep learning techniques, namely the transparent logistic regression, random forests with Shapley values, and LSTM neural networks with attention. Results show that especially neural networks suffer from adversarial attacks, and the former two are mostly robust in terms of both predictive accuracy and explanations.

Some recent articles have revealed that synthetic aperture radar automatic target recognition (SAR-ATR) models based on deep learning are vulnerable to the attacks of adversarial examples and cause security problems. The adversarial attack can make a deep convolutional neural network (CNN)-based SAR-ATR system output the intended wrong label predictions by adding small adversarial perturbations to the SAR images. The existing optimization-based adversarial attack methods generate adversarial examples by minimizing the mean-squared reconstruction error, causing smooth target edge and blurry weak scattering centers in SAR images. In this paper, we build a UNet-generative adversarial network (GAN) to refine the generation of the SAR-ATR models’ adversarial examples. The UNet learns the separable features of the targets and generates the adversarial examples of SAR images. The GAN makes the generated adversarial examples approximate to real SAR images (with sharp target edge and explicit weak scattering centers) and improves the generation efficiency. We carry out abundant experiments using the proposed adversarial attack algorithm to fool the SAR-ATR models based on several advanced CNNs, which are trained on the measured SAR images of the ground vehicle targets. The quantitative and qualitative results demonstrate the high-quality adversarial example generation and excellent attack effectiveness and efficiency improvement.

Most deep neural networks are vulnerable to adversarial attacks, making it crucial to develop effective defenses against them. Adversarial examples are created by applying slight perturbations either to the entire image or specific regions, depending on attack methods. By contrast, benign examples typically lead models to focus attention on contiguous regions of the image during classification. Therefore, adversarial examples can be detected effectively by leveraging attention values. First, we conduct simple experiment to investigate how the model’s attention responds to adversarial perturbations. Second, we derive features by calculating the changes in attention values across ViT layers for adversarial and benign examples. These attention-derived features are then fed into the detection model, ResNet-50, to learn the differences in the activation locations between both examples. Third, we demonstrate the effectiveness of our approach by thorough experiments on various attack methods, datasets, and target models. The experiments prove the effectiveness of our approach, achieving superior performance compared to baseline methods on the ImageNet dataset across ViT-B and ViT-L. Our findings underscore the potential of utilizing attention-derived features as powerful indicators for enhancing the resilience of ViT models against adversarial perturbations.

Local aggressive and physically realizable adversarial attacks on 3D point cloud

In this paper, a novel black box adversarial computer vision attack is proposed. The introduced attack is based on removing from images some components described by their Tchebichef discrete orthogonal moments, rather than to perturb them. The contribution of this work is focused on the addition of one more clue, supporting the critical hypothesis that computer vision systems fail because they support their decisions not only in robust features but also in others non-robust ones. In this, context non-robust image features described in terms of Tchebichef moments are excluded from the original images and the approximated reconstructed versions of them are used as adversarial examples in order to attack some popular deep learning models. The experiments justify the effectiveness of the proposed adversarial attack in terms of imperceptibility and recognition error rate of the deep learning classifiers. It is worth noting that the top-1 accuracy of the attacked models was degraded by a factor between 9.48%-70.89% for adversarial images of 65dB to 57dB PSNR values. The corresponding degradation of the top-5 models’ accuracy was between 6.9% and 55.14% for the same quality images. Moreover, the proposed attack seems to have more strength than the Fast Gradient Sign Method (FGSM) attacking method traditionally applying in most cases. These results reveal that the proposed attack is able to exploit the vulnerability of the deep learning models’ towards degrading their generalization abilities. In this paper, a novel black box adversarial computer vision attack is proposed. The introduced attack is based on removing from images some components described by their Tchebichef discrete orthogonal moments, rather than to perturb them. The contribution of this work is focused on the addition of one more clue, supporting the critical hypothesis that computer vision systems fail because they support their decisions not only in robust features but also in others non-robust ones. In this, context non-robust image features described in terms of Tchebichef moments are excluded from the original images and the approximated reconstructed versions of them are used as adversarial examples in order to attack some popular deep learning models. The experiments justify the effectiveness of the proposed adversarial attack in terms of imperceptibility and recognition error rate of the deep learning classifiers. It is worth noting that the top-1 accuracy of the attacked models was degraded by a factor between 9.48%-70.89% for adversarial images of 65dB to 57dB PSNR values. The corresponding degradation of the top-5 models’ accuracy was between 6.9% and 55.14% for the same quality images. Moreover, the proposed attack seems to have more strength than the Fast Gradient Sign Method (FGSM) attacking method traditionally applying in most cases. These results reveal that the proposed attack is able to exploit the vulnerability of the deep learning models’ towards degrading their generalization abilities.

In adversarial attacks, most existing methods adopt global attack methods, which attack by changing all image pixels, but this is not realistic. On the contrary, sparse attacks indicate that only perturbing local regions of the input image can deceive DNN models into making incorrect predictions. However, this method requires a large number of queries to generate adversarial examples, and the key issues it faces are locating the perturbation area and optimizing the magnitude of the perturbation. Currently, generating high-quality adversarial examples and improving query efficiency in restricted environments is a challenge for black-box attacks. In this paper, we propose a sparse black-box attack method based on the Active Subspace Evolution Strategy AS-ES , which locates the active subspace of the input image through the multi-arm bandit method, and uses the Covariance Matrix Adaptive Evolution Strategy algorithm for perturbation search in the low-dimensional subspace. We model this problem as a bi-level optimization problem, optimizing both the perturbation position and magnitude to generate high-quality adversarial examples while achieving efficient attacks. We conducted extensive experiments on multiple datasets and verified that the AS-ES method generates adversarial examples with higher quality and query efficiency than existing state-of-the-art attack methods.

Studying vulnerability of machine learning models to adversarial examples is an important way to understand their robustness and generalization properties. In this paper, we propose a genetic algorithm for generating adversarial examples for machine learning models. Such approach is able to find adversarial examples without the access to model's parameters. Different models are tested, including both deep and shallow neural networks architectures. We show that RBF networks and SVMs with RBF kernels tend to be rather robust and not prone to misclassification of adversarial examples.

The goal of textual adversarial attack methods is to replace some words in an input text in order to make the victim model misbehave. This article proposes an effective word-level adversarial attack method based on sememes and an improved quantum-behaved particle swarm optimization (QPSO) algorithm. The sememe-based substitute method, which uses the words sharing the same sememes as the substitutes of the original words, is first employed to form the reduced search space. Then, an improved QPSO algorithm, called historical information-guided QPSO with random drift local attractor (HIQPSO-RD), is proposed to search the reduced search space for adversarial examples. The HIQPSO-RD introduces historical information into the current mean best position of the QPSO, for the purpose of improving the convergence speed of the algorithm, by enhancing its exploration ability and preventing the premature convergence of the swarm. The proposed algorithm uses the random drift local attractor technique to make a good balance between its exploration and exploitation, so that the algorithm can find a better adversarial attack example with low grammaticality and perplexity (PPL). In addition, it employs a two-stage diversity control strategy to enhance the search performance of the algorithm. Experiments on three natural language processing (NLP) datasets, with three commonly used nature language processing models as victim models, show that our method achieves higher attack success rates but lower modification rates than the state-of-the-art adversarial attack methods. Moreover, the results of human evaluations show that adversarial examples generated by our method can better maintain the semantic similarity and grammatical correctness of the original input.