Robust Token Gradient and Frequency-Aware Transferable Adversarial Attacks on Vision Transformers

In recent years, deep neural networks (DNNs) have made significant progress on face recognition (FR). However, DNNs have been found to be vulnerable to adversarial examples, leading to fatal consequences in real-world applications. This article focuses on improving the transferability of adversarial examples against FR models. We propose gradient eroding (GE) to make the gradient of the residual blocks more diverse, by eroding the back-propagation dynamically. We also propose a novel black-box adversarial attack named corrasion attack based on GE. Extensive experiments demonstrate that our approach can effectively improve the transferability of adversarial attacks against FR models. Our approach overperforms 29.35% in fooling rate than state-of-the-art black-box attacks. Leveraging adversarial training with adversarial examples generated by us, the robustness of models can be improved by up to 43.2%. Besides, corrasion attack successfully breaks two online FR systems, achieving a highest fooling rate of 89.8%.

Deep neural networks are vulnerable to adversarial attacks, where a small perturbation to an input alters the model prediction. In many cases, malicious inputs intentionally crafted for one model can fool another model. In this paper, we present the first study to systematically investigate the transferability of adversarial examples for text classification networks, and how various factors, including network architecture, tokenization scheme, word embedding, and model capacity, affect the transferability of adversarial examples with extensive experiments. Based on these studies, we pro- pose a genetic algorithm to find an ensemble of models that can be used to induce adversarial examples to fool almost all existing models. Such adversarial examples reflect the defects of the learning process and the bias in training sets. Finally, we generalize these adversarial examples into a set of word replacement rules that can be used for model diagnostics.

Adversarial attacks, particularly targeted transfer-based attacks, can be used to assess the adversarial robustness of large visual-language models (VLMs), allowing for a more thorough examination of potential security flaws before deployment. However, previous transfer-based adversarial attacks incur high costs due to high iteration counts and complex method structure. Furthermore, due to the unnaturalness of adversarial semantics, the generated adversarial examples have low transferability. These issues limit the utility of existing methods for assessing robustness. To address these issues, we propose AdvDiffVLM, which uses diffusion models to generate natural, unrestricted and targeted adversarial examples via score matching. Specifically, AdvDiffVLM uses Adaptive Ensemble Gradient Estimation (AEGE) to modify the score during the diffusion model’s reverse generation process, ensuring that the produced adversarial examples have natural adversarial targeted semantics, which improves their transferability. Simultaneously, to improve the quality of adversarial examples, we use the GradCAM-guided Mask Generation (GCMG) to disperse adversarial semantics throughout the image rather than concentrating them in a single area. Finally, AdvDiffVLM embeds more target semantics into adversarial examples after multiple iterations. Experimental results show that our method generates adversarial examples 5x to 10x faster than state-of-the-art (SOTA) transfer-based adversarial attacks while maintaining higher quality adversarial examples. Furthermore, compared to previous transfer-based adversarial attacks, the adversarial examples generated by our method have better transferability. Notably, AdvDiffVLM can successfully attack a variety of commercial VLMs in a black-box environment, including GPT-4V. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/gq-max/AdvDiffVLM</uri>

Existing research shows that cooperative multi-agent deep reinforcement learning (c-MADRL) is vulnerable to adversarial attacks, and c-MADRL is increasingly being applied to safety-critical domains. However, the robustness of c-MADRL against adversarial attacks has not been fully studied. In the setting of c-MADRL, unlike the single-agent scenario, an adversary can attack multiple agents or all agents at each time step, but the attacker needs more computation to generate adversarial examples and will be more easily detected. Therefore, how the attacker chooses one or several agents instead of all agents to attack is a significant issue in the setting of c-MADRL. Aiming to address this issue, this paper proposes a novel adversarial attack approach, which dynamically groups the agents according to relevant features and selects a group to attack based on the group’s contribution to the overall reward, thus effectively reducing the cost and number of attacks, as well as improving attack efficiency and decreasing the chance of attackers being detected. Moreover, we exploit the transferability of adversarial examples to greatly reduce the computational cost of generating adversarial examples. Our method is tested in multi-agent particle environments (MPE) and in StarCraft II. Experimental results demonstrate that our proposed method can effectively degrade the performance of multi-agent deep reinforcement learning algorithms with fewer attacks and lower computational costs.

Recent studies have shown that deep learning models such as image classification and object detection models are vulnerable to attacks from adversarial examples. These malicious examples may trigger potential security threats and cause damage in security-related fields. In this paper, in order to find model robustness weakness, we obtain the adversarial robustness metric against several adversarial attacks, including gradient-based attacks, optimization-based attacks, etc. The metric quantizes the adversarial robustness of deep learning models and can be used as a guide for extra model training. Meanwhile, we design a testing platform to achieve model adversarial robustness evaluation. Utilizing the front and back end separation strategy, the testing platform consist of 2 modules: one is multiple adversarial example attacks executor and the other is HCI module. In addition, the tool allows parallel comparison among different adversarial attack algorithms under given conditions and measures the quality of generated adversarial examples utilizing FID(Fréchet Inception Distance) metrics. Besides, we developed ERFGSM(edge-RFGSM), a new gradient-based attack using edge information through canny operator.

Ensemble-based Adversarial Training is a principled approach to achieve robustness against adversarial attacks. An important technicality of this approach is to control the transferability of adversarial examples between ensemble members. We propose in this work a simple, but effective strategy to collaborate among committee models of an ensemble model. This is achieved via the secure and insecure sets defined for each model member on a given sample, hence help us to quantify and regularize the transferability. Consequently, our proposed framework provides the flexibility to reduce the adversarial transferability as well as promote the diversity of ensemble members, which are two crucial factors for better robustness in our ensemble approach. We conduct extensive and comprehensive experiments to demonstrate that our proposed method outperforms the state-of-the-art ensemble baselines, at the same time can detect a wide range of adversarial examples with a near perfect accuracy.

Artificial neural networks (ANNs) have gained significant popularity in the last decade for solving narrow AI problems in domains such as healthcare, transportation, and defense. As ANNs become more ubiquitous, it is imperative to understand their associated safety, security, and privacy vulnerabilities. Recently, it has been shown that ANNs are susceptible to a number of adversarial evasion attacks - inputs that cause the ANN to make high-confidence misclassifications despite being almost indistinguishable from the data used to train and test the network. This work explores to what degree finding these examples may be aided by using side-channel information, specifically switching power consumption, of hardware implementations of ANNs. A black-box threat scenario is assumed, where an attacker has access to the ANN hardware’s input, outputs, and topology, but the trained model parameters are unknown. Then, a surrogate model is trained to have similar functional (i.e. input-output mapping) and switching power characteristics as the oracle (black-box) model. Our results indicate that the inclusion of power consumption data increases the fidelity of the model extraction by up to 30% based on a mean square error comparison of the oracle and surrogate weights. However, transferability of adversarial examples from the surrogate to the oracle model was not significantly affected.

Vision-Language Pretrained (VLP) models exhibit strong multimodal understanding and reasoning capabilities, finding wide application in tasks such as image-text retrieval and visual grounding. However, they remain highly vulnerable to adversarial attacks, posing serious reliability concerns in safety-critical scenarios. We observe that existing adversarial examples optimization methods typically rely on individual features from the other modality as guidance, causing the crafted adversarial examples to overfit that modality's learning preferences and thus limiting their transferability. In order to further enhance the transferability of adversarial examples, we propose a novel adversarial attack framework, I&CA (Individual & Common feature Attack), which simultaneously considers individual features within each modality and common features cross-modal interactions. Concretely, I&CA first drives divergence among individual features within each modality to disrupt single-modality learning, and then suppresses the expression of common features during cross-modal interactions, thereby undermining the robustness of the fusion mechanism. In addition, to prevent adversarial perturbations from overfitting to the learning bias of the other modality, which may distort the representation of common features, we simultaneously introduce augmentation strategies to both modalities. Across various experimental settings and widely recognized multimodal benchmarks, the I&CA framework achieves an average transferability improvement of 6.15% over the state-of-the-art DRA method, delivering significant performance gains in both cross-model and cross-task attack scenarios.

Adversarial example generation is a technique that involves perturbing inputs with imperceptible noise to induce misclassifications in neural networks, serving as a means to assess the robustness of such models. Among the adversarial attack algorithms, momentum iterative fast gradient sign Method (MI-FGSM) and its variants constitute a class of highly effective offensive strategies, achieving near-perfect attack success rates in white-box settings. However, these methods’ use of sign activation functions severely compromises gradient information, which leads to low success rates in black-box attacks and results in large adversarial perturbations. In this paper, we introduce a novel adversarial attack algorithm, NA-FGTM. Our method employs the Tanh activation function instead of the sign which can accurately preserve gradient information. In addition, it utilizes the Adam optimization algorithm as well as the Nesterov acceleration, which is able to stabilize gradient update directions and expedite gradient convergence. Above all, the transferability of adversarial examples can be enhanced. Through integration with data augmentation techniques such as DIM, TIM, and SIM, NA-FGTM can further improve the efficacy of black-box attacks. Extensive experiments on the ImageNet dataset demonstrate that our method outperforms the state-of-the-art approaches in terms of black-box attack success rate and generates adversarial examples with smaller perturbations.

Action recognition models using deep learning are vulnerable to adversarial examples, which are transferable across other models trained on the same data modality. Existing transferable attack methods face two major challenges: 1) they heavily rely on the assumption that the decision boundaries of the surrogate (a.k.a., source) model and the target model are similar, which limits the adversarial transferability; and 2) their decision boundary difference makes the attack direction uncertain, which may result in the gradient oscillation, weakening the adversarial attack. This motivates us to propose a Background Mixup-induced Temporal Consistency (BMTC) attack method for action recognition. From the input transformation perspective, we design a model-agnostic background adversarial mixup module to reduce the surrogate-target model dependency. In particular, we randomly sample one video from each category and make its background frame, while selecting the background frame with the top attack ability for mixup with the clean frame by reinforcement learning. Moreover, to ensure an explicit attack direction, we leverage the background category as guidance for updating the gradient of adversarial example, and design a temporal gradient consistency loss, which strengthens the stability of the attack direction on subsequent frames. Empirical studies on two video datasets, i.e., UCF101 and Kinetics-400, and one image dataset, i.e., ImageNet, demonstrate that our method significantly boosts the transferability of adversarial examples across several action/image recognition models.

Transferable adversarial examples can efficiently fool topic models

The transferability of adversarial examples is the key property in practical black-box scenarios. Currently, numerous methods improve the transferability across different models trained on the same modality of data. The investigation of generating video adversarial examples with imagebased substitute models to attack the target video models, i.e., cross-modal transferability of adversarial examples, is rarely explored. A few works on cross-modal transferability directly apply image attack methods for each frame and no factors especial for video data are considered, which limits the cross-modal transferability of adversarial examples. In this paper, we propose an effective cross-modal attack method which considers both the global and local characteristics of video data. Firstly, from the global perspective, we introduce inter-frame interaction into attack process to induce more diverse and stronger gradients rather than perturb each frame separately. Secondly, from the local perspective, we disrupt the inherently local correlation of frames within a video, which prevents black-box video model from capturing valuable temporal clues. Extensive experiments on the UCF-101 and Kinetics-400 validate the proposed method significantly improves cross-modal transferability and even surpasses strong baseline using video models as substitute model. Our source codes are available at https://github.com/lwmming/Cross-Modal-Attack.

Machine learning (ML) algorithms provide better performance than traditional algorithms in various applications. However, some unknown flaws in ML classifiers make them sensitive to adversarial examples generated by adding small but fooled purposeful distortions to natural examples. This paper aims to investigate the transferability of adversarial examples generated on a sparse and structured dataset and the ability of adversarial training in resisting adversarial examples. The results demonstrate that adversarial examples generated by DNN can fool a set of ML classifiers such as decision tree, random forest, SVM, CNN and RNN. Also, adversarial training can improve the robustness of DNN in terms of resisting attacks.

Adversarial examples are crucial tools for assessing the robustness of deep neural networks (DNNs) and revealing potential security vulnerabilities. Adversarial example generation methods based on Generative Adversarial Networks (GANs) have made significant progress in generating image adversarial examples, but still suffer from insufficient sparsity and transferability. To address these issues, this study proposes a novel semi-white-box untargeted adversarial example generation method named Wavelet-AdvGAN, with an explicit threat model defined as follows. Specifically, the attack is strictly untargeted without predefined target categories, aiming solely to mislead DNNs into classifying adversarial examples into any category other than the original label. It adopts a semi-white-box setting where attackers are denied access to the target model’s private information. Regarding the generator’s information dependence, the training phase only utilizes public resources (i.e., the target model’s public architecture and CIFAR-10 public training data), while the test phase generates adversarial examples through one-step feedforward of clean images without interacting with the target model. The method incorporates a Frequency Sub-band Difference (FSD) module and a Wavelet Transform Local Feature (WTLF) extraction module, evaluating the differences between original and adversarial examples from the frequency domain perspective. This approach constrains the magnitude of perturbations, reinforces feature regions, and further enhances the attack effectiveness, thereby improving the sparsity and transferability of adversarial examples. Experimental results demonstrate that the Wavelet-AdvGAN method achieves an average increase of 1.26% in attack success rates under two defense strategies—data augmentation and adversarial training. Additionally, the adversarial transferability improves by an average of 2.7%. Moreover, the proposed method exhibits a lower l0 norm, indicating better perturbation sparsity. Consequently, it effectively evaluates the robustness of deep neural networks.

Some recent articles have revealed that synthetic aperture radar automatic target recognition (SAR-ATR) models based on deep learning are vulnerable to the attacks of adversarial examples and cause security problems. The adversarial attack can make a deep convolutional neural network (CNN)-based SAR-ATR system output the intended wrong label predictions by adding small adversarial perturbations to the SAR images. The existing optimization-based adversarial attack methods generate adversarial examples by minimizing the mean-squared reconstruction error, causing smooth target edge and blurry weak scattering centers in SAR images. In this paper, we build a UNet-generative adversarial network (GAN) to refine the generation of the SAR-ATR models’ adversarial examples. The UNet learns the separable features of the targets and generates the adversarial examples of SAR images. The GAN makes the generated adversarial examples approximate to real SAR images (with sharp target edge and explicit weak scattering centers) and improves the generation efficiency. We carry out abundant experiments using the proposed adversarial attack algorithm to fool the SAR-ATR models based on several advanced CNNs, which are trained on the measured SAR images of the ground vehicle targets. The quantitative and qualitative results demonstrate the high-quality adversarial example generation and excellent attack effectiveness and efficiency improvement.