DiffProtect: Generative adversarial examples using diffusion models for facial privacy protection

The adversarial attack is a cutting-edge technique used to study the vulnerability of deep neural networks (DNNs). However, most existing studies focus on the additive perturbation-based attack, which cannot represent real-world corruption and limit their applications. In particular, haze is a common natural phenomenon that significantly corrupts an image, which inevitably poses a potential threat to deep models. In this work, for the first attempt, we study the effects of haze on DNNs from the perspective of adversarial attacks and propose two adversarial haze attack methods. We first propose the optimization-based adversarial haze attack (OAdvHaze) that optimizes the parameters of the atmospheric scattering model with the guidance of a DNN to synthesize a hazy image, which leads to a high attack success rate. To achieve a more efficient attack, we further propose a predictive adversarial haze attack (PAdvHaze) employing a DNN to predict the hazing parameters through a one-step way. To validate the effectiveness of both methods, we conducted extensive experiments on two publicly available datasets, i.e., ILSVRC 2012 and NIPS 2017. OAdvHaze and PAdvHaze achieve comparable attack success rates and transferability to state-of-the-art attacks. This work would contribute to the evaluation and enhancement of the robustness of DNNs against haze perturbation that may happen in the real world.

The goal of textual adversarial attack methods is to replace some words in an input text in order to make the victim model misbehave. This article proposes an effective word-level adversarial attack method based on sememes and an improved quantum-behaved particle swarm optimization (QPSO) algorithm. The sememe-based substitute method, which uses the words sharing the same sememes as the substitutes of the original words, is first employed to form the reduced search space. Then, an improved QPSO algorithm, called historical information-guided QPSO with random drift local attractor (HIQPSO-RD), is proposed to search the reduced search space for adversarial examples. The HIQPSO-RD introduces historical information into the current mean best position of the QPSO, for the purpose of improving the convergence speed of the algorithm, by enhancing its exploration ability and preventing the premature convergence of the swarm. The proposed algorithm uses the random drift local attractor technique to make a good balance between its exploration and exploitation, so that the algorithm can find a better adversarial attack example with low grammaticality and perplexity (PPL). In addition, it employs a two-stage diversity control strategy to enhance the search performance of the algorithm. Experiments on three natural language processing (NLP) datasets, with three commonly used nature language processing models as victim models, show that our method achieves higher attack success rates but lower modification rates than the state-of-the-art adversarial attack methods. Moreover, the results of human evaluations show that adversarial examples generated by our method can better maintain the semantic similarity and grammatical correctness of the original input.

Identifying objects in surveillance and reconnaissance systems with the human eye can be challenging, underscoring the growing importance of employing deep learning models for the recognition of enemy weapon systems. These systems, leveraging deep neural networks known for their strong performance in image recognition and classification, are currently under extensive research. However, it is crucial to acknowledge that surveillance and reconnaissance systems utilizing deep neural networks are susceptible to vulnerabilities posed by adversarial examples. While prior adversarial example research has mainly utilized publicly available internet data, there has been a significant absence of studies concerning adversarial attacks on data and models specific to real military scenarios. In this paper, we introduce an adversarial example designed for a binary classifier tasked with recognizing helicopters. Our approach generates an adversarial example that is misclassified by the model, despite appearing unproblematic to the human eye. To conduct our experiments, we gathered real attack and transport helicopters and employed TensorFlow as the machine learning library of choice. Our experimental findings demonstrate that the average attack success rate of the proposed method is 81.9%. Additionally, when epsilon is 0.4, the attack success rate is 90.1%. Before epsilon reaches 0.4, the attack success rate increases rapidly, and then we can see that epsilon increases little by little thereafter.

As adversarial attack technology advances rapidly, more individuals are employing it to safeguard crucial and private images. Adversarial attacks modify the pixel values of images to achieve the result of misleading neural network decisions. However, modifying pixels can seriously weaken the effectiveness of digital forensics of pictures in the military and medical fields. Therefore, there is a need to safeguard images and have the capability to restore them to their original state in these fields. Currently, methods for generating reversible adversarial examples exhibit significant limitations, such as the inability to fully embed perturbation information, resulting in unsatisfactory image recovery and protected images with poor visual quality. In this paper, we use reversible information hiding techniques and pixel smoothing operations in the HSV colorspace to produce higher-quality protected images while ensuring the lossless recovery of protected images. Experiments show that the method generates reversible adversarial examples with excellent visual quality compared to existing methods.

Explanation-Guided Adversarial Example Attacks

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

In recent years, deep neural networks (DNNs) have made significant progress on face recognition (FR). However, DNNs have been found to be vulnerable to adversarial examples, leading to fatal consequences in real-world applications. This article focuses on improving the transferability of adversarial examples against FR models. We propose gradient eroding (GE) to make the gradient of the residual blocks more diverse, by eroding the back-propagation dynamically. We also propose a novel black-box adversarial attack named corrasion attack based on GE. Extensive experiments demonstrate that our approach can effectively improve the transferability of adversarial attacks against FR models. Our approach overperforms 29.35% in fooling rate than state-of-the-art black-box attacks. Leveraging adversarial training with adversarial examples generated by us, the robustness of models can be improved by up to 43.2%. Besides, corrasion attack successfully breaks two online FR systems, achieving a highest fooling rate of 89.8%.

In recent years, with the widespread application of Vision Transformer (ViT) in visual trackers, their robustness has received increasing attention. However, by focusing on global interactions between image patches, ViT reduces sensitivity to local noise, posing new challenges for adversarial attacks. Meanwhile, existing decision-based adversarial attack methods often overlook the differences in noise sensitivity between different patches, further limiting the compression efficiency of adversarial noise, especially in ViT. In visual tracking, existing adversarial attack methods primarily target Siamese network-based trackers, and research on adversarial attacks against Transformer-based trackers, particularly decision-based black-box attacks, is still relatively limited. To implement effective black-box attacks on Transformer-based trackers, this paper innovatively proposes patch-based adversarial noise compression (PANC), a decision-based adversarial attack method. This method effectively compresses adversarial noise patch by patch, significantly improving compression efficiency and attack concealment. PANC also introduces a noise sensitivity matrix that dynamically adds and reduces adversarial noise, optimizing the spatial distribution of noise while decreasing the number of queries. We validated the effectiveness of the proposed PANC attack method on several Transformer-based trackers, including OSTrack, STARK, TransT, and MixformerV2, and three public large-scale benchmark datasets: GOT-10k, TrackingNet, and LaSOT. Experimental results show that compared to the existing state-of-the-art adversarial attack method, the IoU attack, PANC compresses the noise level to 10%, improving the attack effectiveness by 162% with the number of queries of only 45.7%. Furthermore, PANC can serve as an initialization or post-processing optimization strategy for other adversarial attack methods, providing a more flexible and efficient mechanism for adversarial example generation. Our work reveals the vulnerabilities of existing Transformer-based visual trackers and offers new ideas for further improving the efficiency and concealment of adversarial attacks.

Natural language processing models based on neural networks are vulnerable to adversarial examples. These adversarial examples are imperceptible to human readers but can mislead models to make the wrong predictions. In a black-box setting, attacker can fool the model without knowing model’s parameters and architecture. Previous works on word-level attacks widely use single semantic space and greedy search as a search strategy. However, these methods fail to balance the attack success rate, quality of adversarial examples and time consumption. In this paper, we propose BeamAttack, a textual attack algorithm that makes use of mixed semantic spaces and improved beam search to craft high-quality adversarial examples. Extensive experiments demonstrate that BeamAttack can improve attack success rate while saving numerous queries and time, e.g., improving at most 7% attack success rate than greedy search when attacking the examples from MR dataset. Compared with heuristic search, BeamAttack can save at most 85% model queries and achieve a competitive attack success rate. The adversarial examples crafted by BeamAttack are highly transferable and can effectively improve model’s robustness during adversarial training. Code is available at https://github.com/zhuhai-ustc/beamattack/tree/master

With the continuous development of deep learning in the field of Synthetic Aperture Radar (SAR) image processing, it is found that image recognition is vulnerable to interference and the accuracy is greatly reduced. Therefore, if a general adversarial attack method is designed to generate adversarial examples(AEs) capable of deceiving different types of convolutional neural network models, it can protect target images from being correctly recognized by adversaries and safeguard image privacy information. However, most of the current adversarial attack methods have too large attack scope and interference intensity, and do not have the ability to universally attack different neural network models, resulting in poor concealment, deception, and transfer deception of adversarial attacks. In this paper, we propose a general Key Points Sparse Attack (KPSA) method for SAR images, which achieves excellent deception and transfer deception while controlling the attack range and jamming strength. It is a general and efficient adversarial attack method. KPSA uses the generator to generate intensity interference images and imposes amplitude constraint, uses the key points extraction method to generate position interference images to realize sparse attack, uses the difference value(d-value) loss function to achieve fast convergence of the training model, and achieves transfer deception performance without depending on the type of discriminator. The deception rate and transfer deception rate experiments were conducted on MSTAR and ATRNet-STAR. By comparing with the advanced methods in the field of adversarial attacks, it was verified that the KPSA method is superior to the current advanced methods, and it was confirmed that the KPSA method has application value in various complex practical scenarios.

Adversarial attacks, particularly targeted transfer-based attacks, can be used to assess the adversarial robustness of large visual-language models (VLMs), allowing for a more thorough examination of potential security flaws before deployment. However, previous transfer-based adversarial attacks incur high costs due to high iteration counts and complex method structure. Furthermore, due to the unnaturalness of adversarial semantics, the generated adversarial examples have low transferability. These issues limit the utility of existing methods for assessing robustness. To address these issues, we propose AdvDiffVLM, which uses diffusion models to generate natural, unrestricted and targeted adversarial examples via score matching. Specifically, AdvDiffVLM uses Adaptive Ensemble Gradient Estimation (AEGE) to modify the score during the diffusion model’s reverse generation process, ensuring that the produced adversarial examples have natural adversarial targeted semantics, which improves their transferability. Simultaneously, to improve the quality of adversarial examples, we use the GradCAM-guided Mask Generation (GCMG) to disperse adversarial semantics throughout the image rather than concentrating them in a single area. Finally, AdvDiffVLM embeds more target semantics into adversarial examples after multiple iterations. Experimental results show that our method generates adversarial examples 5x to 10x faster than state-of-the-art (SOTA) transfer-based adversarial attacks while maintaining higher quality adversarial examples. Furthermore, compared to previous transfer-based adversarial attacks, the adversarial examples generated by our method have better transferability. Notably, AdvDiffVLM can successfully attack a variety of commercial VLMs in a black-box environment, including GPT-4V. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/gq-max/AdvDiffVLM</uri>

Nowadays deep neural networks have been applied widely in many applications of computer vision including medical diagnosis and self-driving cars. However, deep neural networks are threatened by adversarial examples usually in which image pixels were perturbed unnoticeable to humans but enough to fool the deep networks. Compared to 2D image adversarial examples, 3D adversarial models are less invasive in the process of attacks, and thus more realistic. There have been many research works on generating 3D adversarial examples. In this paper, we study the robustness of 3D adversarial attacks when the victim camera is placed at different viewpoints. In particular, we find a method to create 3D adversarial examples that can achieve 100% attack success rate from all viewpoints with any integer spherical coordinates. Our method is simple as we only perturb the texture space. We create 3D models with realistic textures using 3D reconstruction from multiple uncalibrated images. With the help of a differentiable renderer, we then apply gradient based optimization to compute texture perturbations based on a set of rendered images, i.e., training dataset. Our extensive experiments show that even only including 1% of all possible rendered images in training, we can still achieve 99.9% attack success rate with the trained texture perturbations. Furthermore, our thorough experiments show high transferability of the multiview robustness of our 3D adversraial attacks across various state-of-the-art deep neural network models.

Adversarial example generation is a technique that involves perturbing inputs with imperceptible noise to induce misclassifications in neural networks, serving as a means to assess the robustness of such models. Among the adversarial attack algorithms, momentum iterative fast gradient sign Method (MI-FGSM) and its variants constitute a class of highly effective offensive strategies, achieving near-perfect attack success rates in white-box settings. However, these methods’ use of sign activation functions severely compromises gradient information, which leads to low success rates in black-box attacks and results in large adversarial perturbations. In this paper, we introduce a novel adversarial attack algorithm, NA-FGTM. Our method employs the Tanh activation function instead of the sign which can accurately preserve gradient information. In addition, it utilizes the Adam optimization algorithm as well as the Nesterov acceleration, which is able to stabilize gradient update directions and expedite gradient convergence. Above all, the transferability of adversarial examples can be enhanced. Through integration with data augmentation techniques such as DIM, TIM, and SIM, NA-FGTM can further improve the efficacy of black-box attacks. Extensive experiments on the ImageNet dataset demonstrate that our method outperforms the state-of-the-art approaches in terms of black-box attack success rate and generates adversarial examples with smaller perturbations.

FE-DaST: Fast and effective data-free substitute training for black-box adversarial attacks

As a countermeasure against malicious authentication in a face recognition system using a face image obtained from SNS or the like, de-identification methods based on adversarial example have been studied. However, since adversarial example directly uses the gradient information of a face recognition model, it is highly dependent on the model, and a de-identification effect and image quality are difficult to achieve for an unknown recognition model. In this study, we propose a novel deidentification method based on a diffusion model, which has high generalizability to an unknown recognition model by applying minute changes to face shapes. Experiments using LFW showed that the proposed method has a higher de-identification effect for unknown models and better image quality than a conventional method using adversarial example.