Probing unlearned diffusion models: A transferable adversarial attack perspective

DiffProtect: Generative adversarial examples using diffusion models for facial privacy protection

Adversarial attacks, particularly targeted transfer-based attacks, can be used to assess the adversarial robustness of large visual-language models (VLMs), allowing for a more thorough examination of potential security flaws before deployment. However, previous transfer-based adversarial attacks incur high costs due to high iteration counts and complex method structure. Furthermore, due to the unnaturalness of adversarial semantics, the generated adversarial examples have low transferability. These issues limit the utility of existing methods for assessing robustness. To address these issues, we propose AdvDiffVLM, which uses diffusion models to generate natural, unrestricted and targeted adversarial examples via score matching. Specifically, AdvDiffVLM uses Adaptive Ensemble Gradient Estimation (AEGE) to modify the score during the diffusion model’s reverse generation process, ensuring that the produced adversarial examples have natural adversarial targeted semantics, which improves their transferability. Simultaneously, to improve the quality of adversarial examples, we use the GradCAM-guided Mask Generation (GCMG) to disperse adversarial semantics throughout the image rather than concentrating them in a single area. Finally, AdvDiffVLM embeds more target semantics into adversarial examples after multiple iterations. Experimental results show that our method generates adversarial examples 5x to 10x faster than state-of-the-art (SOTA) transfer-based adversarial attacks while maintaining higher quality adversarial examples. Furthermore, compared to previous transfer-based adversarial attacks, the adversarial examples generated by our method have better transferability. Notably, AdvDiffVLM can successfully attack a variety of commercial VLMs in a black-box environment, including GPT-4V. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/gq-max/AdvDiffVLM</uri>

Adversarial attack is an approach that primarily compromise the integrity of deep learning system in medical imaging by adding subtle changes to the model inputs that humans don't notice, it leads the model to make an incorrect decision, so the attacker can corrupt both the input data and the prediction system. In this paper A hybrid detection method based on diffusion purification, deep feature extraction, and ensemble learning has been proposed to address this issue using chest x-ray images.Two phases make it up. First, cleansing the adversarial samples. This is accomplished by a diffusion model. The samples are regenerated to remove the small perturbations fully. In the second phase, the images are categorized into two groups. Clean and purified images are collected in a safe category, while harmful images are unsafe category. In the second stage, we utilize a pre-trained ResNet18 feature extractor to retrieve salient from chest X-rays. Furthermore, a Random Forest model classifies the features obtained from the ResNet18 model into harmful and non-harmful image. The experimental results show that the proposed framework can detect more than 99% of adversarial samples. The satisfactory detection rate of the proposed purification and ensemble-based feature detection for making AI-assisted disease detection more reliable and safer

The emergence of Vision-Language Models (VLMs) is significant advancement in integrating computer vision with Large Language Models (LLMs) to enhance multi-modal machine learning capabilities. However, this progress has made VLMs vulnerable to advanced adversarial attacks, raising concerns about reliability. Objective of this paper is to assess resilience of VLMs against jailbreak attacks that can compromise model safety compliance and result in harmful outputs. To evaluate VLM's ability to maintain robustness against adversarial input perturbations, we propose novel metric called \textbf{Retention Score}. Retention Score is multi-modal evaluation metric that includes Retention-I and Retention-T scores for quantifying jailbreak risks in visual and textual components of VLMs. Our process involves generating synthetic image-text pairs using conditional diffusion model. These pairs are then predicted for toxicity score by VLM alongside toxicity judgment classifier. By calculating margin in toxicity scores, we can quantify robustness of VLM in attack-agnostic manner. Our work has four main contributions. First, we prove that Retention Score can serve as certified robustness metric. Second, we demonstrate that most VLMs with visual components are less robust against jailbreak attacks than corresponding plain VLMs. Additionally, we evaluate black-box VLM APIs and find that security settings in Google Gemini significantly affect score and robustness. Moreover, robustness of GPT4V is similar to medium settings of Gemini. Finally, our approach offers time-efficient alternative to existing adversarial attack methods and provides consistent model robustness rankings when evaluated on VLMs including MiniGPT-4, InstructBLIP, and LLaVA.

Deep neural networks (DNNs) have achieved remarkable success in widespread applications. Meanwhile, its vulnerability towards carefully crafted adversarial attacks captures special attention. Not only adversarial perturbations in digital space will fool the target DNNs-based detectors making a wrong decision, but also actually printed patches can be camouflaged to defeat detectors in physical space. In particular, multi-view physical adversarial attacks pose a more serious threat to practical scenarios. The existing attacks are still challenged in three aspects, i.e., high-cost data augmentation, attack performance gap between digital and physical space, and low attack transferability across DNNs. To overcome the challenges, we introduce PhyCamo, a robust physical camouflage framework based on contrastive learning that distinguishes itself from prior research in various critical ways: (1) data augmentation - it utilizes the diffusion model for data augmentation to efficiently simulate sophisticated physical dynamics in real-world; (2) robustness - it leverages contrastive learning to optimize physical camouflage against encoders with the state-of-the-art (SOTA) attack performance; (3) transferability - it mitigates the model-specific noise in the optimization by adopting diverse input methods, thereby amplifying the transferability between models. Extensive experiments are carried out on a car dataset, a tank dataset, and a pedestrian dataset, comparing with 6 classic multi-view physical adversarial attacks in both digital and physical spaces. The results demonstrate PhyCamo’s superior performance. For instance, it generates more effective physical camouflage (with higher attack success rate~×1.26 and reduce the model's average precision by 55%). PhyCamo can also help to improve the robustness of detectors through adversarial training, which contributes to the application of deep neural networks in the field of security sensitivity.

Salient object detection (SOD) networks are vulnerable to adversarial attacks. As adversarial training is computationally expensive for SOD, existing defense methods instead adopt a noise-against-noise strategy that disrupts adversarial perturbation and restores the image either in input or feature space. However, their limited learning capacity and the need for network modifications limit their applicability. In recent years, the popular diffusion model coincides with the existing defense idea and exhibits excellent purification performance, but there still remains an accuracy gap between the saliency results generated from the purified images and the benign images. In this paper, we propose a Robust and Refined (RoRe) SOD defense framework based on the diffusion model to simultaneously achieve adversarial robustness as well as improved accuracy for benign and purified images. Our proposed RoRe defense consists of three modules: purification, adversarial detection, and refinement. The purification module leverages the powerful generation capability of the diffusion model to purify perturbed input images to achieve robustness. The adversarial detection module utilizes the guidance classifier in the diffusion model for multi-step voting classification. By combining this classifier with a similarity condition, precise adversarial detection can be achieved, providing the possibility of regaining the original accuracy for benign images. The refinement module uses a simple and effective UNet to enhance the accuracy of purified images. The experiments demonstrate that RoRe achieves superior robustness over state-of-the-art methods while maintaining high accuracy for benign images. Moreover, RoRe shows good results against backward pass differentiable approximation (BPDA) attacks.

Diffattack-X: An effective transferable adversarial attack based on diffusion models

The rapid advancements in deep learning (DL) and artificial intelligence (AI) have led to transformative applications across various domains. Edge devices, including mobile and embedded systems, are crucial for the widespread adoption of machine intelligence. However, training and deploying deep neural networks (DNNs) require substantial computational resources, resulting in high energy consumption. This energy often stems from non-renewable sources, leading to significant carbon dioxide emissions and environmental damage. With the emergence of large-scale models for AI-generated content (AIGC), such as large language models and diffusion models, the energy consumption issue has intensified, creating an urgent demand for sustainable AI solutions. These challenges motivate my research on energy-efficient deep learning toward sustainable AI. To build sustainable AI systems, this thesis works on promoting AI sustainability and using AI for sustainability purposes. To promote AI sustainability, a hardware-software co-design approach is adopted, integrating both bottom-up and top-down strategies for practical, energy-efficient deep learning deployments.The first part of the thesis introduces the bottom-up approach, emphasizing AI algorithm-aware efficient system design. Specifically, edge devices utilize dynamic power management strategies, such as dynamic voltage and frequency scaling (DVFS), to manage energy consumption by adjusting voltage-frequency (VF) levels. System-level configurations, including CPU cores, CPU VF levels, and GPU VF levels, significantly impact both latency and power consumption during DNN inference. Developing effective strategies for these configurations enables system-level optimizations, providing a path to deal with the efficiency challenge. The second part of the thesis presents the top-down approach, which focuses on designing hardware-driven efficient AI algorithms. Given the hardware constraints in AI application environments, this approach explores efficient compression techniques, such as weight pruning and knowledge distillation, to create lightweight models with reduced computational demands, thereby enhancing efficiency. The third part of the thesis examines the efficiency challenges posed by the evolution of large-scale models, which have greatly enhanced the capabilities of AIGC for text and image generation. While these models offer substantial generative potential, they also exacerbate the efficiency dilemma. This section explores strategies to optimize the efficiency of these large-scale models based on their unique characteristics. Beyond efficiency, the deployment of AI in real-world applications, such as autonomous vehicles, underscores the importance of prioritizing safety. The fourth part of the thesis addresses issues of AI fairness, trustworthiness, and security to ensure sustainable AI. This section investigates the potential to reverse-engineer adversarial attacks to infer the adversary's underlying information. In conclusion, this thesis works on building energy-efficient DL for sustainable AI. The objective of the work is not only to enhance the efficiency of AI systems but also to address broader concerns of fairness, trust, and security, paving the way for responsible and sustainable AI deployment in real-world scenarios.--Author's abstract

Abstract—The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI). These models are now foundational to a wide range of applications, including conversational AI, recommendation systems, autonomous driving, content generation, medical diagnostics, and scientific discovery. However, their widespread deployment also exposes them to significant safety risks, raising concerns about robustness, reliability, and ethical implications. This survey provides a systematic review of current safety research on large models, covering Vision Foundation Models (VFMs), Large Language Models (LLMs), Vision-Language Pre-training (VLP) models, Vision-Language Models (VLMs), Diffusion Models (DMs), and large-model-based Agents. Our contributions are summarized as follows: (1) We present a comprehensive taxonomy of safety threats to these models, including adversarial attacks, data poisoning, backdoor attacks, jailbreak and prompt injection attacks, energy-latency attacks, data and model extraction attacks, and emerging agent-specific threats. (2) We review defense strategies proposed for each type of attacks if available and summarize the commonly used datasets and benchmarks for safety research. (3) Building on this, we identify and discuss the open challenges in large model safety, emphasizing the need for comprehensive safety evaluations, scalable and effective defense mechanisms, and sustainable data practices. More importantly, we highlight the necessity of collective efforts from the research community and international collaboration. Our work can serve as a useful reference for researchers and practitioners, fostering the ongoing development of comprehensive defense systems and platforms to safeguard AI models.

The field of object detection has witnessed significant advancements in recent years, thanks to the remarkable progress in artificial intelligence and deep learning. These breakthroughs have significantly enhanced the accuracy and efficiency of detecting and categorizing objects in digital images. Nonetheless, contemporary object detection technologies have certain limitations, such as their inability to counter white-box attacks, insufficient denoising, suboptimal reconstruction, and gradient confusion. To overcome these hurdles, this study proposes an innovative approach that uses conditional diffusion models to perturb adversarial examples. The process begins with the application of a random chessboard mask to the adversarial example, followed by the addition of a slight noise to fill the masked area during the forward process. The adversarial image is then restored to its original form through a reverse generative process that only considers the masked pixels, not the entire image. Next, we use the complement of the initial mask as the mask for the second stage to reconstruct the image once more. This two-stage masking process allows for the complete removal of global disturbances and aids in image reconstruction. In particular, we employ a conditional diffusion model based on a class-conditional U-Net architecture, with the source image further conditioned through concatenation. Our method outperforms the recently introduced HARP method by 5% and 6.5% in mAP on the COCO2017 and PASCAL VOC datasets, respectively, under non-APT PGD attacks. Comprehensive experimental results confirm that our method can effectively restore adversarial examples, demonstrating its practical utility.

Multi-View Diffusion Models (MVDMs) enable remarkable improvements in the field of 3D geometric reconstruction, but the issue regarding intellectual property has received increasing attention due to unauthorized imitation. Recently, some works have utilized adversarial attacks to protect copyright. However, all these works focus on single-image generation tasks which only need to consider the inner feature of images. Previous methods are inefficient in attacking MVDMs because they lack the consideration of disrupting the geometric and visual consistency among the generated multi-view images. This paper is the first to address the intellectual property infringement issue arising from MVDMs. Accordingly, we propose a novel latent feature and attention dual erasure attack to disrupt the distribution of latent feature and the consistency across the generated images from multi-view and multi-domain simultaneously. The experiments conducted on SOTA MVDMs indicate that our approach achieves superior performances in terms of attack effectiveness, transferability, and robustness against defense methods. Therefore, this paper provides an efficient solution to protect 3D assets from MVDMs-based 3D geometry reconstruction. The code is publicly available at: https://github.com/super-jw/LFADEA

CNN-based classifiers have become an essential tool in medical imaging for tasks such as chest X-ray classification. However, their widespread adoption is hindered by two significant challenges: vulnerability to adversarial attacks and poor generalization to unseen data. To address these issues, this study proposes a novel framework called Synthetic-Augmented Adversarial Training (SADA), which integrates FGSM-based adversarial training with synthetic image augmentation using a fine-tuned Stable Diffusion model. This approach aims to enhance both model robustness against perturbations and generalization capabilities by increasing training data diversity. Two CNN architectures, ConvNeXt-B and ConvNeXt-S, are trained under three different dataset configurations: clean images only; clean with adversarial images; and our proposed clean with adversarial and synthetic images. The models are evaluated under both white-box and cross-model FGSM attacks using an unseen chest X-ray dataset. The results demonstrate that under a cross-model attack on unseen data with an epsilon ( $$\epsilon$$ ) value of 0.2, models trained with the SADA framework achieved an F1-score of 0.8437 , representing a 17.40% improvement over the Clean + Adversarial configuration. These findings support the hypothesis that synthetic data helps address the generalization gaps of traditional adversarial training. Our findings suggest that combining adversarial and synthetic data offers a promising and scalable strategy for developing more reliable CNN-based diagnostic systems.