Adversarial Attacks and Defenses in Image Classification: A Practical Perspective

Untargeted white-box adversarial attack with heuristic defence methods in real-time deep learning based network intrusion detection system

State-of-the-art neural network models are actively used in various fields, but it is well-known that they are vulnerable to adversarial example attacks. Throughout the efforts to make the models robust against adversarial example attacks, it has been found to be a very difficult task. While many defense approaches were shown to be not effective, adversarial training remains as one of the promising methods. In adversarial training, the training data are augmented by “adversarial” samples generated using an attack algorithm. If the attacker uses a similar attack algorithm to generate adversarial examples, the adversarially trained network can be quite robust to the attack. However, there are numerous ways of creating adversarial examples, and the defender does not know what algorithm the attacker may use. A natural question is: Can we use adversarial training to train a model robust to multiple types of attack? Previous work have shown that, when a network is trained with adversarial examples generated from multiple attack methods, the network is still vulnerable to white-box attacks where the attacker has complete access to the model parameters. In this paper, we study this question in the context of black-box attacks, which can be a more realistic assumption for practical applications. Experiments with the MNIST dataset show that adversarially training a network with an attack method helps defending against that particular attack method, but has limited effect for other attack methods. In addition, even if the defender trains a network with multiple types of adversarial examples and the attacker attacks with one of the methods, the network could lose accuracy to the attack if the attacker uses a different data augmentation strategy on the target network. These results show that it is very difficult to make a robust network using adversarial training, even for black-box settings where the attacker has restricted information on the target network.

Recent years, Machine Learning has been widely used in malware analysis and achieved unprecedented success. However, deep learning models are found to be highly vulnerable to adversarial examples, which leads to the machine learning-based malware analysis methods vulnerable to malware makers. Exploring the attack algorithm can not only promote the generation of more effective malware analysis methods, but also can promote the development of the defense algorithm. Different machine learning models use different malware features as their classification basis, and accordingly there will be different attack methods against them. For malware visualization method, corresponding effective adversarial attack has not yet appeared. Most existing malware adversarial examples for malware visualization are generated at the feature level, and do not consider whether the generated adversarial examples can be executed and complete their original functions. In this paper, we explored how to modify an Android executable file without affecting its original functions and made it become an adversarial example. We proposed an executable adversarial examples attack strategy for machine learning-based malware visualization analysis. Experimental result shows that the executable adversarial examples we generated can be normally run on Android devices without affecting its original functions, and can confuse the malware family classifier with 93% success rate. We explored possible defense methods and hope to contribute to building a more robust malware classification method.

Machine Learning (ML) and Deep Learning (DL) models are increasingly deployed in critical domains such as computer vision, natural language processing (NLP), automatic speech recognition (ASR), time-series analysis, and cybersecurity. Despite their success, these models remain vulnerable to adversarial attacks that can mislead predictions and undermine system reliability. To address this challenge, we conduct a Systematic Literature Review (SLR) of adversarial attacks and defenses in ML/DL, following PRISMA guidelines. Our review analyzes 132 peer-reviewed studies published between 2017 and 2025, identified from leading digital libraries including IEEE, ACM, ScienceDirect, and SpringerLink. The objectives of this study are to: (i) identify state-of-the-art adversarial attack and defense techniques and assess their strengths and limitations; (ii) evaluate how novel threats and countermeasures are addressed across domains; and (iii) propose a unified taxonomy that integrates adversarial attacks, defenses, evaluation metrics, and application domains. Our findings show that adversarial training and certified robustness dominate defense research, but both face trade-offs in scalability and generalizability. Computer vision remains the most studied domain, while NLP, ASR, and time-series research is growing but lacks standardized benchmarks. This review contributes a taxonomy and cross-domain synthesis of adversarial ML research, highlights research gaps, and outlines future directions to advance robustness and support the development of trustworthy AI systems.

The rising sophistication of adversarial attacks poses significant threats to the integrity and reliability of real-time financial fraud monitoring systems. As machine learning (ML) and deep learning models become more integrated into financial security infrastructures, they are increasingly vulnerable to subtle perturbations that can mislead fraud detection mechanisms. This review explores the intersection of explainable artificial intelligence (XAI) and generative models—such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs)—in fortifying financial systems against adversarial threats. The study categorizes existing adversarial attack vectors targeting transaction-level anomaly detection and evaluates the limitations of conventional defense techniques. Furthermore, it assesses the role of XAI methods, such as SHAP and LIME, in interpreting model decisions to uncover adversarial behavior in a transparent and auditable manner. Generative models are reviewed both as tools for generating adversarial examples and for enhancing model robustness through adversarial training and anomaly simulation. Real-time constraints are also examined, including latency, scalability, and system responsiveness. The paper concludes by identifying research gaps and proposing an integrated framework that combines interpretability, real-time detection, and generative defense strategies for resilient and accountable fraud monitoring in financial ecosystems.

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

Over the past decade, deep learning (DL) has been successfully applied to many industrial domain-specific tasks. However, the current state-of-the-art DL software still suffers from quality issues, which raises great concern especially in the context of safety- and security-critical scenarios. Adversarial examples (AEs) represent a typical and important type of defects needed to be urgently addressed, on which a DL software makes incorrect decisions. Such defects occur through either intentional attack or physical-world noise perceived by input sensors, potentially hindering further industry deployment. The intrinsic uncertainty nature of deep learning decisions can be a fundamental reason for its incorrect behavior. Although some testing, adversarial attack and defense techniques have been recently proposed, it still lacks a systematic study to uncover the relationship between AEs and DL uncertainty.

Short Message Service (SMS) spam classification systems based on machine learning and deep learning have achieved high accuracy under standard evaluation settings; however, their vulnerability to adversarial attacks poses significant risks to real-world deployment. Attacks on SMS spam classifiers are realized as precision-level adversarial examples, where carefully crafted perturbations (e.g., character substitution/reordering, word disguising and synonym injection, benign token insertion) maintain human readability while drawing models to misclassify. In this paper, we study universal adversarial attacks on SMS spam clasification models and their adaptation to various feature representations and learning architectures: from classic machine learning models to deep neural networks and transformer-based solutions. And it introduces defense terms like adversarial training, data augmentation, ensemble learning, input sanitization and defensive distillation. Primary challenges such as trade-offs between robustness and accuracy, language diversity, and limited computation in mobile contexts are considered. The findings of the paper show possible future research challenges worth addressing for building robust SMS spam detection systems that are still performing effectively under adversarial conditions.

As machine and deep learning models are increasingly leveraged in predictive process monitoring, the focus has shifted towards making these models explainable. The successful adoption of a model is dependent on whether decision-makers can trust the predictions and explanations made. However, recent studies have shown that deep learning models are vulnerable to adversarial attacks -small perturbations to the inputs-which trick deep learning algorithms into making incorrect predictions. An additional crucial property is that the explanations are robust against these adversarial attacks when the model decision was not affected. Therefore, this paper introduces a robustness assessment framework by investigating the impact of adversarial attacks on the robustness of predictive accuracy and explanations used in the field of predictive process monitoring. First, adversarial examples of cases in the independent test set are generated to examine the robustness of the predictive model against intentionally manipulated data. Next, the predictive models are compared with similar models trained on data imputed with adversarial attacks. We monitor the impact on predictive performance in terms of AUC at different stages of the case execution. Finally, the robustness of the explanations is calculated as the distance between the original explanations and the explanations extracted from the model trained on attacked data. We test multiple machine and deep learning techniques, namely the transparent logistic regression, random forests with Shapley values, and LSTM neural networks with attention. Results show that especially neural networks suffer from adversarial attacks, and the former two are mostly robust in terms of both predictive accuracy and explanations.

With the rise of machine learning (ML) in cybersecurity, adversarial attacks targeting ML models pose new challenges to Indian enterprises. This paper explores the vulnerabilities of machine learning models to adversarial attacks, particularly in critical Indian sectors such as finance, healthcare, and government. The study reviews attack vectors such as evasion, poisoning, and inference attacks, and suggests robust mitigation strategies like adversarial training, gradient masking, and defense distillation. Case studies from Indian organizations implementing adversarial-resistant ML models are analyzed to demonstrate real-world applications and effectiveness.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Machine learning models using transaction records as inputs are popular among financial institutions. The most efficient models use deep-learning architectures similar to those in the NLP community, posing a challenge due to their tremendous number of parameters and limited robustness. In particular, deep-learning models are vulnerable to adversarial attacks: a little change in the input harms the model's output. In this work, we examine adversarial attacks on transaction records data and defenses from these attacks. The transaction records data have a different structure than the canonical NLP or time-series data, as neighboring records are less connected than words in sentences, and each record consists of both discrete merchant code and continuous transaction amount. We consider a black-box attack scenario, where the attack doesn't know the true decision model and pay special attention to adding transaction tokens to the end of a sequence. These limitations provide a more realistic scenario, previously unexplored in the NLP world. The proposed adversarial attacks and the respective defenses demonstrate remarkable performance using relevant datasets from the financial industry. Our results show that a couple of generated transactions are sufficient to fool a deep-learning model. Further, we improve model robustness via adversarial training or separate adversarial examples detection. This work shows that embedding protection from adversarial attacks improves model robustness, allowing a wider adoption of deep models for transaction records in banking and finance.

Recent development in the field of Deep Learning have exposed the underlying vulnerability of Deep Neural Network (DNN) against adversarial examples. In image classification, an adversarial example is a carefully modified image that is visually imperceptible to the original image but can cause DNN model to misclassify it. Training the network with Gaussian noise is an effective technique to perform model regularization, thus improving model robustness against input variation. Inspired by this classical method, we explore to utilize the regularization characteristic of noise injection to improve DNN's robustness against adversarial attack. In this work, we propose Parametric-Noise-Injection (PNI) which involves trainable Gaussian noise injection at each layer on either activation or weights through solving the min-max optimization problem, embedded with adversarial training. These parameters are trained explicitly to achieve improved robustness. To the best of our knowledge, this is the first work that uses trainable noise injection to improve network robustness against adversarial attacks, rather than manually configuring the injected noise level through cross-validation. The extensive results show that our proposed PNI technique effectively improves the robustness against a variety of powerful white-box and black-box attacks such as PGD, C & W, FGSM, transferable attack and ZOO attack. Last but not the least, PNI method improves both clean- and perturbed-data accuracy in comparison to the state-of-the-art defense methods, which outperforms current unbroken PGD defense by 1.1 % and 6.8 % on clean test data and perturbed test data respectively using Resnet-20 architecture.

Some recent articles have revealed that synthetic aperture radar automatic target recognition (SAR-ATR) models based on deep learning are vulnerable to the attacks of adversarial examples and cause security problems. The adversarial attack can make a deep convolutional neural network (CNN)-based SAR-ATR system output the intended wrong label predictions by adding small adversarial perturbations to the SAR images. The existing optimization-based adversarial attack methods generate adversarial examples by minimizing the mean-squared reconstruction error, causing smooth target edge and blurry weak scattering centers in SAR images. In this paper, we build a UNet-generative adversarial network (GAN) to refine the generation of the SAR-ATR models’ adversarial examples. The UNet learns the separable features of the targets and generates the adversarial examples of SAR images. The GAN makes the generated adversarial examples approximate to real SAR images (with sharp target edge and explicit weak scattering centers) and improves the generation efficiency. We carry out abundant experiments using the proposed adversarial attack algorithm to fool the SAR-ATR models based on several advanced CNNs, which are trained on the measured SAR images of the ground vehicle targets. The quantitative and qualitative results demonstrate the high-quality adversarial example generation and excellent attack effectiveness and efficiency improvement.

Neural language models show vulnerability to adversarial examples which are semantically similar to their original counterparts with a few words replaced by their synonyms. A common way to improve model robustness is adversarial training which follows two steps—collecting adversarial examples by attacking a target model, and fine-tuning the model on the augmented dataset with these adversarial examples. The objective of traditional adversarial training is to make a model produce the same correct predictions on an original/adversarial example pair. However, the consistency between model decision-makings on two similar texts is ignored. We argue that a robust model should behave consistently on original/adversarial example pairs, that is making the same predictions (what) based on the same reasons (how) which can be reflected by consistent interpretations. In this work, we propose a novel feature-level adversarial training method named FLAT. FLAT aims at improving model robustness in terms of both predictions and interpretations. FLAT incorporates variational word masks in neural networks to learn global word importance and play as a bottleneck teaching the model to make predictions based on important words. FLAT explicitly shoots at the vulnerability problem caused by the mismatch between model understandings on the replaced words and their synonyms in original/adversarial example pairs by regularizing the corresponding global word importance scores. Experiments show the effectiveness of FLAT in improving the robustness with respect to both predictions and interpretations of four neural network models (LSTM, CNN, BERT, and DeBERTa) to two adversarial attacks on four text classification tasks. The models trained via FLAT also show better robustness than baseline models on unforeseen adversarial examples across different attacks.