Adversarial machine learning and securing AI systems

Untargeted white-box adversarial attack with heuristic defence methods in real-time deep learning based network intrusion detection system

Machine Learning (ML) and Deep Learning (DL) models are increasingly deployed in critical domains such as computer vision, natural language processing (NLP), automatic speech recognition (ASR), time-series analysis, and cybersecurity. Despite their success, these models remain vulnerable to adversarial attacks that can mislead predictions and undermine system reliability. To address this challenge, we conduct a Systematic Literature Review (SLR) of adversarial attacks and defenses in ML/DL, following PRISMA guidelines. Our review analyzes 132 peer-reviewed studies published between 2017 and 2025, identified from leading digital libraries including IEEE, ACM, ScienceDirect, and SpringerLink. The objectives of this study are to: (i) identify state-of-the-art adversarial attack and defense techniques and assess their strengths and limitations; (ii) evaluate how novel threats and countermeasures are addressed across domains; and (iii) propose a unified taxonomy that integrates adversarial attacks, defenses, evaluation metrics, and application domains. Our findings show that adversarial training and certified robustness dominate defense research, but both face trade-offs in scalability and generalizability. Computer vision remains the most studied domain, while NLP, ASR, and time-series research is growing but lacks standardized benchmarks. This review contributes a taxonomy and cross-domain synthesis of adversarial ML research, highlights research gaps, and outlines future directions to advance robustness and support the development of trustworthy AI systems.

Machine Learning-based Intrusion Detection Systems have been proven to be very effective in the protection of IoT Networks. However, the expansion of Adversarial Machine Learning attacks threatens their efficacy affecting also the security of IoT networks. Thus, this paper proposes a Machine Learning-driven methodology for multiclass classification of cyber-attacks in IoT networks and investigates the robustness of the Machine and Deep Learning classifiers against several well-known Adversarial Machine Learning attacks (JSMA, FGSM, DeepFool). Moreover, the effectiveness of the Adversarial Training defense method has been studied in tackling Adversarial Machine Learning attacks. The proposed methodology was evaluated using a new and large IoT dataset (IoTID20) and the experimental results concluded that the Random Forest classifier can classify the cyber-attacks with high classification accuracy (99.9%) as well as the JSMA, FGSM, and DeepFool attacks can significantly reduce the performance of all the classifiers. Finally, based on the evaluation adversarial training can overall enhance the classifiers’ robustness against all the utilized Adversarial Machine Learning attacks without affecting the performance when only normal samples are present.

The Internet of Flying Things (IoFT) holds significant promise in fields like disaster management and surveillance. However, it is increasingly vulnerable to cyberattacks that can compromise the confidentiality, integrity, and availability (CIA) of sensitive data. Despite the growing interest in proposing Intrusion Detection Systems (IDSs) for IoFT networks, current literature faces key limitations, particularly the shortage of publicly available IoFT datasets with diverse attacks, and the fact that existing IDSs lack robustness against sophisticated adversarial machine learning attacks. This paper is the first study to address these limitations by proposing a more resilient and accurate IDS tailored for IoFT networks (RIDS-IoFT). We introduce a novel IDS that leverages Generative Adversarial Networks (GANs) to generate a hybrid dataset that combines real IoFT traffic data with GAN-generated adversarial attacks, addressing the dataset diversity issue. Additionally, we introduce an innovative adversarial training method to enhance the system’s defense against evolving threats, such as Fast Gradient Sign Method (FGSM), Basic Iterative Method (BIM), and Carlini & Wagner (C&W) attacks. The proposed RIDS-IoFT was evaluated using four machine learning models, Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), and Logistic Regression (LR), on two datasets: ECU-IoFT and CICIDS2018. The IDS’s performance was assessed based on its ability to detect both traditional and adversarial attacks. The results show that the Random Forest model achieved the highest detection accuracy, up to 96.5%, demonstrating superior performance across both real and hybrid datasets. The proposed RIDS-IoFT not only enhances detection accuracy but also strengthens resilience against adversarial threats, making it suitable for resource-constrained IoFT environments. In conclusion, this study presents a comprehensive approach to securing IoFT networks by combining real and synthetic data, improving IDS robustness and accuracy against both traditional and adversarial attacks.

We study two important concepts in adversarial deep learning---adversarial training and generative adversarial network (GAN). Adversarial training is the technique used to improve the robustness of discriminator by combining adversarial attacker and discriminator in the training phase. GAN is commonly used for image generation by jointly optimizing discriminator and generator. We show these two concepts are indeed closely related and can be used to strengthen each other---adding a generator to the adversarial training procedure can improve the robustness of discriminators, and adding an adversarial attack to GAN training can improve the convergence speed and lead to better generators. Combining these two insights, we develop a framework called Rob-GAN to jointly optimize generator and discriminator in the presence of adversarial attacks---the generator generates fake images to fool discriminator; the adversarial attacker perturbs real images to fool the discriminator, and the discriminator wants to minimize loss under fake and adversarial images. Through this end-to-end training procedure, we are able to simultaneously improve the convergence speed of GAN training, the quality of synthetic images, and the robustness of discriminator under strong adversarial attacks. Experimental results demonstrate that the obtained classifier is more robust than the state-of-the-art adversarial training approach, and the generator outperforms SN-GAN on ImageNet-143.

With the rise of machine learning (ML) in cybersecurity, adversarial attacks targeting ML models pose new challenges to Indian enterprises. This paper explores the vulnerabilities of machine learning models to adversarial attacks, particularly in critical Indian sectors such as finance, healthcare, and government. The study reviews attack vectors such as evasion, poisoning, and inference attacks, and suggests robust mitigation strategies like adversarial training, gradient masking, and defense distillation. Case studies from Indian organizations implementing adversarial-resistant ML models are analyzed to demonstrate real-world applications and effectiveness.

Adversarial Machine Learning (AML) is an essential research area because of its impact on the security and trustworthiness of artificial intelligence (AI) systems. Attackers take advantage of weaknesses in machine learning (ML) models by injecting well-designed perturbations, resulting in misclassifications, unauthorized access, and security violations. These attacks pose significant risks in high-stakes applications such as cybersecurity, healthcare, autonomous vehicles, and IoT, where adversarial manipulations can lead to severe financial, operational, and safety consequences. This paper presents a thorough review of adversarial attacks and defenses, classifying attacks into three main paradigms: backdoor attacks, weight attacks, and adversarial examples. These attacks occur at various points in the ML life cycle, threatening real- world applications by compromising model integrity, degrading performance, and enabling malicious exploitation. To counter these threats, several defense mechanisms have been investigated, ranging from adversarial training and input preprocessing to robust model architectures. Despite these efforts, existing solutions still struggle with computational complexity, scalability, and adaptability against evolving adversarial strategies. This work further explores the efficacy of AML defenses across different domains, identifying key challenges and open research questions, including the accuracy- robustness trade-off, security vulnerabilities in black-box AI models, and the continuous evolution of adversarial tactics. By analyzing current advancements and identifying gaps in AML research, this study contributes to the development of more resilient AI models capable of withstanding adversarial threats. The findings emphasize the need for adaptive and proactive security measures, calling for scalable, real-time adversarial detection techniques and multi-layered security frameworks to safeguard AI systems against emerging adversarial techniques.

The generative adversarial imitation learning (GAIL) has provided an adversarial learning framework for imitating expert policy from demonstrations in high-dimensional continuous tasks. However, almost all GAIL and its extensions only design a kind of reward function of logarithmic form in the adversarial training strategy with the Jensen-Shannon (JS) divergence for all complex environments. The fixed logarithmic type of reward function may be difficult to solve all complex tasks, and the vanishing gradients problem caused by the JS divergence will harm the adversarial learning process. In this paper, we propose a new algorithm named Wasserstein Distance guided Adversarial Imitation Learning (WDAIL) for promoting the performance of imitation learning (IL). There are three improvements in our method: (a) introducing the Wasserstein distance to obtain more appropriate measure in the adversarial training process, (b) using proximal policy optimization (PPO) in the reinforcement learning stage which is much simpler to implement and makes the algorithm more efficient, and (c) exploring different reward function shapes to suit different tasks for improving the performance. The experiment results show that the learning procedure remains remarkably stable, and achieves significant performance in the complex continuous control tasks of MuJoCo.

In this study, we investigate whether noise-augmented training can concurrently improve adversarial robustness in automatic speech recognition (ASR) systems. We conduct a comparative analysis of the adversarial robustness of four different ASR architectures, each trained under three different augmentation conditions: (1) background noise, speed variations, and reverberations; (2) speed variations only; (3) no data augmentation. We then evaluate the robustness of all resulting models against attacks with white-box or black-box adversarial examples. Our results demonstrate that noise augmentation not only enhances model performance on noisy speech but also improves the model’s robustness to adversarial attacks. • Comparative analysis: We examine for state-of-the-art automatic speech recognition (ASR) architectures under three different training conditions and evaluate their robustness with respect to white- and black-box adversarial attacks. • Practical implications: Our work has direct practical applications, as it shows that noise-augmented training, which is much cheeper than adversarial training, already helps in reducing the risks for adversarial attacks. This has not been explored in the context of automatic speech recognition yet and, thus, is a relevant contribution. • Contribution to security and privacy: Our paper provides necessary and useful insights to improve robustness against adversarial attacks.

Machine learning (ML) provides effective means to learn from spectrum data and solve complex tasks involved in wireless communications. Supported by recent advances in computational resources and algorithmic designs, deep learning (DL) has found success in performing various wireless communication tasks such as signal recognition, spectrum sensing and waveform design. However, ML in general and DL in particular have been found vulnerable to manipulations thus giving rise to a field of study called adversarial machine learning (AML). Although AML has been extensively studied in other data domains such as computer vision and natural language processing, research for AML in the wireless communications domain is still in its early stage. This paper presents a comprehensive review of the latest research efforts focused on AML in wireless communications while accounting for the unique characteristics of wireless systems. First, the background of AML attacks on deep neural networks is discussed and a taxonomy of AML attack types is provided. Various methods of generating adversarial examples and attack mechanisms are also described. In addition, an holistic survey of existing research on AML attacks for various wireless communication problems as well as the corresponding defense mechanisms in the wireless domain are presented. Finally, as new attacks and defense techniques are developed, recent research trends and the overarching future outlook for AML in next-generation wireless communications are discussed.

CardioDefense: Defending against adversarial attack in ECG classification with adversarial distillation training

Machine learning is one of the predominant methods used in computer science and has been widely and successfully applied in many areas such as computer vision, pattern recognition, natural language processing, cyber security etc. In cyber security, the application of machine learning algorithms for network intrusion detection system (NIDS) has seen promising results for anomaly detection mostly with the adoption of deep learning and is still growing. However, machine learning algorithms are vulnerable to adversarial attacks resulting in significant performance degradation. Adversarial attacks are security threats that aim to deceive the learning algorithm by manipulating its predictions, and Adversarial machine learning is a research area that studies both the generation and defense of such attacks. Researchers have extensively worked on the adversarial machine learning in computer vision but not many works in Intrusion detection system. However, failure in this critical Intrusion detection area could compromise the security of an entire system, and need much attention. This paper provides a review of the advancement in adversarial machine learning based intrusion detection and explores the various defense techniques applied against. Finally discuss their limitations for future research direction in this emerging area.KeywordsAdversarial attackCyber securityIntrusion detectionMachine learningDeep learningPoisoning attackEvasion attack

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

Machine learning algorithms represent the intelligence that controls many information systems and applications around us. As such, they are targeted by attackers to impact their decisions. Text created by machine learning algorithms has many types of applications, some of which can be considered malicious especially if there is an intention to present machine-generated text as human-generated. In this paper, we surveyed major subjects in adversarial machine learning for text processing applications. Unlike adversarial machine learning in images, text problems and applications are heterogeneous. Thus, each problem can have its own challenges. We focused on some of the evolving research areas such as: malicious versus genuine text generation metrics, defense against adversarial attacks, and text generation models and algorithms. Our study showed that as applications of text generation will continue to grow in the near future, the type and nature of attacks on those applications and their machine learning algorithms will continue to grow as well. Literature survey indicated an increasing trend in using pre-trained models in machine learning. Word/sentence embedding models and transformers are examples of those pre-trained models. Adversarial models may utilize same or similar pre-trained models as well. In another trend related to text generation models, literature showed effort to develop universal text perturbations to be used in both black-and white-box attack settings. Literature showed also using conditional GANs to create latent representation for writing types. This usage will allow for a seamless lexical and grammatical transition between various writing styles. In text generation metrics, research trends showed developing successful automated or semi-automated assessment metrics that may include human judgement. Literature showed also research trends of designing and developing new memory models that increase performance and memory utilization efficiency without validating real-time constraints. Many research efforts evaluate different defense model approaches and algorithms. Researchers evaluated different types of targeted attacks, and methods to distinguish human versus machine generated text.

Smart city applications that request sensitive user information necessitate a comprehensive data privacy solution. Federated learning (FL), also known as privacy by design, is a new paradigm in machine learning (ML). However, FL models are susceptible to adversarial attacks, similar to other AI models. In this paper, we propose federated adversarial training (FAT) strategies to generate robust global models that are resistant to adversarial attacks. We apply two adversarial attack methods, projected gradient descent (PGD) and the fast gradient sign method (FGSM), to our air pollution dataset to generate adversarial samples. We then evaluate the effectiveness of our FAT strategies in defending against these attacks. Our experiments show that FGSM-based adversarial attacks have a negligible impact on the accuracy of global models, while PGD-based attacks are more effective. However, we also show that our FAT strategies can make global models robust enough to withstand even PGD-based attacks. For example, the accuracy of our FAT-PGD and FL-mixed-PGD models is 81.13% and 82.60%, respectively, compared to 91.34% for the baseline FL model. This represents a reduction in accuracy of 10%, but this could be potentially mitigated by using a more complex and larger model. Our results demonstrate that FAT can enhance the security and privacy of sustainable smart city applications. We also show that it is possible to train robust global models from modest datasets per client, which challenges the conventional wisdom that adversarial training requires massive datasets.