The Many Faces of Adversarial Machine Learning

We present a new algorithm to train a robust malware detector. Malware is a prolific problem and malware detectors are a front-line defense. Modern detectors rely on machine learning algorithms. Now, the adversarial objective is to devise alterations to the malware code to decrease the chance of being detected whilst preserving the functionality and realism of the malware. Adversarial learning is effective in improving robustness but generating functional and realistic adversarial malware samples is non-trivial. Because: i) in contrast to tasks capable of using gradient-based feedback, adversarial learning in a domain without a differentiable mapping function from the problem space (malware code inputs) to the feature space is hard; and ii) it is difficult to ensure the adversarial malware is realistic and functional. This presents a challenge for developing scalable adversarial machine learning algorithms for large datasets at a production or commercial scale to realize robust malware detectors. We propose an alternative; perform adversarial learning in the feature space in contrast to the problem space. We prove the projection of perturbed, yet valid malware, in the problem space into feature space will always be a subset of adversarials generated in the feature space. Hence, by generating a robust network against feature-space adversarial examples, we inherently achieve robustness against problem-space adversarial examples. We formulate a Bayesian adversarial learning objective that captures the distribution of models for improved robustness. To explain the robustness of the Bayesian adversarial learning algorithm, we prove that our learning method bounds the difference between the adversarial risk and empirical risk and improves robustness. We show that Bayesian neural networks (BNNs) achieve state-of-the-art results; especially in the False Positive Rate (FPR) regime. Adversarially trained BNNs achieve state-of-the-art robustness. Notably, adversarially trained BNNs are robust against stronger attacks with larger attack budgets by a margin of up to 15% on a recent production-scale malware dataset of more than 20 million samples. Importantly, our efforts create a benchmark for future defenses in the malware domain.

Machine learning is one of the predominant methods used in computer science and has been widely and successfully applied in many areas such as computer vision, pattern recognition, natural language processing, cyber security etc. In cyber security, the application of machine learning algorithms for network intrusion detection system (NIDS) has seen promising results for anomaly detection mostly with the adoption of deep learning and is still growing. However, machine learning algorithms are vulnerable to adversarial attacks resulting in significant performance degradation. Adversarial attacks are security threats that aim to deceive the learning algorithm by manipulating its predictions, and Adversarial machine learning is a research area that studies both the generation and defense of such attacks. Researchers have extensively worked on the adversarial machine learning in computer vision but not many works in Intrusion detection system. However, failure in this critical Intrusion detection area could compromise the security of an entire system, and need much attention. This paper provides a review of the advancement in adversarial machine learning based intrusion detection and explores the various defense techniques applied against. Finally discuss their limitations for future research direction in this emerging area.KeywordsAdversarial attackCyber securityIntrusion detectionMachine learningDeep learningPoisoning attackEvasion attack

Machine learning (ML) provides effective means to learn from spectrum data and solve complex tasks involved in wireless communications. Supported by recent advances in computational resources and algorithmic designs, deep learning (DL) has found success in performing various wireless communication tasks such as signal recognition, spectrum sensing and waveform design. However, ML in general and DL in particular have been found vulnerable to manipulations thus giving rise to a field of study called adversarial machine learning (AML). Although AML has been extensively studied in other data domains such as computer vision and natural language processing, research for AML in the wireless communications domain is still in its early stage. This paper presents a comprehensive review of the latest research efforts focused on AML in wireless communications while accounting for the unique characteristics of wireless systems. First, the background of AML attacks on deep neural networks is discussed and a taxonomy of AML attack types is provided. Various methods of generating adversarial examples and attack mechanisms are also described. In addition, an holistic survey of existing research on AML attacks for various wireless communication problems as well as the corresponding defense mechanisms in the wireless domain are presented. Finally, as new attacks and defense techniques are developed, recent research trends and the overarching future outlook for AML in next-generation wireless communications are discussed.

Machine learning algorithms provide the ability to quickly adapt and find patterns in large diverse data sources and therefore are a potential asset to application developers in enterprise systems, networks, and security domains. They make analyzing the security implications of these tools a critical task for machine learning researchers and practitioners alike, spawning a new subfield of research into adversarial learning for security-sensitive domains. The work presented in this book advanced the state of the art in this field of study with five primary contributions: a taxonomy for qualifying the security vulnerabilities of a learner, two novel practical attack/defense scenarios for learning in real-world settings, learning algorithms with theoretical guarantees on training-data privacy preservation, and a generalization of a theoretical paradigm for evading detection of a classifier. However, research in adversarial machine learning has only begun to address the field's complex obstacles—many challenges remain. These challenges suggest several new directions for research within both fields of machine learning and computer security. In this chapter we review our contributions and list a number of open problems in the area. Above all, we investigated both the practical and theoretical aspects of applying machine learning in security domains. To understand potential threats, we analyzed the vulnerability of learning systems to adversarial malfeasance. We studied both attacks designed to optimally affect the learning system and attacks constrained by real-world limitations on the adversary's capabilities and information.We further designed defense strategies, which we showed significantly diminish the effect of these attacks. Our research focused on learning tasks in virus, spam, and network anomaly detection, but also is broadly applicable across many systems and security domains and has farreaching implications to any system that incorporates learning. Here is a summary of the contributions of each component of this book followed by a discussion of open problems and future directions for research. Framework for Secure Learning The first contribution discussed in this book was a framework for assessing risks to a learner within a particular security context (see Table 3.1). The basis for this work is a taxonomy of the characteristics of potential attacks. From this taxonomy (summarized in Table 9.1), we developed security games between an attacker and defender tailored to the particular type of threat posed by the attacker.

Cyber Physical Systems (CPS) are characterized by their ability to integrate\nthe physical and information or cyber worlds. Their deployment in critical\ninfrastructure have demonstrated a potential to transform the world. However,\nharnessing this potential is limited by their critical nature and the far\nreaching effects of cyber attacks on human, infrastructure and the environment.\nAn attraction for cyber concerns in CPS rises from the process of sending\ninformation from sensors to actuators over the wireless communication medium,\nthereby widening the attack surface. Traditionally, CPS security has been\ninvestigated from the perspective of preventing intruders from gaining access\nto the system using cryptography and other access control techniques. Most\nresearch work have therefore focused on the detection of attacks in CPS.\nHowever, in a world of increasing adversaries, it is becoming more difficult to\ntotally prevent CPS from adversarial attacks, hence the need to focus on making\nCPS resilient. Resilient CPS are designed to withstand disruptions and remain\nfunctional despite the operation of adversaries. One of the dominant\nmethodologies explored for building resilient CPS is dependent on machine\nlearning (ML) algorithms. However, rising from recent research in adversarial\nML, we posit that ML algorithms for securing CPS must themselves be resilient.\nThis paper is therefore aimed at comprehensively surveying the interactions\nbetween resilient CPS using ML and resilient ML when applied in CPS. The paper\nconcludes with a number of research trends and promising future research\ndirections. Furthermore, with this paper, readers can have a thorough\nunderstanding of recent advances on ML-based security and securing ML for CPS\nand countermeasures, as well as research trends in this active research area.\n

Machine learning algorithms represent the intelligence that controls many information systems and applications around us. As such, they are targeted by attackers to impact their decisions. Text created by machine learning algorithms has many types of applications, some of which can be considered malicious especially if there is an intention to present machine-generated text as human-generated. In this paper, we surveyed major subjects in adversarial machine learning for text processing applications. Unlike adversarial machine learning in images, text problems and applications are heterogeneous. Thus, each problem can have its own challenges. We focused on some of the evolving research areas such as: malicious versus genuine text generation metrics, defense against adversarial attacks, and text generation models and algorithms. Our study showed that as applications of text generation will continue to grow in the near future, the type and nature of attacks on those applications and their machine learning algorithms will continue to grow as well. Literature survey indicated an increasing trend in using pre-trained models in machine learning. Word/sentence embedding models and transformers are examples of those pre-trained models. Adversarial models may utilize same or similar pre-trained models as well. In another trend related to text generation models, literature showed effort to develop universal text perturbations to be used in both black-and white-box attack settings. Literature showed also using conditional GANs to create latent representation for writing types. This usage will allow for a seamless lexical and grammatical transition between various writing styles. In text generation metrics, research trends showed developing successful automated or semi-automated assessment metrics that may include human judgement. Literature showed also research trends of designing and developing new memory models that increase performance and memory utilization efficiency without validating real-time constraints. Many research efforts evaluate different defense model approaches and algorithms. Researchers evaluated different types of targeted attacks, and methods to distinguish human versus machine generated text.

Machine Learning (ML) and Deep Learning (DL) models are increasingly deployed in critical domains such as computer vision, natural language processing (NLP), automatic speech recognition (ASR), time-series analysis, and cybersecurity. Despite their success, these models remain vulnerable to adversarial attacks that can mislead predictions and undermine system reliability. To address this challenge, we conduct a Systematic Literature Review (SLR) of adversarial attacks and defenses in ML/DL, following PRISMA guidelines. Our review analyzes 132 peer-reviewed studies published between 2017 and 2025, identified from leading digital libraries including IEEE, ACM, ScienceDirect, and SpringerLink. The objectives of this study are to: (i) identify state-of-the-art adversarial attack and defense techniques and assess their strengths and limitations; (ii) evaluate how novel threats and countermeasures are addressed across domains; and (iii) propose a unified taxonomy that integrates adversarial attacks, defenses, evaluation metrics, and application domains. Our findings show that adversarial training and certified robustness dominate defense research, but both face trade-offs in scalability and generalizability. Computer vision remains the most studied domain, while NLP, ASR, and time-series research is growing but lacks standardized benchmarks. This review contributes a taxonomy and cross-domain synthesis of adversarial ML research, highlights research gaps, and outlines future directions to advance robustness and support the development of trustworthy AI systems.

In recent years, machine learning technology has made great improvements in social networks applications such as social network recommendation systems, sentiment analysis, and text generation. However, it cannot be ignored that machine learning algorithms are vulnerable to adversarial examples, that is, adding perturbations that are imperceptible to the human eye to the original data can cause machine learning algorithms to make wrong outputs with high probability. This also restricts the widespread use of machine learning algorithms in real life. In this paper, we focus on adversarial machine learning algorithms on social networks in recent years from three aspects: sentiment analysis, recommendation system, and spam detection, We review some typical applications of machine learning algorithms and adversarial example generation and defense algorithms for machine learning algorithms in the above three aspects in recent years. besides, we also analyze the current research progress and prospects for the directions of future research.

The field of machine learning artificial intelligence algorithms has made significant progress. With the development of the times, the security brought by machine learning is worthy of consideration, and adversarial machine learning improves the reliability of machine learning algorithms through continuous training algorithms. Hence, the research of adversarial machine learning based on network security is well worth exploring.

Learning an effective distance measurement between sample pairs plays an important role in visual analysis, where the training procedure largely relies on hard negative samples. However, hard negative samples usually account for the tiny minority in the training set, which may fail to fully describe the data distribution close to the decision boundary. In this paper, we present a deep adversarial metric learning (DAML) framework to generate synthetic hard negatives from the original negative samples, which is widely applicable to existing supervised deep metric learning algorithms. Different from existing sampling strategies which simply ignore numerous easy negatives, our DAML aim to exploit them by generating synthetic hard negatives adversarial to the learned metric as complements. We simultaneously train the feature embedding and hard negative generator in an adversarial manner, so that adequate and targeted synthetic hard negatives are created to learn more precise distance metrics. As a single transformation may not be powerful enough to describe the global input space under the attack of the hard negative generator, we further propose a deep adversarial multi-metric learning (DAMML) method by learning multiple local transformations for more complete description. We simultaneously exploit the collaborative and competitive relationships among multiple metrics, where the metrics display unity against the generator for effective distance measurement as well as compete for more training data through a metric discriminator to avoid overlapping. Extensive experimental results on five benchmark datasets show that our DAML and DAMML effectively boost the performance of existing deep metric learning approaches through adversarial learning.

Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems

Recursive reasoning-based training-time adversarial machine learning

In the era of artificial intelligence, machine learning methods are successfully used in various fields. Machine learning has attracted extensive attention from investors in the financial market, especially in stock price prediction. However, one argument for the machine learning methods used in stock price prediction is that they are black-box models which are difficult to interpret. In this paper, we focus on the future stock price prediction with the historical stock price by machine learning and deep learning methods, such as support vector machine (SVM), random forest (RF), Bayesian classifier (BC), decision tree (DT), multilayer perceptron (MLP), convolutional neural network (CNN), bi-directional long-short term memory (BiLSTM), the embedded CNN, and the embedded BiLSTM. Firstly, we manually design several financial time series where the future price correlates with the historical stock prices in pre-designed modes, namely the curve-shape-feature (CSF) and the non-curve-shape-feature (NCSF) modes. In the CSF mode, the future prices can be extracted from the curve shapes of the historical stock prices. Conversely, in the NCSF mode, they can’t. Secondly, we apply various algorithms to those pre-designed and real financial time series. We find that the existing machine learning and deep learning algorithms fail in stock price prediction because in the real financial time series, less information of future prices is contained in the CSF mode, and perhaps more information is contained in the NCSF. Various machine learning and deep learning algorithms are good at handling the CSF in historical data, which are successfully applied in image recognition and natural language processing. However, they are inappropriate for stock price prediction on account of the NCSF. Therefore, accurate stock price prediction is the key to successful investment, and new machine learning algorithms handling the NCSF series are needed.

Untargeted white-box adversarial attack with heuristic defence methods in real-time deep learning based network intrusion detection system

Aim: Machine learning (ML) and deep learning (DL) predictive models have been employed widely in clinical settings. Their potential support and aid to the clinician of providing an objective measure that can be shared among different centers enables the possibility of building more robust multicentric studies. This study aimed to propose a user-friendly and low-cost tool for COVID-19 mortality prediction using both an ML and a DL approach. Method: We enrolled 2348 patients from several hospitals in the Province of Reggio Emilia. Overall, 19 clinical features were provided by the Radiology Units of Azienda USL-IRCCS of Reggio Emilia, and 5892 radiomic features were extracted from each COVID-19 patient’s high-resolution computed tomography. We built and trained two classifiers to predict COVID-19 mortality: a machine learning algorithm, or support vector machine (SVM), and a deep learning model, or feedforward neural network (FNN). In order to evaluate the impact of the different feature sets on the final performance of the classifiers, we repeated the training session three times, first using only clinical features, then employing only radiomic features, and finally combining both information. Results: We obtained similar performances for both the machine learning and deep learning algorithms, with the best area under the receiver operating characteristic (ROC) curve, or AUC, obtained exploiting both clinical and radiomic information: 0.803 for the machine learning model and 0.864 for the deep learning model. Conclusions: Our work, performed on large and heterogeneous datasets (i.e., data from different CT scanners), confirms the results obtained in the recent literature. Such algorithms have the potential to be included in a clinical practice framework since they can not only be applied to COVID-19 mortality prediction but also to other classification problems such as diabetic prediction, asthma prediction, and cancer metastases prediction. Our study proves that the lesion’s inhomogeneity depicted by radiomic features combined with clinical information is relevant for COVID-19 mortality prediction.