Efficient Discovery of Abnormal Event Sequences in Enterprise Security Systems

Malware authors are continuously developing crime toolkits. This has led to the situation of zero-day attacks, where malware harm computer systems despite the protection from existing Intrusion Detection Systems (IDSs). We propose an Intelligent Intrusion Detection and Prevention System (IIDPS) approach that combines the Signature based Intrusion Detection system (SIDS), Anomaly based Intrusion Detection System (AIDS) and Response Intrusion Detection System (RIDS). We used a risk assessment approach to determine an appropriate response action against each attack event. We also demonstrated the IIDPS make the detection and prevention of malware more effective.

The novelty and growing sophistication of cyber threats mean that high accuracy and interpretable machine learning models are needed more than ever before for Intrusion Detection and Prevention Systems. This study aims to solve this challenge by applying Explainable AI techniques, including Shapley Additive explanations feature selection, to improve model performance, robustness, and transparency. The method systematically employs different classifiers and proposes a new hybrid method called Hybrid Bagging-Boosting and Boosting on Residuals. Then, performance is taken in four steps: the multistep evaluation of hybrid ensemble learning methods for binary classification and fine-tuning of performance; feature selection using Shapley Additive explanations values retraining the hybrid model for better performance and reducing overfitting; the generalization of the proposed model for multiclass classification; and the evaluation using standard information metrics such as accuracy, precision, recall, and F1-score. Key results indicate that the proposed methods outperform state-of-the-art algorithms, achieving a peak accuracy of 98.47% and an F1 score of 96.19%. These improvements stem from advanced feature selection and resampling techniques, enhancing model accuracy and balancing precision and recall. Integrating Shapley Additive explanations-based feature selection with hybrid ensemble methods significantly boosts the predictive and explanatory power of Intrusion Detection and Prevention Systems, addressing common pitfalls in traditional cybersecurity models. This study paves the way for further research on statistical innovations to enhance Intrusion Detection and Prevention Systems performance.

With the increasing rate and types of cyber attacks against information systems and communication infrastructures, many tools are needed to detect and mitigate against such attacks, for example, Intrusion Detection Systems (IDSs). Unfortunately, traditional Signature‐based IDSs (SIDSs) perform poorly against previously unseen adversarial attacks. Anomaly‐based IDSs (AIDSs) use Machine Learning (ML) and Deep Learning (DL) approaches to overcome these limitations. However, AIDS performance can be poor when trained on imbalanced datasets. To address the challenge of AIDS performance caused by these unbalanced training datasets, generative adversarial models are proposed to obtain adversarial attacks from one side and analyse their quality from another. According to extensive usage and reliability criteria for generative adversarial models in different disciplines, Generative Adversarial Networks (GANs) , Bidirectional GAN (BiGAN) , and Wasserstein GAN (WGAN) are employed to serve AIDS. The authors have extensively assessed their abilities and robustness to deliver high‐quality attacks for AIDS. AIDSs are constructed, trained, and tuned based on these models to measure their impacts. The authors have employed two datasets: NSL‐KDD and CICIDS‐2017 for generalisation purposes, where ML and DL approaches are utilised to implement AIDSs. Their results show that the WGAN model outperformed GANs and BiGAN models in binary and multiclass classifications for both datasets.

Chapter 5 - Intrusion Prevention and Detection Systems

An Intrusion Detection System (IDS) is a software used to monitor a single computer or a network of computers to combat network attacks, block access to certain websites, or otherwise the attack on network protocols. The dynamic and complicated nature of cyber attacks on computer systems is beyond the capabilities of the majority of current IDS solutions. By combating intrusion attempts, network security professionals ensure that the services can be accessed at all times. One of the tools at disposal to detect and categorize suspicious behaviour is IDS. For almost 40 years, researchers in both academia and industry have been proposing strategies for detecting and avoiding such security breaches, and even constructing systems to do so. Intrusion detection systems that rely on attack signatures examine network traffic for suspicious activity. Anomaly-based intrusion detection systems, on the other hand, build a model to tell good users apart from bad ones, allowing them to spot previously unseen attacks. Machine learning (ML) techniques are increasingly being employed as a means of categorizing normal and abnormal activities. Many other ML-based intrusion detection systems have been proposed in the past few years. The purpose of this work is to provide a critical overview of the state of the art in machine learning (ML)-based intrusion detection methods. Researchers focusing on ML-based systems that detect intrusions could use this survey as a supplement to previous generic surveys on intrusion detection and a source to recent work done in the area.

An intrusion detection system (IDS) attempts to identify attacks by comparing collected data to predefined signatures known to be malicious (signature-based IDS) or to a model of legal behaviour (anomaly-based IDS). Anomaly-based approaches have the advantage of being able to detect previously unknown attacks, but they suffer from the difficulty of building robust models of acceptable behaviour which may result in a large number of false alarms. Two reasons for the large number of false alarms, caused by incorrect classification of events in current systems, one is the simplistic aggregation of model outputs inthe decision phase. The other reason is the lack of integration of additional information into the decision process. To mitigate these shortcomings, this paper proposes a two stratum Bayesian networks based anomaly detection and decision model for intrusion detection system. Bayesian networks improve the aggregation of outputs, such as empirical data and allow one to seamlessly incorporate additional information. Experimental results clearly demonstrate the efficiency of our approach to improve the accuracy of the intrusion detection and decision process in an anomaly based IDS.

With the rapid expansion of digital networks and the increasing complexity of cyber threats, network security has emerged as a crucial field within information and communications technology (ICT). Modern organizations face growing risks from sophisticated cyber-attacks that threaten the confidentiality, integrity, and availability of critical systems. Intrusion Detection Systems (IDS) have become a foundational tool for identifying and mitigating unauthorized access and suspicious activities within networks. These systems are broadly categorized into Signature-Based Intrusion Detection Systems (SIDS), which rely on known threat patterns, and Anomaly-Based Intrusion Detection Systems (AIDS), which detect deviations from normal behavior to identify unknown attacks. Despite their strengths, both types face significant limitations, including high false-positive rates and challenges in real-time detection. This study explores the evolving landscape of network security threats and critically analyzes existing intrusion detection techniques. It highlights the integration of alert correlation and prediction mechanisms that enhance early detection and response capabilities. Furthermore, the study addresses key limitations and challenges in IDS implementation and identifies emerging trends aimed at improving scalability, intelligence, and adaptability of detection mechanisms. By evaluating current systems and proposing directions for future research, this work contributes to the development of more robust and proactive network defense strategies in an increasingly interconnected digital world.

The growing development of IoT (Internet of Things) devices creates a large attack surface for cybercriminals to conduct potentially more destructive cyberattacks; as a result, the security industry has seen an exponential increase in cyber-attacks. Many of these attacks have effectively accomplished their malicious goals because intruders conduct cyber-attacks using novel and innovative techniques. An anomaly-based IDS (Intrusion Detection System) uses machine learning techniques to detect and classify attacks in IoT networks. In the presence of unpredictable network technologies and various intrusion methods, traditional machine learning techniques appear inefficient. In many research areas, deep learning methods have shown their ability to identify anomalies accurately. Convolutional neural networks are an excellent alternative for anomaly detection and classification due to their ability to automatically categorize main characteristics in input data and their effectiveness in performing faster computations. In this paper, we design and develop a novel anomaly-based intrusion detection model for IoT networks. First, a convolutional neural network model is used to create a multiclass classification model. The proposed model is then implemented using convolutional neural networks in 1D, 2D, and 3D. The proposed convolutional neural network model is validated using the BoT-IoT, IoT Network Intrusion, MQTT-IoT-IDS2020, and IoT-23 intrusion detection datasets. Transfer learning is used to implement binary and multiclass classification using a convolutional neural network multiclass pre-trained model. Our proposed binary and multiclass classification models have achieved high accuracy, precision, recall, and F1 score compared to existing deep learning implementations.

With the advent of anomaly-based intrusion detection systems, many approaches and techniques have been developed to track novel attacks on the systems.High detection rate of 98% at a low alarm rate of 1% can be achieved by using these techniques.Though anomaly-based approaches are efficient, signature-based detection is preferred for mainstream implementation of intrusion detection systems.As a variety of anomaly detection techniques were suggested, it is difficult to compare the strengths, weaknesses of these methods.The reason why industries don"t favor the anomaly-based intrusion detection methods can be well understood by validating the efficiencies of the all the methods.To investigate this issue, the current state of the experiment practice in the field of anomalybased intrusion detection is reviewed and survey recent studies in this.This paper contains summarization study and identification of the drawbacks of formerly surveyed works.

With the wide application of information technolo-gies in communication based train control (CB TC) systems, the information security risks it confronts to are ever increasing. It is very difficult to design an intrusion detection system (IDS) which can detect both known and unknown cyber attacks with high true positive rate (TPR) and low false positive rate (FPR), simultaneously. In this paper, we propose a hybrid intrusion detection system (RIDS) which is composed of a signature-based intrusion detection system (SIDS) and an anomaly-based intrusion detection system (AIDS). An improved stacking ensemble algorithm (ISEA) is proposed to enhance the TPR of the SIDS which uses weighted classification probabilities outputted by base-classifiers to generate meta-features for the meta-classifier. The FPR of the AIDS is significantly reduced through adopting the proposed false positive elimination strategy (FPES). In addition, a Bayesian optimization algorithm is used for hyper-parameters adjustment to optimize the overall performance of the RIDS. The proposed RIDS is evaluated using both a KDD99 dataset and a CBTC dataset collected on a hardware-in-the-Ioop simulation platform. For the KDD99 dataset, the TPR and FPR of the proposed RIDS are 98.6% and 1.3%, respectively. For the CBTC dataset, the TPR and FPR are 98.1% and 1.1 %, respectively. Based on the performance comparison among different RIDSs, it is can be concluded that the proposed RIDS is superior to the existing RIDS in both TPR and FPR.

Detecting anomalous traffic (and above all new ad-hoc attacks) with low false alarm rates is of primary interest in IP networks management. To this aim a key research topic in network security is represented by anomaly-based IDSs (Intrusion Detection Systems) thanks to their ability to face unknown attacks. Starting from more than a decade of research experience by the authors, the aim of this paper is to revise some of the most promising statistical approaches, namely Wavelets, Principal Component Analysis, CUSUM (cumulative sum control chart) and Information Theoretical methods (based on different definitions of the Entropy). Moreover, issues related to the choice of the relevant traffic parameters, use of sketches and availability of dataset for performance comparison are also discussed to highlight the main problems in intrusion detection.

This paper reviews popular signature and anomaly-based intrusion detection systems (IDS). Dimensionality reduction techniques (DRT) are used to increase the efficiency of such systems for real-time operation. Autoencoder-based IDS is rapidly gaining in popularity, primarily due to its inherent ability to denoise data and reduce dimensionality. In addition to the efficiency, we also look at the classification techniques used by various authors, and the overall impact of a model in terms of performance metrics. This article is written for novices in cybersecurityto get a jumpstart on the latest IDS algorithms. The purpose is to give useful insights into the broad and progressive view of various techniques in wide use, expose high-impact future research areas and to summarize classic IDS methods and feature selection techniques.

Anomaly based Intrusion Detection System (IDS) identifies intrusion by training itself to recognize acceptable behavior of the network. It then raises an alarm whenever any anomalous network behaviors outside the boundaries of its training sets are observed. However, anomaly based IDS are usually prone to high false positive rate due to difficulties involved in defining normal and abnormal network traffic patterns. In this paper, we employ two different statistical methods viz. Linear Discriminant Analysis (LDA) and Logistic Regression (LR) to develop new anomaly based IDS models. We then evaluate the performance of these IDS models on the benchmark NSL-KDD data set and analyze their performance against other IDS models based on Naive Bayes, C4.5 and Support Vector Machine (SVM). Experimental results show that the performance (Accuracy and Detection Rate) of both the LDA and the LR based models are at par and in some cases even better than other IDS models. Moreover, unlike the IDS model based on complex method like the SVM, the proposed LDA and LR based IDS models are computationally more efficient, which makes them more suited for deployment in real time network monitoring and intrusion detection analysis.

Blockchain and federated learning-based intrusion detection approaches for edge-enabled industrial IoT networks: a survey

Intrusion detection systems (IDSs) attempt to identify attacks by comparing collected data to predefined signatures known to be malicious (misuse-based IDSs) or to a model of legal behavior (anomaly-based IDSs). Anomaly-based approaches have the advantage of being able to detect previously unknown attacks, but they suffer from the difficulty of building robust models of acceptable behavior, which may result in a large number of false alarms. Almost all current anomaly-based intrusion detection systems classify an input event as normal or anomalous by analyzing its features, utilizing a number of different models. A decision for an input event is made by aggregating the results of all employed models. We have identified two reasons for the large number of false alarms, caused by incorrect classification of events in current systems. One is the simplistic aggregation of model outputs in the decision phase. Often, only the sum of the model results is calculated and compared to a threshold. The other reason is the lack of integration of additional information into the decision process. This additional information can be related to the models, such as the confidence in a model's output, or can be extracted from external sources. To mitigate these shortcomings, we propose an event classification scheme that is based on Bayesian networks. Bayesian networks improve the aggregation of different model outputs and allow one to seamlessly incorporate additional information. Experimental results show that the accuracy of the event classification process is significantly improved using our proposed approach.