IDEAs: Intrusion Detection and Event Analysis System Based on Data-Mining Approaches

Nowadays, one critical challenge of cybersecurity administrators is the protection of online resources from network intrusions. Despite several academic and industry research initiatives, full protection of online resources from these network intrusions is not feasible. Therefore, several techniques have been developed that use network audit data for accurate detection of network intrusions effectively and efficiently and are used in network intrusion detection systems (NIDSs). But, most of NIDSs reported low detection accuracy with high false alarm rate and provide a single solution that lacks in classification trade-offs. In this paper, the authors present a hybrid approach of multi-objective genetic algorithm and neural networks for creating a set of ensemble solutions for detecting network intrusions effectively. The proposed approach works in two phases that initially creates a set of non-dominating solutions or Pareto optimal solutions of base techniques and then creates ensemble solutions. In the outcome of individual solutions or models in the ensemble are aggregated using most popular method of majority voting. The proposed hybrid approach is evaluated using benchmark datasets of NSL_KDD and ISCX-2012 datasets for intrusion detection. The evaluation results using benchmark datasets demonstrate that the proposed hybrid approach enables detecting network intrusions effectively as compared to conventional ensemble approaches, namely bagging and boosting. The resultant ensemble solutions are non-dominating and provide classification trade-offs for cybersecurity administrators. The results also show that the proposed hybrid approach detects both minority and majority intrusion types accurately. The proposed hybrid approach demonstrated a detection accuracy of 97% and 88% with FPR of 2.4% and 2% for ISCX-2012 and NSL_KDD datasets, respectively.

Network intrusion detection system is a significant guarantee means for network security. However, the current network intrusion detection system generally suffers from less prior knowledge which then leads to poor generalizing ability. Network intrusion detection system based on support vector machine is endowed with favorable generalizing ability even with small sample size (less prior knowledge). The paper first gives an introduction to the basic concept of intrusion detection and the basic principle of the classifier based on support vector machine, then discusses algorithm of support vector machine, and finally forms network intrusion detection system combining anomaly intrusion detection and misuse intrusion detection, based on support vector machine.

This paper describes the ESIDE-Depian intrusion detection and prevention system, which uses Bayesian structural and parametric learning and also evidence propagation and adaptation, in order to improve the accuracy and manageability of network intrusion detection systems (NIDS). Current NIDS do not consider the two main detection paradigms, i.e. misuse detection and anomaly detection, in an unified style, so the analysis is not inherently complete. Besides, historical data are not generally used, neither for analysis nor for sequential adaptation of the knowledge representation models used for detection; hence this wealthy information about the essence and the potential trends of the target system is not commonly considered. Thus, by the generalized use of Bayesian belief networks, ESIDE-Depian achieves the main goal of detecting and preventing both well-known and also zero-day attacks with excellent results, by means of unified real-time analysis of network traffic.

Network intrusion detection systems (NIDS) are essential for protecting computer networks against cyberattacks. The selection of a nominal set of essential features that may adequately discriminate malicious traffic from normal traffic is indispensable while developing a NIDS. As such, a more reliable and accurate detection result may be realized when intrusion detection is carried out on a dataset based on an inclusive feature representation. This work presents the pre-processing and feature selection workflow as well as its results in the case of the CIC-IDS-2017 dataset, with a focus on two cyber-attacks, namely Denial-of-Service (DoS) and PortScan. The study applied an ensemble feature selection method based on information gain and Random Forest to filter out important features. Recursive Feature Elimination method was then applied to the reduced features to optimize the selected feature subset. The selected feature subset was experimented with using two classification algorithms, namely support vector machine and multi-layer perceptron. In the evaluation process, four widely used performance metrics were considered. The study results demonstrated the efficacy of the proposed ensemble approach to optimize the selected feature subset for detecting PortScan and DoS attacks in network traffic. Experimental results revealed that the support vector machine had a slight advantage in accuracy and could train more quickly. According to the study's evaluation, the NIDS may be able to shorten processing times without sacrificing the ability to detect PortScan and DoS attacks accurately by choosing a narrow subset of informative features. This suggests the approach might be applicable to real-world NIDS scenarios involving these attacks. The study also provides encouraging perspectives on how ensemble feature selection utilizing MLP and SVM can enhance the effectiveness of NIDS. Building on these findings, more research can create NIDS solutions that are even more reliable and efficient for the dynamic field of cybersecurity.

Traditional Network Intrusion Detection Systems (NIDSes) are generally implemented on vendor proprietary appliances or middleboxes with poor versatility and flexibility. Emerging Network Function Virtualization (NFV) and Software-Defined Networking (SDN) technologies can virtualize NIDSes and elastically scale them to deal with attack traffic variations. However, such an elasticity feature must not come at the cost of decreased detection effectiveness and expensive provisioning. In this paper, we propose an innovative NIDS architecture, vNIDS, to enable safe and efficient virtualization of NIDSes. vNIDS addresses two key challenges with respect to effective intrusion detection and non-monolithic NIDS provisioning in virtualizing NIDSes. The former challenge is addressed by detection state sharing while minimizing the sharing overhead in virtualized environments. In particular, static program analysis is employed to determine which detection states need to be shared. vNIDS addresses the latter challenge by provisioning virtual NIDSes as microservices and employing program slicing to partition the detection logic programs so that they can be executed by each microservice separately. We implement a prototype of vNIDS to demonstrate the feasibility of our approach. Our evaluation results show that vNIDS could offer both effective intrusion detection and efficient provisioning for NIDS virtualization.

In this paper the software system for network traffic analysis and intrusion detection has been developed. The system for monitoring and analyzing of traffic to detect anomalies and identify attacks based on Hurst parameter has been proposed. This system allows determining the Hurst flow parameter at different intervals of observation. This system can be installed on the network provider to more effectively use the bandwidth by balancing the load between different protocols and subscribers.

Signature-based Network Intrusion Detection Systems (NIDS) are an integral part of modern network security solutions. They help to detect and prevent network attacks and intrusions. However, they show critical performance problems in today's high speed networks. Filters have been proposed to reduce the amount of traffic to be analyzed by a NIDS, yet, such filters need to be very carefully designed in order not to miss relevant data. We address this problem by proposing a novel concept for filtering taking into account the pipelining architecture of modern web traffic. Our concept, which we named HTTP-based Payload Aggregation (HPA), is able to retain the first N bytes of the basic Protocol Data Unit (PDU) of an application layer protocol and discard the rest, arguing that the retained payload portion contains almost all relevant data for intrusion detection. We demonstrate the feasibility of our approach focusing on HTTP traffic as the most prominent protocol in many Internet applications. The idea is, thus, to capture the first N bytes of every pipelined session and forward this data to a NIDS. In our evaluation, we show that for the used traces we still detect more than 97% of the events with only 2.5% of the network traffic to be analyzed. We achieve an increase in packet throughput of up to 44 in our experiments.

Cyberspace is a concept describing a widespread, interconnected digital technology with numerous users. New standards add more concerns with tremendous information gathered from various network sources, which can be utilized for focused cyber-attacks. Digital attacks in networks are getting more complex and subsequently introducing expanding difficulties in precisely distinguishing network intrusions. Inability to forestall the network intrusions could debase the validity of safety administrations, for example, information privacy, trustworthiness, and accessibility. In this work, we center around developing a Network Intrusion Detection System which is implemented utilizing Machine Learning Techniques. IDS dependent on ML techniques are successful and exact in distinguishing varied networks intrusions. Also, many of the ML-based IDS experience the ill effects of an increment in false-positive rate, leading to lower precision and accuracy. Consequently, we present an exploration of the UNSW-NB15 intrusion detection dataset that will be utilized for preparing the machine learning models. In our tests, we carry out various ML approaches namely, Support Vector Machine (SVM), Logistic Regression (LR), Random Forest Classifier (RF), Decision Tree (DT), Gradient boosted decision trees (GBDT). The results demonstrated that the hyperparameter optimization technique using Grid Search and Random Search methods and evaluation using k-fold cross-validation gives better optimization of model parameters and henceforth brings about better execution.KeywordsCross-validationCyberattackDecision tree (DT)Gradient boosted decision trees (GBDT)Hyperparameter TuningLogistic regression (LR)Network intrusion detection system (IDS)Random forest classifier (RF)Support vector machine (SVM)

The security of the Industrial Internet of Things (IIoT) is of vital importance, and the Network Intrusion Detection System (NIDS) plays an indispensable role in this. Although there is an increasing number of studies on the use of deep learning technology to achieve network intrusion detection, the limited local data of the device may lead to poor model performance because deep learning requires large-scale datasets for training. Some solutions propose to centralize the local datasets of devices for deep learning training, but this may involve user privacy issues. To address these challenges, this study proposes a novel federated learning (FL)-based approach aimed at improving the accuracy of network intrusion detection while ensuring data privacy protection. This research combines convolutional neural networks with attention mechanisms to develop a new deep learning intrusion detection model specifically designed for the IIoT. Additionally, variational autoencoders are incorporated to enhance data privacy protection. Furthermore, an FL framework enables multiple IIoT clients to jointly train a shared intrusion detection model without sharing their raw data. This strategy significantly improves the model's detection capability while effectively addressing data privacy and security issues. To validate the effectiveness of the proposed method, a series of experiments were conducted on a real-world Internet of Things (IoT) network intrusion dataset. The experimental results demonstrate that our model and FL approach significantly improve key performance metrics such as detection accuracy, precision, and false-positive rate (FPR) compared to traditional local training methods and existing models.

Network Intrusion Detection System (NIDS) has become more important as a large number of diverse devices connect to the Internet of Things (IoT). Generally, training an effective NIDS requires a large amount of high-quality and centralized attack data. However, in real-world scenarios, it is difficult to centralize the distributed data for training NIDS in the IoT due to the privacy concerns and data format heterogeneity. To solve this problem, a novel NIDS combining federated learning and cost-sensitive learning is proposed, named FIDS-CL. Specifically, federated learning with dynamic weights aggregation tackles the problem of non-clusterable data, where multiple clients collaboratively enhance the overall performance while dynamically aggregating weights to maximize the retention of high-performing client models. Moreover, cost-sensitive learning is employed to alleviate the problem of class imbalance in NIDS by dynamically adjusting the gradient descent weights of different classes in the loss function, thereby emphasizing the importance of minority classes. Therefore, our method can effectively handle data imbalance while safeguarding data privacy of clients, which is more effective to detect network intrusions. The experiments conducted across various scenarios validate the superior detection capabilities and computational efficiency of FIDS-CL when compared to other state-of-the-art NIDSs.

Network intrusion detection system (NIDS) monitors network traffic to detect an unauthorized activity in computer networks. The NIDS is classified according to detection technique into signature and anomaly based. Each of them has its own advantage and disadvantage. The signature-based is more effective in detecting known attacks but it is unable to detect new attacks. The anomaly-based is better in detecting new attacks but it may produce many false alarms. NIDS which use both of them try to exploit the strengths of them. In this paper we propose an improvement of the well known Snort NIDS using clonal selection algorithm (CSA). The proposed approach is evaluated using the 1999 DARPA Intrusion Detection Evaluation Data Sets of MIT (Massachusetts Institute of Technology) as a testbed. The conducted experiments compare the recall, precision, and F-score of Snort NIDS on its own, Snort NIDS improved by negative selection algorithm (NSA), and the proposed approach. The obtained results show that the proposed approach is more powerful than the others.

Predominant network intrusion detection systems (NIDS) aim to identify malicious traffic patterns based on a handcrafted dataset of rules. Recently, the application of machine learning in NIDS helps alleviate the enormous effort of human observation. Federated learning (FL) is a collaborative learning scheme concerning distributed data. Instead of sharing raw data, it allows a participant to share only a trained local model. Despite the success of existing FL solutions, in NIDS, a network’s traffic data distribution does not always fit into the single global model of FL; some networks have similarities with each other but other networks do not. We propose Segmented-Federated Learning (Segmented-FL), where by employing periodic local model evaluation and network segmentation, we aim to bring similar network environments to the same group. A comparison between FL and our method was conducted against a range of metrics including the weighted precision, recall, and F1 score, using a collected dataset from 20 massively distributed networks within 60 days. By studying the optimized hyperparameters of Segmented-FL and employing three evaluation methods, it shows that Segmented-FL has better performance in all three types of intrusion detection tasks, achieving validation weighted F1 scores of 0.964, 0.803, and 0.912 with Method A, Method B, and Method C respectively. For each method, this scheme shows a gain of 0.1%, 4.0% and 1.1% in performance compared with FL.

With the increasing scale of network, attacks against network emerge one after another, and security problems become increasingly prominent. Network intrusion detection system is a widely used and effective security means at present. In addition, with the development of machine learning technology, various intelligent intrusion detection algorithms also start to sprout. By flexibly combining these intelligent methods with intrusion detection technology, the comprehensive performance of intrusion detection can be improved, but the vulnerability of machine learning model in the adversarial environment can not be ignored. In this paper, we study the defense problem of network intrusion detection system against adversarial samples. More specifically, we design a defense algorithm for NIDS against adversarial samples by using bidirectional generative adversarial network. The generator learns the data distribution of normal samples during training, which is an implicit model reflecting the normal data distribution. After training, the adversarial sample detection module calculates the reconstruction error and the discriminator matching error of sample. Then, the adversarial samples are removed, which improves the robustness and accuracy of NIDS in the adversarial environment.

This paper presents an overview of the technologies and the methodologies used in Network Intrusion Detection and Prevention Systems (NIDPS). Intrusion Detection and Prevention System (IDPS) technologies are differentiated by types of events that IDPSs can recognize, by types of devices that IDPSs monitor and by activity. NIDPSs monitor and analyze the streams of network packets in order to detect security incidents. The main methodology used by NIDPSs is protocol analysis. Protocol analysis requires good knowledge of the theory of the main protocols, their definition, how each protocol works.Keywords: Intrusion Detection and Prevention System, Protocol Analysis, Sensor, Signature, State(ProQuest: ... denotes formulae omitted.)1 IntroductionIncreasing size and complexity of the Internet and Intranet networks have led to increasing number of vulnerabilities that could be exploited. Thus, the internal and external attacks on the information systems are increasing at an alarming rate. Also, these are becoming more severe and sophisticated. The attackers find ingenious ways to bypass the security controls and to compromise the security and the well functioning of the information systems. They are motivated by financial, political, and military objectives. In this context, defending wide area networks from malicious traffic, unauthorized access to systems involves many problems.In security information systems Network Intrusion Detection and Prevention Systems (NIDPS) are important tools to detect possible incidents and also, to attempt to stop them in real time. Due to changing attacks, intrusion detection methodologies and technologies continuously evolve, adding new detection capabilities, to avoid detection. They must adapt to new forms of malware, to the public networks, increased traffic.2 Concepts of Intrusion DetectionAn intrusion is a successful action to gain access to an information system, to compromise it or to make it unavailable. This is possible due to the presence of vulnerability in the target system that can be exploited by a motivated intruder.Intrusion Detection and Prevention is the process of monitoring the information systems by sensors or agents and analyzing the collected information to detect and to attempt to stop the attacks in real time, identifying vulnerabilities, the violation of security policies or standard security practices.An Intrusion Detection and Prevention System (IDPS) is a tool that monitors information systems, collects, analyzes information, and initiates responses when an intrusion is detected.Intrusion Detection Systems (IDSs) mainly work as defensive mechanisms. They only alert the system administrators that an incident has occurred. Intrusion Prevention Systems (IPSs) can take some actions to attempt to stop the attack, such as breaking the connection or modifying the firewall rules to deny access to the intruder. The response of the classic IDS can be slow if the system administrator is busy while the response of the IPS is automatic. An architecture that uses together IPS and IDS technologies is the best solution for defense in depth.Conceptually, a generic IDPS consists of modular components. It mainly has the following components: monitoring system, storage, analyzer, and responder.* Monitoring system - monitors and logs the events in a computer system or network;* Storage - stores information, called audit record, about suspicious activities or intrusions; also, the security policies used in analysis are stored;* Analyzer - uses different analysis methodologies to detect the incidents;* Responder - the response mechanism of incidents.The IDPSs could be classified as:* By detection methodology [12], [18]:- misuse-based detection- anomaly-based detection- stateful protocol analysis* By activity [12]:- network-based- wireless-based- network behavior analysis- host-based* By behavior on detection:- passive- active* By collection and analysis frequency:- continuous- periodicThe detection methodologies describe the characteristics of the analyzer. …

Network intrusion attacks are a known threat. To detect such attacks, network intrusion detection systems (NIDSs) have been developed and deployed. These systems apply machine learning models to high-dimensional vectors of features extracted from network traffic to detect intrusions. Advances in NIDSs have made it challenging for attackers, who must execute attacks without being detected by these systems. Prior research on bypassing NIDSs has mainly focused on perturbing the features extracted from the attack traffic to fool the detection system, however, this may jeopardize the attack’s functionality. In this work, we present TANTRA, a novel end-to-end Timing-based Adversarial Network Traffic Reshaping Attack that can bypass a variety of NIDSs. Our evasion attack utilizes a long short-term memory (LSTM) deep neural network (DNN) which is trained to learn the time differences between the target network’s benign packets. The trained LSTM is used to set the time differences between the malicious traffic packets (attack), without changing their content, such that they will “behave” like benign network traffic and will not be detected as an intrusion. We evaluate TANTRA on eight common intrusion attacks and three state-of-the-art NIDS systems, achieving an average success rate of 99.99% in network intrusion detection system evasion. We also propose a novel mitigation technique to address this new evasion attack.