Identification of anomalous network security token usage via clustering and density estimation

Network-based Intrusion Detection Systems (NIDSs) are an important mechanism to identify malicious behaviour or policy violations within a network. Such detection systems typically face several challenges, among which are the base-rate fallacy and the resilience against adaptive adversaries. These challenges are often countered in modern NIDSs by combining multiple detection systems to diversify the used feature levels or utilize the advantages of multiple detection methods. However, currently there exists no suitable framework for a detailed analysis of such composed systems. Therefore, the contribution of this work is an evaluation framework for composed systems, which builds on previous information-theoretic approaches and highlights the utility of information-theoretic redundancies for robustness evaluations. This framework enables an attribution of the overall system performance to its individual components, to fine-tune parameters and to study the dynamics between classifiers. The versatility of the framework is demonstrated by designing and evaluating a composed NIDS example based on systems described in the literature and using an open data set. Studying the impact of an evasion attempt with adversarial examples on this system highlighted the importance of robustness against false-alarms as well as detection evasion. Moreover, the framework enables general insights on how to improve the design of composed NIDSs: based on the dynamics between classifiers, it can be shown that optimizing the operation point of each component individually does not necessarily maximize the overall system performance from an information-theoretic perspective. Additionally, it can be shown that existing classification redundancies might not be fully utilized during an attack on the NIDS components, due to a static system design.

Network-based intrusion detection and prevention systems -NIDS /NIPS- have been widely implemented in order to build layered information security infrastructures. Many intrusion detection and prevention algorithms have been developed and its operation modes imply benefits and drawbacks that are not taken into account when choosing a network IDS/IPS. This paper assesses the network security provided by five NIDS/NIPS software solutions, by analyzing the protection against malicious traffic. Security platforms against network intrusion such as Snort Inline, Stonegate IPS, Strataguard, Intrupro and Packetalarm are tested and analyzed.

: Network-based intrusion detection systems (NIDSs) are one component of a comprehensive network security solution. The use of IPsec, which encrypts network traffic, renders network intrusion detection virtually useless unless traffic is decrypted at network gateways. One alternative to NIDSs, host-based intrusion detection systems (HIDSs), provides some of the functionality of NIDSs but with limitations. HIDSs cannot perform a network-wide analysis and can be subverted if a host is compromised. We propose an approach to intrusion detection that combines HIDS, NIDS, and a version of IPsec that encrypts the header and the body of IP packets separately. We refer to the latter generically as TwoKey IPsec. We show that all of the network events currently detectable by the Snort NIDS on un- encrypted network traffic are also detectable on encrypted network traffic using this approach. The NIDS detects network-level events that HIDSs have trouble detecting and HIDSs detect application-level events that can't be detected by the NIDS.

LIO-IDS: Handling class imbalance using LSTM and improved one-vs-one technique in intrusion detection system

Detecting cyber threats is one of the most important security issues today, and network based intrusion detection systems (NIDS) play a vital role. Many studies have adopted Artificial Intelligence/Machine Learning (AI/ML) technology to create powerful NIDS to detect cyber threats. While most of NIDS studies focus on improving classification/detection accuracy by proposing new AI/ML models, all models produce a ratio of false positive alerts in the field. Very few studies discuss the methodology to handle false positive alerts. Giving a busy network, handling the number of false positive alerts become a time consuming task for security personnel. For this reason, false positive alert filtering automation is an important issue. In this paper, we propose a scheme to automate false positive alert filtering by leveraging kernel density estimation. Disregard of the deployed NIDS, our proposed scheme can assist security personnel in alert verification task. Our experiments show that our proposed scheme is 34% to 62% better in performance (in terms of error ratio) compared to other algorithms. Our proposed scheme also reduces 75% of the time in alert verification process.

All currently available Network-based Intrusion Detection Systems (NIDS) rely upon passive protocol analysis which is fundamentally flawed as an attack can evade detection by exploiting ambiguities in the traffic stream as seen by the NIDS. We observe that different attack variations can be derived from the original attack using simple transformations. This paper proposes a semantic model for attack mutation based on dynamic description logics (DDL(X)), extensions of description logics (DLs) with a dynamic dimension, and explores the possibility of using DDL(X) as a basis for evasion composition. The attack mutation model describes all the possible transformations and how they can be applied to the original attack to generate a large number of attack variations. Furthermore, this paper presents a heuristics planning algorithm for the automation of evasion composition at the functional level based on DDL(X). Our approach employs classical DL-TBoxes to capture the constraints of the domain, DL-ABoxes to present the attack, and DL-formulas to encode the objective sequence of packets respectively. In such a way, the evasion composition problem is solved by a decidable tableau procedure. The preliminary results certify the potential of the approach.KeywordsIntrusion Detection/Prevention SystemsMulti-protocol EvasionsAdvanced Evasion TechniquesKnowledge Representation and ReasoningDynamic Description Logics

The effectiveness and precision of network-based intrusion detection signatures can be evaluated either by direct analysis of the signatures (if they are available) or by using black-box testing (if the system is closed-source). Recently, several techniques have been proposed to generate test cases by automatically deriving variations (or mutations) of attacks. Even though these techniques have been useful in identifying "blindspots" in the signatures of closed-source, network-based intrusion detection systems, the generation of test cases is performed in a random, un- guided fashion. The reason is that there is no information available about the signatures to be tested. As a result, identifying a test case that is able to evade detection is difficult. In this paper, we propose a novel approach to drive the generation of test cases by using the information gathered by analyzing the dynamic behavior of the intrusion detection system. Our approach applies dynamic dataflow analysis techniques to the intrusion detection system to identify which parts of a network stream are used to detect an attack and how these parts are matched by a signature. The result of our analysis is a set of constraints that is used to guide the black-box testing process, so that the mutations are applied to only those parts of the attack that are relevant for detection. By doing this, we are able to perform a more focused generation of the test cases and improve the process of identifying an attack variation that evades detection.

In Network-based Intrusion Detection, signatures discovery is an important issue, since the performance of an intrusion detection system heavily depends on accuracy and abundance of signatures. In most cases, we have to find these signatures manually. This is a time-consuming and error-prone work. Some papers introduce data mining into Intrusion Detection System. However, there are some drawbacks in these schemes. We present a data mining based approach to supporting signature discovery in network-based Intrusion Detection System. It has people find signatures of an intrusion easily. The main idea is that: First, Signature Discovery System (SDS) tries to find the most possible signatures that occur very frequently in the communication monitored. Second, SDS will find the relationships between these candidate signatures and construct rules based on these relationships found. Finally, SDS gives two kinds of hints: one is the signatures whose frequency of occurrence is greater than a threshold; the other is a set of rules composed of a set of signatures that are created by SDS in the second step. An experimental system called SigSniffer has been implemented to test the feasibility of the proposed approach.

A high growth rate in network traffic and the complexity of cyber threats have made it necessary to create more effective and flexible intrusion detection systems. Most traditional Network-based Intrusion Detection Systems (NIDS) can become weak at detecting new patterns of attacks due to the use of obsolete data or traditional machine learning models. To overcome the mentioned constraints, the current research presents a new deep learning solution that combines Sequential Deep Neural Networks (DNN) and Rectified Linear Unit (ReLU) activation unit with an Extra Tree Classifier feature selection procedure. The proposed model is trained and tested on the new rich and up-to-date UNSW-NB15 set, which provides a realistic reflection of the real-life network traffic and attack vectors. The interesting novelty of this study is the tactical use of ReLU-based DNN combined with feature optimization through the Extra Tree Classifier, which not only overcomes general problems like vanishing gradients and overfitting but also greatly increases the interpretability of the model and the efficiency of its computation. This dimensional reduction of the feature space (43 to only 8 highly relevant features) retains the high accuracy of the model but with better inference speed, which is a crucial aspect of the real-time deployment of NIDS. The results show that with the Sequential DNN approach, the binary class (0 for normal and 1 for attack records) achieved 97.93% accuracy, 97% Precision, 97% Recall and 97% F1-score. Furthermore, the detailed experimental testing, such as ROC curves and Confusion Matrices, confirmed that the Sequential DNN performed well in comparison to other Existing Studies. These findings underscore the effectiveness of deep learning architectures enhanced with optimized feature selection in detecting network intrusions, making the proposed system a promising solution for securing critical infrastructure in sectors such as finance, healthcare, and government networks.

Network based intrusion detection systems (NIDS) play a vital role in protecting valuable assets or applications in a wide range of industrial domains. Especially commercial NIDS providers need to address very specific requirements and challenges for theses domains, such as supporting a variety of different network protocols. Despite such challenges, most commercial NIDS vendors offer one solution for multiple industrial domains. In contrast, most NIDSs proposed by researchers are evaluated on only a few domain-specific datasets due to the lack of publicly available industrial datasets. Therefore, conclusions about the applicability of research-oriented NIDS across industrial domains cannot be made. Domain-agnostic threat detection methods are required when advanced persistent threats (APT) are evolving across multiple sectors. This research work presents a cross-domain physical cybersecurity testbed environment, CrossTest, for the development and evaluation of domain-agnostic threat detection methods. For this purpose, two testbeds were designed, one for the energy and another for the production domain. Multiple cyber-attacks were implemented in both testbeds and network traffic was recorded as PCAP files. The dataset containing PCAP files with corresponding description will be made publicly available upon request. Furthermore, we demonstrate the evaluation of an open source network traffic analysis tool, Malcolm, with CrossTest. The evaluation identified major issues that are briefly described in this work.

Malware-infected hosts have typically been detected using network-based Intrusion Detection Systems on the basis of characteristic patterns of HTTP requests collected with dynamic malware analysis. Since attackers continuously modify malicious HTTP requests to evade detection, novel HTTP requests sent from new malware samples need to be exhaustively collected in order to maintain a high detection rate. However, analyzing all new malware samples for a long period is infeasible in a limited amount of time. Therefore, we propose a system for efficiently collecting HTTP requests with dynamic malware analysis. Specifically, our system analyzes a malware sample for a short period and then determines whether the analysis should be continued or suspended. Our system identifies malware samples whose analyses should be continued on the basis of the network behavior in their short-period analyses. To make an accurate determination, we focus on the fact that malware communications resemble natural language from the viewpoint of data structure. We apply the recursive neural network, which has recently exhibited high classification performance in the field of natural language processing, to our proposed system. In the evaluation with 42,856 malware samples, our proposed system collected 94% of novel HTTP requests and reduced analysis time by 82% in comparison with the system that continues all analyses.

NIDSs identify malicious activities by analyzing network traffic. NIDSs are trained with the samples of benign and intrusive network traffic. Training samples belong to either majority or minority classes depending upon the number of available instances. Majority classes consist of abundant samples for the normal traffic as well as for recurrent intrusions. Whereas, minority classes include fewer samples for unknown events or infrequent intrusions. NIDSs trained on such imbalanced data tend to give biased predictions against minority attack classes, causing undetected or misclassified intrusions. Past research works handled this class imbalance problem using data-level approaches that either increase minority class samples or decrease majority class samples in the training data set. Although these data-level balancing approaches indirectly improve the performance of NIDSs, they do not address the underlying issue in NIDSs i.e. they are unable to identify attacks having limited training data only. This paper proposes an algorithm-level approach called I-SiamIDS, which is a two-layer ensemble for handling class imbalance problem. I-SiamIDS identifies both majority and minority classes at the algorithm-level without using any data-level balancing techniques. The first layer of I-SiamIDS uses an ensemble of b-XGBoost, Siamese-NN and DNN for hierarchical filtration of input samples to identify attacks. These attacks are then sent to the second layer of I-SiamIDS for classification into different attack classes using m-XGBoost. As compared to its counterparts, I-SiamIDS showed significant improvement in terms of Accuracy, Recall, Precision, F1-score and values of AUC for both NSL-KDD and CIDDS-001 datasets. To further strengthen the results, computational cost analysis was also performed to study the acceptability of the proposed I-SiamIDS.

Application features such as port numbers are used by network-based intrusion detection systems (NIDSs) to detect attacks coming from networks. System calls and the operating system related information are used by host-based intrusion detection systems (HIDSs) to detect intrusions towards a host. However, the relationship between hardware architecture events and denial-of-service (DoS) attacks has not been well revealed. When increasingly sophisticated intrusions emerge, some attacks are able to bypass both the application and the operating system level feature monitors. Therefore, a more effective solution is required to enhance existing HIDSs. In this paper, we identify the following hardware architecture features: instruction count, cache miss, bus traffic and integrate them into a novel HIDS framework based on a modern statistical gradient boosting trees model. Through the integration of application, operating system and architecture level features, our proposed HIDS demonstrates a significant improvement of the detection rate in terms of sophisticated DoS intrusions.

Network-based Intrusion Detection Systems (NIDS), e.g., Snort, Bro or NSM, try to detect malicious network activity such as Denial of Service (DoS) attacks and port scans by monitoring network traffic. Research from network traffic measurement has identified various patterns that exploits on today's Internet typically exhibit. However, there has not been any significant attempt, so far, to design algorithms with provable guarantees for detecting exploit patterns from network traffic packets. In this work, we develop and apply data streaming algorithms to detect exploit patterns from network packet streams. In network intrusion detection, it is necessary to analyze large volumes of data in an online fashion. Our work addresses scalable analysis of data under the following situations. (1) Attack traffic can be stealthy in nature, which means detecting a few covert attackers might call for checking traffic logs of days or even months, (2) Traffic is multidimensional and correlations between multiple dimensions maybe important, and (3) Sometimes traffic from multiple sources may need to be analyzed in a combined manner. Our algorithms offer provable bounds on resource consumption and approximation error. Our theoretical results are supported by experiments over real network traces and synthetic datasets.

Intrusion detection is a challenging and critical problem in network security. Extensive research activities have been aimed at network-based intrusion detection systems, but most of them are proved unsatisfactory. This paper presents an effective intrusion detection classification model based on projection pursuit. With maximizing a projection index, Projection Pursuit uses Genetic Algorithm to search for the optimal projection direction, projects network connections records from high-dimensional space into 1-dimensional space, and uses the projection values to analyze the data structure and classify the type of intrusion. This intrusion detection classification model not only cuts down the computing complexity in the process of network connection records, but also opens out non-linear structure not like in latent semantics analysis only discovering linear structure, and the results of classification can also be visualized. The results of the experiments on KDD CUP 1999 data show that this model not only has good performance, but also reduces the number of false alarms effectively.