Visualizing Realistic Benchmarked IDS Dataset: CIRA-CIC-DoHBrw-2020

A novel approach for handling missing data to enhance network intrusion detection system

Due to the escalating network throughput and security risks, the exploration of intrusion detection systems (IDSs) has garnered significant attention within the computer science field. The majority of modern IDSs are constructed using deep learning techniques. Nevertheless, these IDSs still have shortcomings where most datasets used for IDS lies in their high imbalance, where the volume of samples representing normal traffic significantly outweighs those representing attack traffic. This imbalance issue restricts the performance of deep learning classifiers for minority classes, as it can bias the classifier in favor of the majority class. To address this challenge, many solutions are proposed in the literature. TDCGAN is an innovative Generative Adversarial Network (GAN) based on a model-driven approach used to address imbalanced data in the IDS dataset. This paper investigates the performance of TDCGAN by employing it to balance data across four benchmark IDS datasets which are CIC-IDS2017, CSE-CIC-IDS2018, KDD-cup 99, and BOT-IOT. Next, four machine learning methods are employed to classify the data, both on the imbalanced dataset and on the balanced dataset. A comparison is then conducted between the results obtained from each to identify the impact of having an imbalanced dataset on classification accuracy. The results demonstrated a notable enhancement in the classification accuracy for each classifier after the implementation of the TDCGAN model for data balancing.

Nowadays, Internet-driven culture securing computer networks in Higher Learning Institutions (HLIs) has become a major responsibility. Intrusion Detection Systems (IDS) are crucial for protecting networks from unauthorized activity and cyber threats. This paper examines the process of improving network security by creating a comprehensive IDS dataset using real traffic from HLIs, highlighting the importance of accurate and representative data in improving the system's ability to identify and mitigate future cyber-attacks. The IDS model was created using a variety of machine learning (ML) techniques. Metrics like accuracy, precision, recall, and F1-score were used to assess the performance of each model. The dataset used for training and testing was real-world network traffic data obtained from the institution's computer network. The results showed that the developed IDS obtained exceptional accuracy rates, with Random Forest, Gradient Boosting, and XGBoost models all achieving an accuracy of around 93%. Precision and recall values were likewise quite high across all algorithms. Furthermore, the study discovered that data quality has a substantial impact on IDS performance. Proper data preparation, feature engineering, and noise removal were found to be helpful in improving model accuracy and reducing false positives. While the IDS models performed well throughout validation and testing, implementing such systems in a production setting necessitates careful thought. As a result, the essay also examined the procedures for testing and deploying the IDS models in a real-world scenario. It underlined the significance of ongoing monitoring and maintenance in order to keep the model effective in identifying intrusions. The research aids in the progress of network security in HLI. Educational institutions can better protect their precious assets and sensitive information from cyberattacks by understanding the impact of data quality on IDS performance and implementing effective deployment techniques

Due to the recent advances in the Internet and communication technologies, network systems and data have evolved rapidly. The emergence of new attacks jeopardizes network security and make it really challenging to detect intrusions. Multiple network attacks by an intruder are unavoidable. Our research targets the critical issue of class imbalance in intrusion detection, a reflection of the real-world scenario where legitimate network activities significantly out number malicious ones. This imbalance can adversely affect the learning process of predictive models, often resulting in high false-negative rates, a major concern in Intrusion Detection Systems (IDS). By focusing on datasets with this imbalance, we aim to develop and refine advanced algorithms and techniques, such as anomaly detection, cost-sensitive learning, and oversampling methods, to effectively handle such disparities. The primary goal is to create models that are highly sensitive to intrusions while minimizing false alarms, an essential aspect of effective IDS. This approach is not only practical for real-world applications but also enhances the theoretical understanding of managing class imbalance in machine learning. Our research, by addressing these significant challenges, is positioned to make substantial contributions to cybersecurity, providing valuable insights and applicable solutions in the fight against digital threats and ensuring robustness and relevance in IDS development. An intrusion detection system (IDS) checks network traffic for security, availability, and being non-shared. Despite the efforts of many researchers, contemporary IDSs still need to further improve detection accuracy, reduce false alarms, and detect new intrusions. The mean convolutional layer (MCL), feature-weighted attention (FWA) learning, a bidirectional long short-term memory (BILSTM) network, and the random forest algorithm are all parts of our unique hybrid model called MCL-FWA-BILSTM. The CNN-MCL layer for feature extraction receives data after preprocessing. After convolution, pooling, and flattening phases, feature vectors are obtained. The BI-LSTM and self-attention feature weights are used in the suggested method to mitigate the effects of class imbalance. The attention layer and the BI-LSTM features are concatenated to create mapped features before feeding them to the random forest algorithm for classification. Our methodology and model performance were validated using NSL-KDD and UNSW-NB-15, two widely available IDS datasets. The suggested model's accuracies on binary and multi-class classification tasks using the NSL-KDD dataset are 99.67% and 99.88%, respectively. The model's binary and multi-class classification accuracies on the UNSW-NB15 dataset are 99.56% and 99.45%, respectively. Further, we compared the suggested approach with other previous machine learning and deep learning models and found it to outperform them in detection rate, FPR, and F-score. For both binary and multiclass classifications, the proposed method reduces false positives while increasing the number of true positives. The model proficiently identifies diverse network intrusions on computer networks and accomplishes its intended purpose. The suggested model will be helpful in a variety of network security research fields and applications.

One of the key challenges of the machine learning (ML) based intrusion detection system (IDS) is the expensive computation time which is largely caused by the redundant, incomplete, and unrelated features contain in the IDS datasets. To overcome such challenges and ensure building efficient and more accurate IDS models, many researchers utilize preprocessing techniques such as normalization and feature selection, and a hybrid modeling approach is typically used. In this work, we propose a hybrid IDS modeling approach with an algorithm for feature selection (FS) and another for building the IDS. The FS method is a wrapper-based FS with a decision tree as the feature evaluator. Five selected ML algorithms are individually used in combination with the proposed FS method to build five IDS models using the UNSW-NB15 dataset. As a baseline, five more IDS models are built, in a single modeling approach, using the full features of the datasets. We evaluate the effectiveness of our proposed method by comparing it with the baseline models and also with state-of-the-art works. Our method achieves the best DR of 97.95% and proved to be quite effective in comparison to state-of-the-art works. We, therefore, recommend its usage especially in IDS modeling with the UNSW-NB15 dataset.

The significant increase in technology development over the internet makes network security a crucial issue. An intrusion detection system (IDS) shall be introduced to protect the networks from various attacks. Even with the increased amount of works in the IDS research, there is a lack of studies that analyze the available IDS datasets. Therefore, this study presents a comprehensive analysis of the relevance of the features in the KDD99 and UNSW-NB15 datasets. Three methods were employed: a rough-set theory (RST), a back-propagation neural network (BPNN), and a discrete variant of the cuttlefish algorithm (D-CFA). First, the dependency ratio between the features and the classes was calculated, using the RST. Second, each feature in the datasets became an input for the BPNN, to measure their ability for a classification task concerning each class. Third, a feature-selection process was carried out over multiple runs, to indicate the frequency of the selection of each feature. From the result, it indicated that some features in the KDD99 dataset could be used to achieve a classification accuracy above 84%. Moreover, a few features in both datasets were found to give a high contribution to increasing the classification’s performance. These features were present in a combination of features that resulted in high accuracy; the features were also frequently selected during the feature selection process. The findings of this study are anticipated to help the cybersecurity academics in creating a lightweight and accurate IDS model with a smaller number of features for the developing technologies.

Intrusion detection systems (IDSs) play a critical role in many computer networks to combat attacks from external environments. However, due to the rapid spread of various new attacks, developing a robust IDS that can effectively detect novel attacks and prevent them from devastating network systems is a challenging task. Recently, deep neural networks (DNNs) have been widely used to enhance the accuracy of IDSs in detecting network intrusions. Nevertheless, the performance of DNN highly depends on the representation of the input data. In this paper, we introduce a novel method called DeepInsight-Convolutional Neural Network-Intrusion Detection System (DC-IDS). In CD-IDS, the DeepInsight technique is used to transform the network traffic data into a new representation in the form of an image. This new representation of the traffic data is then used as the input of a Convolutional Neural Network (CNN). We evaluate our proposed technique using an extensive experiment on five IDS datasets. The experimental results show that the proposed model enhances the performance of IDSs in detecting various network attacks compared to different popular machine learning algorithms.

The Smart Grid aims to enhance the electric grid’s reliability, safety, and efficiency by utilizing digital information and control technologies. Real-time analysis and state estimation methods are crucial for ensuring proper control implementation. However, the reliance of Smart Grid systems on communication networks makes them vulnerable to cyberattacks, posing a significant risk to grid reliability. To mitigate such threats, efficient intrusion detection and prevention systems are essential. This paper proposes a hybrid deep-learning approach to detect distributed denial-of-service attacks on the Smart Grid’s communication infrastructure. Our method combines the convolutional neural network and recurrent gated unit algorithms. Two datasets were employed: The Intrusion Detection System dataset from the Canadian Institute for Cybersecurity and a custom dataset generated using the Omnet++ simulator. We also developed a real-time monitoring Kafka-based dashboard to facilitate attack surveillance and resilience. Experimental and simulation results demonstrate that our proposed approach achieves a high accuracy rate of 99.86%.

An automated intrusion detection system in IoT system using Attention based Deep Bidirectional Sparse Auto Encoder model

Software-defined Networking (SDN) is the key catalyst in the next-generation network. Cloud computing has shown rapid growth due to the development of SDN. Due to the growth in security difficulties and threats, SDN networks will have new security issues to overcome, and a new layer of security procedures, such as intrusion detection systems (IDS), is required. Nowadays, web services are flexible, economical, and user-friendly. Although this may be true, they still seem to be more susceptible to security issues than legacy systems. Therefore, SDN architecture also incorporates several machine learning methods to find classify malicious attacks and legitimate traffic. We include a rigorous study of the current strategies for malicious traffic identification and we define the classical machine learning method's shortcomings, and a deep learning technique is also being investigated to reach higher levels of efficiency and accuracy. This article explores SDN security and IDS about security concerns. A dataset of IDS, publicly accessible, KDD-CUP99, and NSL-KDD Dataset are used to determine the possible behaviour of security flaws.

An intrusion detection system (IDS) is a device or software application that monitors a network for malicious activity or policy violations. It scans a network or a system for a harmful activity or security breaching. IDS protects networks (Network-based intrusion detection system NIDS) or hosts (Host-based intrusion detection system HIDS), and work by either looking for signatures of known attacks or deviations from normal activity. Deep learning algorithms proved their effectiveness in intrusion detection compared to other machine learning methods. In this paper, we implemented deep learning solutions for detecting attacks based on Long Short-Term Memory (LSTM). PCA (principal component analysis) and Mutual information (MI) are used as dimensionality reduction and feature selection techniques. Our approach was tested on a benchmark data set, KDD99, and the experimental outcomes show that models based on PCA achieve the best accuracy for training and testing, in both binary and multiclass classification.

Today, since the world increasingly depends on computers and automation, securing applications and systems has become critical. The possibility of threat and violence is expanding rapidly due to the growing complexity of modern networks and services. This study focuses on current datasets, the gaps and limitations of these datasets, and their effects on creating Intrusion Detection Systems (IDS). The study outlines the existing datasets that show an apparent absence of real-world threats, attack representation, and outdated threats, limiting the efficiency of machine learning IDS techniques. This work presents the most Machine Learning techniques for implementing IDS and a comprehensive review of the available benchmark datasets. We believe this review will enable future researchers in this field to understand better different kinds of IDS datasets, advantages, and limitations.

Systems employing the Internet and Voice over Internet Protocol (VoIP) are alternative to the legacy landline telephony system. The services offered by these systems allow users to communicate with their family members, friends, banks and business partners whenever they are online. Particularly, the VoIP service is popular with the Internet users because charges are bound into usually fixed access costs making the price of the long distance calls themselves appear economical and even free. However, VoIP services have several disadvantages. One of the disadvantages of VoIP is, it is exposed to Internet security vulnerabilities, threats and attacks. The attacks come in many forms. One of the methods to attack the VoIP service, is by sending a Spam over Internet Telephony (SPIT). SPIT is normally executed by malicious parties initiating automated, unsolicited, and unwanted communications that used VoIP or video conferencing services, just like an email SPAM. Solutions to mitigate SPIT are still lacking. To defense a VoIP system from SPIT, we have come out with the idea of Automated Intrusion Detection and Prevention System over SPIT (AIDPoS) that can detect and prevent SPIT from external parties to enter the VoIP network. AIDPoS is a combination system of Intrusion Detection System (IDS) and Intrusion Prevention System (IPS) on Voice over IP (VoIP) server. This is an experimental research to find a solution to combat SPIT if it happens in the future. Our result shows that an automated Intrusion detection and Prevention system over SPIT attack is feasible. Our work is on an open source VoIP network. Other vulnerabilities and threats are important, but beyond our research scope and is not covered in this paper.

Attack-aware Security Function Management

Lateral Movement (LM) represents a growing threat, frequently employed by advanced persistent threat groups to escalate privileges and navigate systems towards high-value assets. Recognizing the limitations of existing literature, this work leverages the unique LM-focused LMD-2023 imbalanced benchmark dataset, which comprises Microsoft Windows Sysmon logs, to provide a multifaceted novel contribution to the LM Intrusion Detection System (IDS) domain. We investigate the impact of various open-source over-undersampling balancing techniques on the performance of LM IDS frameworks. Specifically, we address the research question: How does the sample distribution within a benchmark dataset affect the performance evaluation metrics of LM-oriented IDS models, either shallow or DNN? To this end, we adopt a multiclass supervised approach, classifying network activity into Normal, Exploitation of Remote Services, and Exploitation of Hashing Techniques. We scrutinize the effect of oversampling, undersampling, and hybrid-sampling techniques across 13 machine learning algorithms, nine shallow and four deep neural network techniques, using the LMD-2023 corpus. Our key findings reveal that balanced versions of the dataset generally improved performance. Shallow models trained on resampled data achieved a marginal convergence of approximately +0.05% in AUC and F1-score compared to the imbalanced scenario. Notably, DNN models exhibited a more substantial performance gain of around 3.5% across most balancing techniques. Furthermore, analysis of False Positive Rate (FPR) and False Negative Rate (FNR) revealed crucial trade-offs. Notably, while some balanced datasets led to near-zero FNR with ensemble methods like Bagging, others, particularly with DNNs and techniques like ADASYN, showed a higher propensity for false alarms. These observations underscore the critical role of data balancing in optimizing LM IDS performance and highlight the varying impact of different techniques on the FPR/FNR trade-off for shallow versus deep learning models.