Network Intrusion Detection Based on Supervised Adversarial Variational Auto-Encoder With Regularization

The security of the Industrial Internet of Things (IIoT) is of vital importance, and the Network Intrusion Detection System (NIDS) plays an indispensable role in this. Although there is an increasing number of studies on the use of deep learning technology to achieve network intrusion detection, the limited local data of the device may lead to poor model performance because deep learning requires large-scale datasets for training. Some solutions propose to centralize the local datasets of devices for deep learning training, but this may involve user privacy issues. To address these challenges, this study proposes a novel federated learning (FL)-based approach aimed at improving the accuracy of network intrusion detection while ensuring data privacy protection. This research combines convolutional neural networks with attention mechanisms to develop a new deep learning intrusion detection model specifically designed for the IIoT. Additionally, variational autoencoders are incorporated to enhance data privacy protection. Furthermore, an FL framework enables multiple IIoT clients to jointly train a shared intrusion detection model without sharing their raw data. This strategy significantly improves the model's detection capability while effectively addressing data privacy and security issues. To validate the effectiveness of the proposed method, a series of experiments were conducted on a real-world Internet of Things (IoT) network intrusion dataset. The experimental results demonstrate that our model and FL approach significantly improve key performance metrics such as detection accuracy, precision, and false-positive rate (FPR) compared to traditional local training methods and existing models.

As the range of security attacks increases across diverse network applications, intrusion detection systems are of central interest. Such detection systems are more crucial for the Internet of Things (IoT) due to the voluminous and sensitive data it produces. However, the real-world network produces imbalanced traffic including different and unknown attack types. Due to this imbalanced nature of network traffic, the traditional learning-based detection techniques suffer from lower overall detection performance, higher false-positive rate, and lower minority-class attack detection rates. To address the issue, we propose a novel deep generative-based model called Class-wise Focal Loss Variational AutoEncoder (CFLVAE) which overcomes the data imbalance problem by generating new samples for minority attack classes. Furthermore, we design an effective and cost-sensitive objective function called Class-wise Focal Loss (CFL) to train the traditional Variational AutoEncoder (VAE). The CFL objective function focuses on different minority class samples and scrutinizes high-level feature representation of observed data. This leads the VAE to generate more realistic, diverse, and quality intrusion data to create a well-balanced intrusion dataset. The balanced dataset results in improving the intrusion detection accuracy of learning-based classifiers. Therefore, a Deep Neural Network (DNN) classifier with a unique architecture is then trained using the balanced intrusion dataset to enhance the detection performance. Moreover, we utilize a challenging and highly imbalanced intrusion dataset called NSL-KDD to conduct an extensive experiment with the proposed model. The results demonstrate that the proposed CFLVAE with DNN (CFLVAE-DNN) model obtains promising performance in generating realistic new intrusion data samples and achieves superior intrusion detection performance. Additionally, the proposed CFLVAE-DNN model outperforms several state-of-the-art data generation and traditional intrusion detection methods. Specifically, the CFLVAE-DNN achieves 88.08% overall intrusion detection accuracy and 3.77% false positive rate. More significantly, it obtains the highest low-frequency attack detection rates for U2R (79.25%) and R2L (67.5%) against all the state-of-the-art algorithms.

The integration of industrial control communication networks and the Internet into the Industrial Control System (ICS) increases their vulnerability to cyber-attacks, leading to devastating consequences. Traditional intrusion detection systems (IDS) mostly rely on predefined models and are trained mostly on specific cyber attacks, which means that traditional IDS cannot deal with unknown attacks In addition, most IDSs do not take into account the imbalance of ICS datasets, and therefore suffer from low accuracy and high false positives when used. In the article, we propose an NCO-double-layer DIFF_RF-OPFYTHON intrusion detection method for ICS, which consists of NCO modules, two-layer DIFF_RF modules, and OPFYTHON modules. Detected traffic will be divided into three categories by the two-layer DIFF_RF module: known attacks, unknown attacks, and normal traffic. Next, the known attacks will be classified by the OPFYTHON module into specific attacks according to the characteristics of the attack traffic. We use the NCO module to improve model inputs and improve model accuracy. The results show that the proposed method outperforms traditional intrusion detection methods such as XGboost and SVM. Detecting unknown attacks is also significant. The accuracy of the data set used in this article reaches 98.13%. The detection rate of unknown and known attacks reaches 98.21% and 95.1%, respectively. In this article, the basic modules of the classifier are shallow machine learning algorithms. This can be improved by using a more powerful neural network architecture. In the article, all detected unknown attacks are marked as one category. In further studies, the unknown attacks can be clustered and analyzed to further classify the detected unknown attacks. It is worth noting that the number of unknown attacks is only a small part, so it is difficult to classify unknown attacks using cluster analysis. Since training a more powerful neural network requires a lot of data, it is important to investigate how to train a new model when the number of samples belonging to that class is limited.

Network intrusion detection systems (NIDS) are essential security building-blocks for today's organizations to ensure safe and trusted communication of information. In this paper, we study the feasibility of off-line deep learning based NIDSes by constructing the detection engine with multiple advanced deep learning models and conducting a quantitative and comparative evaluation of those models. We first introduce the general deep learning methodology and its potential implication on the network intrusion detection problem. We then review multiple machine learning solutions to two network intrusion detection tasks (NSL-KDD and UNSW-NB15 datasets). We develop a TensorFlow-based deep learning library, called NetLearner, and implement a handful of cutting-edge deep learning models for NIDS. Finally, we conduct a quantitative and comparative performance evaluation of those models using NetLearner.

With the swift growth of digital networks and information in both public and private sectors, it is essential to deal with the considerable threat that network attacks pose to data integrity and confidentiality. Consequently, there is a pressing requirement for the establishment of effective mechanisms to detect and provide recommendations for addressing intrusion attacks. In this paper, we propose a semantic-based intrusion detection system that aims to improve performance by incorporating semantic representations consisting of feature groups and their associated weights, leading to the creation of a weighted knowledge graph. The weights of the features are determined using sparse autoencoders. From these weights, the most significant features are normalized to a specific range. This approach comprises a combination of a Deep Auto Encoder (AE) and Long Short-Term Memory (LSTM) networks for intrusion detection. Furthermore, the ensemble method of Extreme Gradient Boosting (XGBoost) is used to identify and recommend high-probability attack scenarios. The dataset used to evaluate is the CSE-CIC-IDS dataset. Performance metrics such as accuracy, precision, recall, false positive rate, receiver operating characteristic metrics, loss, and error rate are used to measure the performance, and the results show the approach demonstrates substantial improvements in detection accuracy, minimizing false positives, enhancing reliability, and outperforming existing models. The combination of semantic knowledge, deep learning, and ensemble learning ensures a proactive and adaptive cybersecurity framework.

Advancements in machine learning and artificial intelligence have been widely utilised in the security domain, including but not limited to intrusion detection techniques. With the large training datasets of modern traffic, intelligent algorithms and powerful machine learning tools, security researchers have been able to greatly improve on the intrusion detection models and enhance their ability to detect malicious traffic more accurately. Nonetheless, the problem of detecting completely unknown security attacks is still an open area of research. The enormous number of newly developed attacks constitutes an eccentric challenge for all types of intrusion detection systems. Additionally, the lack of a standard definition of what constitutes an unknown security attack in the literature and the industry alike adds to the problem. In this paper, the researchers reviewed the studies on detecting unknown attacks over the past 10 years and found that they tended to use inconsistent definitions. This formulates the need for a standard consistent definition to have comparable results. The researchers proposed a new categorisation of two types of unknown attacks, namely Type-A, which represents a completely new category of unknown attacks, and Type-B, which represents unknown attacks within already known categories of attacks. The researchers conducted several experiments and evaluated modern intrusion detection systems based on shallow and deep artificial neural network models and their ability to detect Type-A and Type-B attacks using two well-known benchmark datasets for network intrusion detection. The research problem was studied as both a binary and multi-class classification problem. The results showed that the evaluated models had poor overall generalisation error measures, where the classification error rate in detecting several types of unknown attacks from 92 experiments was 50.09%, which highlights the need for new approaches and techniques to address this problem.

Traditional firewalls and data encryption techniques can no longer match the demands of current IoT network security due to the rising amount and variety of network threats. In order to manage IoT network risks, intrusion detection solutions have been advised. Even though machine learning (ML) helps the widely used intrusion detection techniques currently in use, these algorithms struggle with low detection rates and the requirement for extensive feature engineering. The deep learning model for IoT network intrusion detection is a method for traffic anomaly detection that is suggested by this study. To extract the sequence properties of data flow through a CNN, it combines an attention mechanism with a Long Short Term Memory (LSTM) network. This method uses adaptive synthetic sampling (ADASYN) to increase the size of minority-class samples. The proposed models demonstrated acceptable precision and recall for each class when used for binary-class classification, proving their stability and capacity to identify all classes correctly. The MLP classifier’s accuracy, precision, recall, and F1 value were 87%, 89%, 87%, and 89%, respectively, with an AUC score of 0.88. Overall, the proposed models performed well. The attack and all-class models exhibited good AUCs and macro metrics, the same as the proposed MLP classifier, which had an F1 score of 83% and an AUC score of 0.94. Additionally, it trained the MLP classifier and integrated the ADAM optimizer and category cross-entropy loss function for all-class classification. With an AUC value of 94%, it possessed 84% accuracy, 87% precision, 84% recall, and an 83% F1 score. A further indication of the hybrid model’s ability to combine the benefits of both models to improve overall performance was that it regularly outperformed the MLP model. This model’s accuracy and F1 score are better than those of earlier comparable algorithms, according to experimental results using the publicly accessible benchmark dataset for network intrusion detection (NSL–KDD).

Intrusion detection has been a vast-surveyed topic for many decades as network attacks are tremendously growing. This has heightened the need for security in networks as web-based communication systems are advanced nowadays. The proposed work introduces an intelligent semi-supervised intrusion detection system based on different algorithms to classify the network attacks accurately. Initially, the pre-processing is accomplished using null value dropping and standard scaler normalization. After pre-processing, an enhanced Deep Reinforcement Learning (EDRL) model is employed to extract high-level representations and learn complex patterns from data by means of interaction with the environment. The enhancement of deep reinforcement learning is made by associating a deep autoencoder (AE) and an improved flamingo search algorithm (IFSA) to approximate the Q-function and optimal policy selection. After feature representations, a support vector machine (SVM) classifier, which discriminates the input into normal and attack instances, is employed for classification. The presented model is simulated in the Python platform and evaluated using the UNSW-NB15, CICIDS2017, and NSL-KDD datasets. The overall classification accuracy is 99.6%, 99.93%, and 99.42% using UNSW-NB15, CICIDS2017, and NSL-KDD datasets, which is higher than the existing detection frameworks.

Intrusion detection system (IDS) has played an important role as a device to defend our networks from cyber attacks. However, since it is unable to detect unknown attacks, i.e., 0-day attacks, the ultimate challenge in intrusion detection field is how we can exactly identify such an attack by an automated manner. Over the past few years, several studies on solving these problems have been made on anomaly detection using unsupervised learning techniques such as clustering, one-class support vector machine (SVM), etc. Although they enable one to construct intrusion detection models at low cost and effort, and have capability to detect unforeseen attacks, they still have mainly two problems in intrusion detection: a low detection rate and a high false positive rate. In this paper, we propose a new anomaly detection method based on clustering and multiple one-class SVM in order to improve the detection rate while maintaining a low false positive rate. We evaluated our method using KDD Cup 1999 data set. Evaluation results show that our approach outperforms the existing algorithms reported in the literature; especially in detection of unknown attacks.

Due to the recent advances in the Internet and communication technologies, network systems and data have evolved rapidly. The emergence of new attacks jeopardizes network security and make it really challenging to detect intrusions. Multiple network attacks by an intruder are unavoidable. Our research targets the critical issue of class imbalance in intrusion detection, a reflection of the real-world scenario where legitimate network activities significantly out number malicious ones. This imbalance can adversely affect the learning process of predictive models, often resulting in high false-negative rates, a major concern in Intrusion Detection Systems (IDS). By focusing on datasets with this imbalance, we aim to develop and refine advanced algorithms and techniques, such as anomaly detection, cost-sensitive learning, and oversampling methods, to effectively handle such disparities. The primary goal is to create models that are highly sensitive to intrusions while minimizing false alarms, an essential aspect of effective IDS. This approach is not only practical for real-world applications but also enhances the theoretical understanding of managing class imbalance in machine learning. Our research, by addressing these significant challenges, is positioned to make substantial contributions to cybersecurity, providing valuable insights and applicable solutions in the fight against digital threats and ensuring robustness and relevance in IDS development. An intrusion detection system (IDS) checks network traffic for security, availability, and being non-shared. Despite the efforts of many researchers, contemporary IDSs still need to further improve detection accuracy, reduce false alarms, and detect new intrusions. The mean convolutional layer (MCL), feature-weighted attention (FWA) learning, a bidirectional long short-term memory (BILSTM) network, and the random forest algorithm are all parts of our unique hybrid model called MCL-FWA-BILSTM. The CNN-MCL layer for feature extraction receives data after preprocessing. After convolution, pooling, and flattening phases, feature vectors are obtained. The BI-LSTM and self-attention feature weights are used in the suggested method to mitigate the effects of class imbalance. The attention layer and the BI-LSTM features are concatenated to create mapped features before feeding them to the random forest algorithm for classification. Our methodology and model performance were validated using NSL-KDD and UNSW-NB-15, two widely available IDS datasets. The suggested model's accuracies on binary and multi-class classification tasks using the NSL-KDD dataset are 99.67% and 99.88%, respectively. The model's binary and multi-class classification accuracies on the UNSW-NB15 dataset are 99.56% and 99.45%, respectively. Further, we compared the suggested approach with other previous machine learning and deep learning models and found it to outperform them in detection rate, FPR, and F-score. For both binary and multiclass classifications, the proposed method reduces false positives while increasing the number of true positives. The model proficiently identifies diverse network intrusions on computer networks and accomplishes its intended purpose. The suggested model will be helpful in a variety of network security research fields and applications.

A NSGA2-LR wrapper approach for feature selection in network intrusion detection

Enhancing CNN-based network intrusion detection through hyperparameter optimization

With a paradigm shift to untact environments, security threats on the network also have been significantly increasing all over the world. To monitor and detect intrusion attempts under enormous network traffic, Security Operation Center (SOC) essentially exploits various security devices. Above all, Network Intrusion Detection System (NIDS) has been operated in public/private sectors as a spearhead to fight against cyber threats. In particular, state-of-the-art technologies, especially ML and AI, have been being studied to achieve quick and accurate intrusion detection. Despite much effort to guarantee a secure network, however, SOCs are still struggling for overcoming various types of threats as well as attacks of similar form with benign traffic. Even though the advanced techniques may find out a complex and unknown attack, operating and managing them in real-world situations cause counterproductively more pressure to agents in the SOC. In order to solve these difficulties, this study introduces an easy-to-use framework to build intrusion detection models based on AI techniques, as well as to operate them depending on a situation using a graphical user interface. The framework supports generating various types of AI- and ML-based intrusion detection models with optimized parameters by only a few steps. Furthermore, an interactive graphical interface makes it easier to manage detection models according to different threat situations. Finally, the performance of models made by the framework is evaluated in terms of accuracy, especially under the real-world SOC environment with live network traffic.

Federated transfer learning-based intrusion detection system in 5G networks

With the advent of the Internet of Things (IoT), the security of the network layer in the IoT is getting more and more attention. The traditional intrusion detection technologies cannot be well adapted in the complex Internet environment of IoT. For the deep learning algorithm of intrusion detection, a neural network structure may have fine detection accuracy for one kind of attack, but it may not have a good detection effect when facing other attacks. Therefore, it is urgent to design a self-adaptive model to change the network structure for different attack types. This paper presents an intrusion detection model based on improved genetic algorithm (GA) and deep belief network (DBN). Facing different types of attacks, through multiple iterations of the GA, the optimal number of hidden layers and number of neurons in each layer are generated adaptively, so that the intrusion detection model based on the DBN achieves a high detection rate with a compact structure. Finally, the NSL-KDD dataset was used to simulate and evaluate the model and algorithms. The experimental results show that the improved intrusion detection model combined with DBN can effectively improve the recognition rate of intrusion attacks and reduce the complexity of the neural network structure.