A Clustering Method for Improving Performance of Anomaly-Based Intrusion Detection System

Intrusion detection system (IDS) has played an important role as a device to defend our networks from cyber attacks. However, since it is unable to detect unknown attacks, i.e., 0-day attacks, the ultimate challenge in intrusion detection field is how we can exactly identify such an attack by an automated manner. Over the past few years, several studies on solving these problems have been made on anomaly detection using unsupervised learning techniques such as clustering, one-class support vector machine (SVM), etc. Although they enable one to construct intrusion detection models at low cost and effort, and have capability to detect unforeseen attacks, they still have mainly two problems in intrusion detection: a low detection rate and a high false positive rate. In this paper, we propose a new anomaly detection method based on clustering and multiple one-class SVM in order to improve the detection rate while maintaining a low false positive rate. We evaluated our method using KDD Cup 1999 data set. Evaluation results show that our approach outperforms the existing algorithms reported in the literature; especially in detection of unknown attacks.

As the range of security attacks increases across diverse network applications, intrusion detection systems are of central interest. Such detection systems are more crucial for the Internet of Things (IoT) due to the voluminous and sensitive data it produces. However, the real-world network produces imbalanced traffic including different and unknown attack types. Due to this imbalanced nature of network traffic, the traditional learning-based detection techniques suffer from lower overall detection performance, higher false-positive rate, and lower minority-class attack detection rates. To address the issue, we propose a novel deep generative-based model called Class-wise Focal Loss Variational AutoEncoder (CFLVAE) which overcomes the data imbalance problem by generating new samples for minority attack classes. Furthermore, we design an effective and cost-sensitive objective function called Class-wise Focal Loss (CFL) to train the traditional Variational AutoEncoder (VAE). The CFL objective function focuses on different minority class samples and scrutinizes high-level feature representation of observed data. This leads the VAE to generate more realistic, diverse, and quality intrusion data to create a well-balanced intrusion dataset. The balanced dataset results in improving the intrusion detection accuracy of learning-based classifiers. Therefore, a Deep Neural Network (DNN) classifier with a unique architecture is then trained using the balanced intrusion dataset to enhance the detection performance. Moreover, we utilize a challenging and highly imbalanced intrusion dataset called NSL-KDD to conduct an extensive experiment with the proposed model. The results demonstrate that the proposed CFLVAE with DNN (CFLVAE-DNN) model obtains promising performance in generating realistic new intrusion data samples and achieves superior intrusion detection performance. Additionally, the proposed CFLVAE-DNN model outperforms several state-of-the-art data generation and traditional intrusion detection methods. Specifically, the CFLVAE-DNN achieves 88.08% overall intrusion detection accuracy and 3.77% false positive rate. More significantly, it obtains the highest low-frequency attack detection rates for U2R (79.25%) and R2L (67.5%) against all the state-of-the-art algorithms.

With the wide application of information technolo-gies in communication based train control (CB TC) systems, the information security risks it confronts to are ever increasing. It is very difficult to design an intrusion detection system (IDS) which can detect both known and unknown cyber attacks with high true positive rate (TPR) and low false positive rate (FPR), simultaneously. In this paper, we propose a hybrid intrusion detection system (RIDS) which is composed of a signature-based intrusion detection system (SIDS) and an anomaly-based intrusion detection system (AIDS). An improved stacking ensemble algorithm (ISEA) is proposed to enhance the TPR of the SIDS which uses weighted classification probabilities outputted by base-classifiers to generate meta-features for the meta-classifier. The FPR of the AIDS is significantly reduced through adopting the proposed false positive elimination strategy (FPES). In addition, a Bayesian optimization algorithm is used for hyper-parameters adjustment to optimize the overall performance of the RIDS. The proposed RIDS is evaluated using both a KDD99 dataset and a CBTC dataset collected on a hardware-in-the-Ioop simulation platform. For the KDD99 dataset, the TPR and FPR of the proposed RIDS are 98.6% and 1.3%, respectively. For the CBTC dataset, the TPR and FPR are 98.1% and 1.1 %, respectively. Based on the performance comparison among different RIDSs, it is can be concluded that the proposed RIDS is superior to the existing RIDS in both TPR and FPR.

In this paper, by combining the known deep learning related literature in the field of intrusion detection, and the existing in the design of intrusion detection system based on the deep learning common low detection efficiency and high rate of false positives, as well as the training data set used by the problems of unbalanced data, etc., through the depth study of feature extraction and data processing ability. To study the intrusion detection method of the Internet of Things based on deep learning. The work and achievements are as follows:First of all, will be used for feature selection embedded model (EM) and used in intrusion detection of the convolution neural network (CNN), combined with lightweight intrusion detection model (XCNN), because it is an embedded model (EM), so can reduce the impact on the equipment, at the same time compared with other algorithms, the computing time is shorter, the same or even better.Second, proposed to the Attention mechanism (Self - Attention) is applied to intrusion detection system, due to the nature of the Attention mechanism, the mechanism and long Attention can greatly reduce the training time, through introducing the residual network, application feature coding technique again the sparse features thickened into low density at high altitudes, and join to its relative position encoding, So it can sense global information and solve the long-term dependence problem of recurrent neural network.Finally, the results of the intrusion detection method were analyzed on the Pthon simulation platform, and the relevant indicators were verified by using THE NSL-KDD dataset, THE CIC-IDS2017 dataset and the CSE-CIC-IDS2018 dataset. The effectiveness of the method was proved by building a laboratory Internet of Things environment for actual testing. The experimental results show that the algorithm can greatly reduce training time, improve training efficiency, and have the same or even better performance results.

In order to solve the problem of low intrusion detection rate and weak generalization ability of Intrusion Detection System (IDS), it proposes a new hybrid method based on the relationship of feature and spatial correlation for IDS. The proposed IDS reduces the dimension of network data flow by spatial correlation-based dimension reduction method (SCDR). It improves the effectiveness of intrusion detection, and the negative feedback learning method can also improve the generalization ability of IDS. Experiments show that the method in this paper can improve the detection rate notably and also enhance the detection ability of unknown attacks.

Network intrusion detection system are significant features that contribute to the enterprises and organization network success. In the past decade, Intrusion Detection system (IDS) using several methods have been proposed and implemented to safeguard the networks within an organization to be reliable, secure, and readily available. Although, some of these methods are built utilizing machine learning concepts. IDS utilizing machine learning techniques are accurate, effective and efficient in spotting attacks. However, the machine learning model performance reduces with high dimensional intrusion dataset. Therefore, it is sacrosanct to implement an efficient feature selection technique that can generate a great impact on the classification stage. Also, the feature selection phase is the most serious phase in IDS based on machine learning. This phase is expensive both in time and efforts. Furthermore, most of the proposed machine learning intrusion detection system suffered from low detection accuracy and high false positive rate when the models are experimented on high dimensional dataset. The aim of this paper is to propose a binary firefly algorithm (BFFA) based feature selection for IDS. We first performed normalization in the first stage of the model. Subsequently, the BFFA algorithm was used for feature selection stage. We adopted random forest algorithm for the classification phase. The experiment was performed on high dimensional University of New South Wales-NB 2015 (UNSW-NB15) dataset with seventy five percent of the data used for training the model and twenty percent for testing. The findings showed an accuracy of 99.72%, detection rate of 99.84%, precision of 99.27%, recall of 99.84% and F-score of 99.56%. The results were gauge with the state-of-the-art results and our results were found outstanding.

Intrusion Detection Systems (IDS) are vital for securing cloud environments, where dynamic scalability and distributed architecture present significant security challenges. Accurate, real-time threat detection with minimal false positives is essential for effective cloud protection. Traditional machine learning models such as Support Vector Machines (SVM) and Random Forests have been applied in IDS but often struggle with high false positive rates and limited effectiveness against novel or zero-day attacks. A deep neural network (DNN) architecture is designed, incorporating multiple ReLU-activated hidden layers and dropout layers to prevent overfitting. The model is trained using the Adam optimizer and categorical cross-entropy loss to ensure efficient learning and responsiveness. Evaluation on NSL-KDD and CICIDS2017 datasets demonstrates high detection accuracy—98.7% and 97.4% respectively—along with reduced false positive rates and promising results in identifying zero-day attacks during simulation. Future work includes integrating federated learning to support distributed deployment and enhance privacy preservation.

Intrusion detection, as a technology used to monitor abnormal behavior and maintain network security, has attracted many researchers’ attention in recent years. Thereinto, association rule mining is one of the mainstream methods to construct intrusion detection systems (IDS). However, the existing association rule algorithms face the challenges of high false positive rate and low detection rate. Meanwhile, too many rules might lead to the uncertainty increase that affects the performance of IDS. In order to tackle the above problems, a modified genetic network programming (GNP) is proposed for class association rule mining. Specifically, based on the property that node connections in the directed graph structure of GNP can be used to construct attribute associations, we propose to introduce information gain into GNP node selection. The most important attributes are thus selected, and the irrelevant attributes are removed before the rule is extracted. Moreover, not only the uncertainty among the class association rules is alleviated and also time consumption is reduced. The extracted rules can be applied to any classifier without affecting the detection performance. Experiment results based on NSL-KDD and KDDCup99 verify the performance of our proposed algorithm.

Intrusion detection systems play a crucial role in preventing security threats and defending networks from attacks. Among the attacks, distributed Denial-of-Service (DDoS) attacks literally get into the network and, in addition, they are terribly troublesome to avoid. With the advent of unknown threats, traditional machine learning approaches are impacted by lower detection rates and higher false-positive rates. As a result, the DDoS detection system requires an over-performing machine learning classifier with minimal false-positive and high detection accuracy. In this context, we propose an Improved Deep Sparse Autoencoder-based Framework (EDSA) for DDoS Attack Detection with a cost minimization strategy. The sparse autoencoder is used for dataset extraction functionality, while the softmax layer is used for traffic classification as malicious or bengin. However, intrusion detection includes the risk elements of inaccurate prediction; hence, we have used research metrics such as accuracy, precision, detection rate and specificity for our model analysis. The proposed solution uses the CICDDoS 2019 datasets and demonstrates high detection accuracy with a much less false positives percentage.

CANET: A hierarchical CNN-Attention model for Network Intrusion Detection

Intrusion detection is an emerging area of research in the computer security and net-works with the growing usage of internet in everyday life. Most intrusion detection systems (IDSs) mostly use a single classifier algorithm to classify the network traffic data as normal behavior or anomalous. However, these single classifier systems fail to provide the best possible attack detection rate with low false alarm rate. In this paper,we propose to use a hybrid intelligent approach using a combination of classifiers in order to make the decision intelligently, so that the overall performance of the resul-tant model is enhanced. The general procedure in this is to follow the supervised or un-supervised data filtering with classifier or cluster first on the whole training dataset and then the output are applied to another classifier to classify the data. In this re- search, we applied Neural Network with Supervised and Unsupervised Learning in order to implement the intrusion detection system. Moreover, in this project, we used the method of Parallelization with real time application of the system processors to detect the systems intrusions.Using this method enhanced the speed of the intrusion detection. In order to train and test the neural network, NSLKDD database was used. Creating some different intrusion detection systems, each of which considered as a single agent, we precisely proceeded with the signature-based intrusion detection of the network.In the proposed design, the attacks have been classified into 4 groups and each group is detected by an Agent equipped with intrusion detection system (IDS).These agents act independently and report the intrusion or non-intrusion in the system; the results achieved by the agents will be studied in the Final Analyst and at last the analyst reports that whether there has been an intrusion in the system or not. Keywords: Intrusion Detection, Multi-layer Perceptron, False Positives, Signature- based intrusion detection, Decision tree, Nave Bayes Classifier

Chapter 5 - Intrusion Prevention and Detection Systems

Intrusion Detection Systems (IDSs) play a significant role in reducing dynamic cyber threats. However, current machine learning-centric IDSs are not without issues, as they may have a high false positive rate and suboptimal feature selection, resulting in a low detection rate. This paper proposes an ensemble IDS architecture that utilizes RFE and IG for feature selection, aiming to enhance anomaly detection performance and reduce computational intensity. We begin with a preprocessing pipeline that includes data cleaning, one-hot encoding of categorical features, and normalization to scale the features. The most discriminative attributes are selected to minimize redundancy. Then, the selected feature subset is fed to build a set of ensemble classifiers, including Random Forest, XGBoost, Extra Trees, and a weighted Voting Classifier. Extensive experimental results on the CIC-IDS2017 datasets demonstrate that the proposed ensemble-level approach outperforms in all aspects, achieving 97.5% accuracy, 97.2% precision, 97.8% recall, and 97.5% F1-score. Overall, the ensemble model exhibits an improvement in terms of recall and hence robustness compared to the two baseline classifiers, namely the standalone Random Forest (recall: 96.5%) and XGBoost (recall: 97.3%). We also conducted an ablation study that confirms the effectiveness of RFE and Information Gain by comparing settings with and without feature selection. These findings indicate that the proposed IDS architecture can be feasibly and scalably implemented for real-time network anomaly detection. Adaptive feature selection and deployment in a streaming setting could be investigated to enhance its resistance to novel attacks in the future.

Detecting anomalous traffic (and above all new ad-hoc attacks) with low false alarm rates is of primary interest in IP networks management. To this aim a key research topic in network security is represented by anomaly-based IDSs (Intrusion Detection Systems) thanks to their ability to face unknown attacks. Starting from more than a decade of research experience by the authors, the aim of this paper is to revise some of the most promising statistical approaches, namely Wavelets, Principal Component Analysis, CUSUM (cumulative sum control chart) and Information Theoretical methods (based on different definitions of the Entropy). Moreover, issues related to the choice of the relevant traffic parameters, use of sketches and availability of dataset for performance comparison are also discussed to highlight the main problems in intrusion detection.

Screening for aneuploidy using maternal age has a low detection rate and high false positive rate. Second trimester maternal serum screening increases trisomy 21 detection and decreases the false positive rate. First trimester screening would enable definitive diagnosis with chorionic villus sampling, and simple surgical termination of affected pregnancies would still be an option. Nuchal translucency (NT), free beta human chorionic gonadotrophin (f beta HCG) and maternal age were assessed in 302 patients before chorionic villus sampling. NT positively and f beta HCG negatively correlated with gestation, but neither correlated with maternal age nor with each other. Both NT and f beta HCG were increased in trisomy 21. NT was increased and f beta HCG was decreased in trisomy 18. Multivariate discriminant analysis enabled 87.5% detection of trisomy 21 in this high-risk population, for a 14% false positive rate. In a simulated normal population, using a risk cut-off of 1 in 250, 71% detection was achieved for a 7% false positive rate. The combination of NT, f beta HCG and maternal age is a simple, readily available and viable first trimester screening strategy.