Detection of Unauthorized Actions in Networks Using Wavelet Analysis

The paper proposes an approach to security information processing in order to detect computer attacks and network anomalies based on big data technologies. The main contribution of the work is in the development, implementation and investigation of the proposed combined framework for processing security data using parallel computing environment and measuring the performance of the implemented system for detection of network attacks and anomalies. The research goal is to increase the performance of attack detection (under the given requirements for accuracy of solutions) compared to the traditional IDS application. The implemented approach is built using the open source systems Snort and Spark. The paper discusses the capabilities and performance assessment of parallel data processing in order to detect computer attacks and network anomalies, as well as key principles of working with big data. The presented main results of an experimental performance evaluation of the applied approach confirm its high efficiency for analyzing network traffic and security events.

The complexity of the Internet and the volume of network traffic have dramatically increased in the last few years, making it more challenging to design scalable Network Traffic Monitoring and Analysis (NTMA) systems. Critical NTMA applications such as the detection of network attacks and anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. The high-dimensionality of network data provided by current network monitoring systems opens the door to the massive application of machine learning approaches to improve the detection and classification of network attacks and anomalies, but this higher dimensionality comes with an extra data processing overhead. In this paper we present Big-DAMA, a big data analytics framework (BDAF) for NTMA applications. Big-DAMA is a flexible BDAF, capable to analyze and store big amounts of both structured and unstructured heterogeneous data sources, with both stream and batch processing capabilities. Big-DAMA uses off-the-shelf big data storage and processing engines to offer both stream data processing and batch processing capabilities, decomposing separate engines for stream, batch and query, following a Data Stream Warehouse (DSW) paradigm. Big-DAMA implements several algorithms for anomaly detection and network security using supervised and unsupervised machine learning (ML) models, using off-the-shelf ML libraries. We apply Big-DAMA to the detection of different types of network attacks and anomalies, benchmarking multiple supervised ML models. Evaluations are conducted on top of real network measurements collected at the WIDE backbone network, using the well-known MAWILab dataset for attacks labeling. Big-DAMA can speed up computations by a factor of 10 when compared to a standard Apache Spark cluster, and can be easily deployed in cloud environments, using hardware virtualization technology.

As network attacks are evolving along with extreme growth in the amount of data that is present in networks, there is a significant need for faster and more effective anomaly detection methods. Even though current systems perform well when identifying known attacks, previously unknown attacks are still difficult to identify under occurrence. To emphasize, attacks that might have more than one ongoing attack vectors in one network at the same time, or also known as APT (Advanced Persistent Threat) attack, may be hardly notable since it masquerades itself as legitimate traffic. Furthermore, with the help of hiding functionality, this type of attack can even hide in a network for years. Additionally, the expected number of connected devices as well as the fast-paced development caused by the Internet of Things, raises huge risks in cyber security that must be dealt with accordingly. When considering all above-mentioned reasons, there is no doubt that there is plenty of room for more advanced methods in network anomaly detection hence more advanced statistical methods and machine learning based techniques have been proposed recently in detecting anomalies. The papers reviewed showed that different methods vary greatly in their performance to detect anomalies. Every method had its advantages and disadvantages, however most of the presented methods cannot detect previously unknown attacks but on the contrary, for example, detects DDoS attacks extremely well.

In this era of digital transformation, the importance of network anomaly detection has been amplified to safeguard the security and integrity of various vital applications. These applications include the protection of critical infrastructures, prevention of cyber-attacks, and upkeep of network performance, among others. Our study presents a thorough evaluation of an array of machine learning algorithms for the proficient detection of network anomalies. These algorithms include, but are not limited to, Random Forest, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM). We illustrate an in-depth comparison between these chosen algorithms, analyzing their performance metrics, strengths, and weaknesses, with a specific focus on their practical applicability and influence on network security methodologies. The investigation into these machine learning algorithms exhibits potential advantages and constraints of employing such methodologies for network anomaly detection. We further shed light on the determinants that affect their performance in diverse scenarios. Based on the exhaustive analysis, we provide suggestive guidelines for choosing the most suitable algorithm depending on specific use cases or requirements. This study thus serves as a comprehensive guide to understanding the role and impact of machine learning in the critical field of network anomaly detection.

Last decades have witnessed the monumental growth of Inter-net applications, and the network security fields have attracted increasing interests as it has become a necessity due to proliferation of information technologies in everyday life. Net-work anomaly detection aims to detect the potential anomalous behavior in traffic data, which is critical for detecting and reacting to the presence of attacks in the network. In this pa-per, we present our solution for the ICASSP 2021 Network Anomaly Detection Challenge (NAD) challenge. Firstly, we present our feature extraction method in detecting network anomalies, as efficiently extracting the discriminate features from the traffic data over time can be greatly helpful to improve the anomaly detection performance. Secondly, both gradient boosting decision trees and TabTransformer-Based classification model are trained for the multi-class classification task. The TabTransformer is based on self-attention and can transform the categorical feature' embeddings into robust contextual embeddings, thus, can achieve higher prediction accuracy. Finally, we ensemble the results to improve the performance further. To demonstrate the effectiveness of our approach, extensive experiments have been conducted on the NAD datasets. Our approach achieves the score of 0.625 and ranked as 2nd place in the final leaderboard.

Internet of Things (IoT) is a technology that allows various physical devices to connect to each other and exchange data via the internet network. The application of IoT is increasingly widespread in various sectors such as smart homes, manufacturing industries, smart agriculture, and healthcare. However, along with the increasing number of devices and the volume of data traffic sent, the potential risk of cybersecurity threats is also increasing. The large number of IoT devices that have limited computing capabilities makes the system more vulnerable to various attacks, including intrusion, exploitation of system weaknesses, and Distributed Denial of Service (DDoS) attacks. Therefore, early detection of anomalies in network traffic is a crucial aspect to maintain the security and stability of IoT systems. This study aims to develop and implement a Support Vector Machine (SVM)-based architecture as a classification method in an anomaly detection system on an IoT network. SVM was chosen because of its ability to handle high-dimensional data and non-linear classification effectively. The methodology used includes the process of extracting features from IoT network traffic datasets, data normalization, model training using the SVM algorithm, and evaluating model performance in distinguishing between normal and anomalous traffic. Thus, the implementation of SVM architecture can be an effective and efficient solution in intrusion detection systems for IoT networks. This research also opens up opportunities for the development of more adaptive security systems by integrating machine learning-based detection models into large-scale IoT infrastructures.

The importance of the ability to predict trends in social media has been growing rapidly in the past few years with the growing dominance of social media in our everyday’s life. Whereas many works focus on the detection of anomalies in networks, there exist little theoretical work on the prediction of the likelihood of anomalous network pattern to globally spread and become “trends”. In this work we present an analytic model for the social diffusion dynamics of spreading network patterns. Our proposed method is based on information diffusion models, and is capable of predicting future trends based on the analysis of past social interactions between the community’s members. We present an analytic lower bound for the probability that emerging trends would successfully spread through the network. We demonstrate our model using two comprehensive social datasets — the Friends and Family experiment that was held in MIT for over a year, where the complete activity of 140 users was analyzed, and a financial dataset containing the complete activities of over 1.5 million members of the eToro social trading community.

The detection of anomalies in network traffic is a crucial issue affecting the security of Internet users. A statistical network anomaly detection algorithm is a promising way of detecting such anomalies, however, it has to be given appropriate parameters for accurate detection and identification. In general, it is very difficult to obtain appropriate parameter settings a priori, because network traffic is not stable in time or space. Thus, although many anomaly detection methods have been proposed, there has been little discussion about their parameter tunings. In this paper, we investigate an automatic and dynamic parameter tuning of a statistical network traffic anomaly detection method. In particular, we clarify whether one can consistently use the best parameter fixed for a certain instance; this choice clearly depends on the macroscopic and dynamic behavior of Internet traffic anomalies. We ascertain the appropriate learning period for setting a parameter of an anomaly detection algorithm based on a sketch and multi-scale gamma-function model by using real network traces measured in a trans-Pacific link over a period of six months. The main results of our study are as follows: (1) Without learning, the best parameter varies day by day. (2) With a longer learning period, the best parameter setting is affected by significant data during the learning period. (3) The appropriate period of the learning is about 3 days. (4) The performance degradation from introducing dynamic parameter tuning is 17% in the best case.

Network anomalies are unusual traffic mainly induced by network attacks or network failures. Therefore it is important for network operators as end users to detect and diagnose them to protect their network. However, these anomalies keep changing in time, it is therefore important to propose detectors which can learn from the traffic and spot anomalies without relying on any previous knowledge. Unsupervised network anomaly detectors reach this goal by taking advantage of machine learning and statistical techniques to spot the anomalies. There exists many unsupervised network anomaly detectors in the literature. Each algorithm puts forward its good detection performance, therefore it is difficult to select one detector among the large set of available detectors. Therefore, this paper, presents an extensive study and assessment of a set of well known unsupervised network anomaly detectors, and underlines their strengths and weaknesses. This study overwhelms previous similar evaluation by considering for the comparison some new, original and of premier importance parameters as detection similarity, detectors sensitivity and curse of dimensionality, together with the classical detection performance, and execution time parameters.

The present study, aimed to detect cyber-attacks, and unexpected access requests on devices in the telecommunication networks, enabling the necessary measures to be taken early. With K-Nearest Neighbors (KNN) and Naive Bayes machine learning methods, predicted whether the raw data packets contain cyber-attack according to different properties of these packets using the UNSW-NB15 dataset. KNN algorithms with different K values and the Naive Bayes method were compared according to accuracy rates and the results were given in the table. As a result, changes in accuracy rates were observed according to different k neighbor values in the KNN algorithm. Higher accuracy rates than Naive Bayes were achieved in the models created with the KNN algorithm.

The article deals with detection of network anomalies. Network anomalies include everything that is quite different from the normal operation. For detection of anomalies were used machine learning systems. Machine learning can be considered as a support or a limited type of artificial intelligence. A machine learning system usually starts with some knowledge and a corresponding knowledge organization so that it can interpret, analyse, and test the knowledge acquired. There are several machine learning techniques available. We tested Decision tree learning and Bayesian networks. The open source data-mining framework WEKA was the tool we used for testing the classify, cluster, association algorithms and for visualization of our results. The WEKA is a collection of machine learning algorithms for data mining tasks.

The prosperity of mobile networks and social networks brings revolutionary conveniences to our daily lives. However, due to the complexity and fragility of the network environment, network attacks are becoming more and more serious. Characterization of network traffic is commonly used to model and detect network anomalies and finally to raise the cybersecurity awareness capability of network administrators. As a tool to characterize system running status, entropy-based time-series complexity measurement methods such as Multiscale Entropy (MSE), Composite Multiscale Entropy (CMSE), and Fuzzy Approximate Entropy (FuzzyEn) have been widely used in anomaly detection. However, the existing methods calculate the distance between vectors solely using the two most different elements of the two vectors. Furthermore, the similarity of vectors is calculated using the Heaviside function, which has a problem of bouncing between 0 and 1. The Euclidean Distance-Based Multiscale Fuzzy Entropy (EDM-Fuzzy) algorithm was proposed to avoid the two disadvantages and to measure entropy values of system signals more precisely, accurately, and stably. In this paper, the EDM-Fuzzy is applied to analyze the characteristics of abnormal network traffic such as botnet network traffic and Distributed Denial of Service (DDoS) attack traffic. The experimental analysis shows that the EDM-Fuzzy entropy technology is able to characterize the differences between normal traffic and abnormal traffic. The EDM-Fuzzy entropy characteristics of ARP traffic discovered in this paper can be used to detect various types of network traffic anomalies including botnet and DDoS attacks.

Anomaly detection systems are significant in recognizing intruders or suspicious activities by detecting unseen and unknown attacks. In this paper, we have worked on a benchmark network anomaly detection dataset UNSW-NB15, that reflects modern-day network traffic. Previous works on this dataset either lacked a proper validation approach or followed only one evaluation setup which made it difficult to compare their contributions with others using the same dataset but with different validation steps. In this paper, we have used a machine learning classifier LightGBM to perform binary classification on this dataset. We have presented a thorough study of the dataset with feature engineering, preprocessing, feature selection. We have evaluated the performance of our model using different experimental setups (used in several previous works) to clearly evaluate and compare with others. Using ten-fold cross-validation on the train, test, and combined (training and test) dataset, our model has achieved 97.21%, 98.33%, and 96.21% f1_scores, respectively. Also, the model fitted only on train data, achieved 92.96% f1_score on the separate test data. So our model also provides significant performance on unseen data. We have presented complete comparisons with the prior arts using all performance metrics available on them. And we have also shown that our model outperformed them in most metrics and thus can detect network anomalies better.

This paper explores the potential of machine-mastering models to automate network Anomaly Detection (NAD) through Time series analysis. We employ a two-level method wherein the primary degree entails function selection thru foremost component analysis (PCA), accompanied by gadget mastering (ML) model choice from more than a few supervised studying algorithms. The second stage evaluates the overall performance of the numerous selected ML models and optimizes theirhyperparameters when necessary. Our experiments demonstrate that ML-driven computerized network Anomaly Detection can provide accurate and well-timed detection of network anomalies with little supervision and parameter tuning attempts. The outcomes of our experiments display that Random Forests and Support Vector Machines (SVMs) carry out first-rate some of the model’s grid searches, demonstrating aggressive accuracy and precision ratings from an anomaly detection perspective. We also intensely evaluate the consequences and provide insightful discussion on the possibilities and challenges surrounding using ML for automatic community Anomaly Detection.

This paper proposes a novel detection engine, called the Wavelet-Recurrence-Clustering (WRC) detection model, to study the network anomaly detection problem that is widely attractive in Internet security area. The WRC model firstapplies the wavelet transform and recurrence analysis to calculate the multi-scale dynamic characteristics of network traffic, and then identifies network anomalies throughthe clustering algorithm with those dynamic characteristics. The evaluation results on DARPA 1999 dataset indicate that the WRC detection model can effectively improve the detection accuracy with a low false alarm rate.