Attention-Based Anomaly Detection in Dynamic Network

A multi-scale reconstruction method for the anomaly detection in stochastic dynamic networks

The paper proposes an approach to security information processing in order to detect computer attacks and network anomalies based on big data technologies. The main contribution of the work is in the development, implementation and investigation of the proposed combined framework for processing security data using parallel computing environment and measuring the performance of the implemented system for detection of network attacks and anomalies. The research goal is to increase the performance of attack detection (under the given requirements for accuracy of solutions) compared to the traditional IDS application. The implemented approach is built using the open source systems Snort and Spark. The paper discusses the capabilities and performance assessment of parallel data processing in order to detect computer attacks and network anomalies, as well as key principles of working with big data. The presented main results of an experimental performance evaluation of the applied approach confirm its high efficiency for analyzing network traffic and security events.

Social networks are becoming more prevalent all across the globe. With all of its advantages, criminality and fraudulent conduct in this medium are on the rise. As a result, there is an urgent need to detect abnormalities in these networks before they do substantial harm. Traditional Non-Deep Learning (NDL) approaches fails to perform effectively when the size and scope of real-world social networks increase. As a result, DL techniques for anomaly detection in social networks are required. Several studies have been conducted using DL on node and edge anomaly detection. However, in the current scenario, subgraph anomaly detection utilizing Deep Learning (DL) is still in its nascent stages. This paper proposes a method called Clustering-based Deep Autoencoders (CDA) to detect subgraph anomalies in static attributed social networks. It converts the input graph into node embeddings using an encoder, clusters these nodes into communities or subgraphs, and then finds anomalies among these subgraph embeddings. The model is tested on seven open-access social network datasets, and the findings indicate that the proposed model detects the most anomalies. In the future, it is also recommended that the present experiment be aimed at dynamic social networks.

As the Internet is growing drastically, interest in online social networks has also dramatically increased over the past ten years. They are among the most widely used websites on the Internet, with applications in practically every aspect of life, including telemarketing, entertainment, healthcare, and business. However, as they are being used broadly, they have become the main targets for malevolent individuals who try to engage in a wide range of fraudulent or unlawful activities and harm other users. Therefore, it is vital to detect unusual activity, especially in social networks, as this enables the identification of crucial data pertaining to the behavior of unusual individuals (Anomaly detection). It is challenging to identify unusual and unpredicted user behavior by diving into the network's hidden patterns, as these users' engagement patterns differ greatly from those of other users. Although numerous techniques have been created for identifying anomalies in various problematic environments, this discipline is quite new and expanding quickly. As a result, this growing field of anomaly identification in social networks requires systematic analysis. By following the systematic literature review practice, the most relevant research papers from the data resources are selected by using search string keywords, de fi ne d inclusion/exclusion criteria, and the proper analysis of the selected research papers. The data collected is used to answer research questions. An exhaustive analysis of different techniques and approaches is performed by assessing them on the research methodology. This entire activity took around three to four months for a thorough analysis of relevant research papers. Upcoming researchers may work on the research gaps by checking on the proposals mentioned in this review paper.

The detection of anomalies in network traffic is a crucial issue affecting the security of Internet users. A statistical network anomaly detection algorithm is a promising way of detecting such anomalies, however, it has to be given appropriate parameters for accurate detection and identification. In general, it is very difficult to obtain appropriate parameter settings a priori, because network traffic is not stable in time or space. Thus, although many anomaly detection methods have been proposed, there has been little discussion about their parameter tunings. In this paper, we investigate an automatic and dynamic parameter tuning of a statistical network traffic anomaly detection method. In particular, we clarify whether one can consistently use the best parameter fixed for a certain instance; this choice clearly depends on the macroscopic and dynamic behavior of Internet traffic anomalies. We ascertain the appropriate learning period for setting a parameter of an anomaly detection algorithm based on a sketch and multi-scale gamma-function model by using real network traces measured in a trans-Pacific link over a period of six months. The main results of our study are as follows: (1) Without learning, the best parameter varies day by day. (2) With a longer learning period, the best parameter setting is affected by significant data during the learning period. (3) The appropriate period of the learning is about 3 days. (4) The performance degradation from introducing dynamic parameter tuning is 17% in the best case.

The complexity of the Internet and the volume of network traffic have dramatically increased in the last few years, making it more challenging to design scalable Network Traffic Monitoring and Analysis (NTMA) systems. Critical NTMA applications such as the detection of network attacks and anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. The high-dimensionality of network data provided by current network monitoring systems opens the door to the massive application of machine learning approaches to improve the detection and classification of network attacks and anomalies, but this higher dimensionality comes with an extra data processing overhead. In this paper we present Big-DAMA, a big data analytics framework (BDAF) for NTMA applications. Big-DAMA is a flexible BDAF, capable to analyze and store big amounts of both structured and unstructured heterogeneous data sources, with both stream and batch processing capabilities. Big-DAMA uses off-the-shelf big data storage and processing engines to offer both stream data processing and batch processing capabilities, decomposing separate engines for stream, batch and query, following a Data Stream Warehouse (DSW) paradigm. Big-DAMA implements several algorithms for anomaly detection and network security using supervised and unsupervised machine learning (ML) models, using off-the-shelf ML libraries. We apply Big-DAMA to the detection of different types of network attacks and anomalies, benchmarking multiple supervised ML models. Evaluations are conducted on top of real network measurements collected at the WIDE backbone network, using the well-known MAWILab dataset for attacks labeling. Big-DAMA can speed up computations by a factor of 10 when compared to a standard Apache Spark cluster, and can be easily deployed in cloud environments, using hardware virtualization technology.

Social media networks are dynamic. As such, the order in which network ties develop is an important aspect of the network dynamics.This study proposes a novel dynamic network model, the Nodal Attribute-based Temporal Exponential Random Graph Model (NATERGM) for dynamic network analysis. The proposed model focuses on how the nodal attributes of a network affect the order in which the network ties develop. Empirical results showed that the NATERGM demonstrated an enhanced pattern testing capability compared to benchmark models. The proposed NATERGM model helps explain the roles of nodal attributes in the formation process of dynamic networks.

The importance of the ability to predict trends in social media has been growing rapidly in the past few years with the growing dominance of social media in our everyday’s life. Whereas many works focus on the detection of anomalies in networks, there exist little theoretical work on the prediction of the likelihood of anomalous network pattern to globally spread and become “trends”. In this work we present an analytic model for the social diffusion dynamics of spreading network patterns. Our proposed method is based on information diffusion models, and is capable of predicting future trends based on the analysis of past social interactions between the community’s members. We present an analytic lower bound for the probability that emerging trends would successfully spread through the network. We demonstrate our model using two comprehensive social datasets — the Friends and Family experiment that was held in MIT for over a year, where the complete activity of 140 users was analyzed, and a financial dataset containing the complete activities of over 1.5 million members of the eToro social trading community.

Last decades have witnessed the monumental growth of Inter-net applications, and the network security fields have attracted increasing interests as it has become a necessity due to proliferation of information technologies in everyday life. Net-work anomaly detection aims to detect the potential anomalous behavior in traffic data, which is critical for detecting and reacting to the presence of attacks in the network. In this pa-per, we present our solution for the ICASSP 2021 Network Anomaly Detection Challenge (NAD) challenge. Firstly, we present our feature extraction method in detecting network anomalies, as efficiently extracting the discriminate features from the traffic data over time can be greatly helpful to improve the anomaly detection performance. Secondly, both gradient boosting decision trees and TabTransformer-Based classification model are trained for the multi-class classification task. The TabTransformer is based on self-attention and can transform the categorical feature' embeddings into robust contextual embeddings, thus, can achieve higher prediction accuracy. Finally, we ensemble the results to improve the performance further. To demonstrate the effectiveness of our approach, extensive experiments have been conducted on the NAD datasets. Our approach achieves the score of 0.625 and ranked as 2nd place in the final leaderboard.

Internet of Things (IoT) is a technology that allows various physical devices to connect to each other and exchange data via the internet network. The application of IoT is increasingly widespread in various sectors such as smart homes, manufacturing industries, smart agriculture, and healthcare. However, along with the increasing number of devices and the volume of data traffic sent, the potential risk of cybersecurity threats is also increasing. The large number of IoT devices that have limited computing capabilities makes the system more vulnerable to various attacks, including intrusion, exploitation of system weaknesses, and Distributed Denial of Service (DDoS) attacks. Therefore, early detection of anomalies in network traffic is a crucial aspect to maintain the security and stability of IoT systems. This study aims to develop and implement a Support Vector Machine (SVM)-based architecture as a classification method in an anomaly detection system on an IoT network. SVM was chosen because of its ability to handle high-dimensional data and non-linear classification effectively. The methodology used includes the process of extracting features from IoT network traffic datasets, data normalization, model training using the SVM algorithm, and evaluating model performance in distinguishing between normal and anomalous traffic. Thus, the implementation of SVM architecture can be an effective and efficient solution in intrusion detection systems for IoT networks. This research also opens up opportunities for the development of more adaptive security systems by integrating machine learning-based detection models into large-scale IoT infrastructures.

This paper explores the potential of machine-mastering models to automate network Anomaly Detection (NAD) through Time series analysis. We employ a two-level method wherein the primary degree entails function selection thru foremost component analysis (PCA), accompanied by gadget mastering (ML) model choice from more than a few supervised studying algorithms. The second stage evaluates the overall performance of the numerous selected ML models and optimizes theirhyperparameters when necessary. Our experiments demonstrate that ML-driven computerized network Anomaly Detection can provide accurate and well-timed detection of network anomalies with little supervision and parameter tuning attempts. The outcomes of our experiments display that Random Forests and Support Vector Machines (SVMs) carry out first-rate some of the model’s grid searches, demonstrating aggressive accuracy and precision ratings from an anomaly detection perspective. We also intensely evaluate the consequences and provide insightful discussion on the possibilities and challenges surrounding using ML for automatic community Anomaly Detection.

Anomaly Detection in Dynamic Networks plays a critical role in various real-world applications such as cyberse-curity, e-commerce, and social media. Recent approaches based on graph neural networks have achieved fruitful results in static networks, and most of the researches focus on learning node embeddings to represent the large-scale complex networks, thereby facilitating downstream anomaly detection task. However, these methods require the whole network to extract intrinsic properties, so they are sensitive to the frequent changes of nodes and incapable of capturing the dynamism as realworld networks evolve over time. In this paper, we propose a novel framework DynAD for anomalous edge detection on time-evolving networks, which performs adaptive parameter learning in an end-to-end manner. In particular, DynAD first extracts fixed-size node embedding from each snapshot with temporal graph convolution and pooling operations. Then, it captures the temporal information of the graph sequence using Gated recurrent units (GRU) for anomaly detection, where an attention mechanism is employed to highlight the differences between the dynamic patterns. Experimental results on three real-world datasets illustrate that DynAD significantly outperforms the state-of-the-art baseline methods in anomaly detection.

Revealing the community structure in social networks witnessed a determined effort. In this respect, a different category of social network can be handled, such as, dynamic social networks, social networks with node attributes, etc. In this article, we introduce a new method to solve this thriving issue in the social network with node attributes. This latter can be represented by a bipartite graph, which consists of a two sets of nodes and edges connecting these nodes. The tendency of people with similar node attributes leads to the hidden information of clusters or communities. A wealthy number of community-detection algorithms have been proposed for bipartite graphs and applied to several domains in the literature. To palliate some of the highlighted shortcomings, we introduce a new approach, called Fast-Bi Community Detection (FBCD), that aims to an efficient community detection in social networks. The main idea of this approach is to explore the set of maximum matching in the bipartite graph in order to reduce the complexity of our algorithm. The carried out experiments show the high quality of the obtained communities versus those by the pioneering ones of the literature.

Signal processing techniques are used to analyze and detect network anomalies because of their ability to detect new and unknown intrusions. The paper proposes a method of modeling network signals for the detection of network anomalies, which combines wavelet approximation and the theory of system identification. To characterize the behavior of network traffic, fifteen functions are provided, which are used as input signals within the system. At the same time, it is assumed that security violations within the network can be detected by checking abnormal patterns of system functioning according to audit data.
 Despite the fact that machine learning methods have achieved significant results in detecting network anomalies, they still face the difficulty of using the implemented algorithms, in the presence of differences in the behavior of the training data and test data, which in turn leads to inefficient performance of the algorithms. This effect is exacerbated by the limitation of algorithms to detect previously unknown types of attacks due to the large number of false positives.
 The paper develops a new method of modeling network signals for detecting anomalies in networks using wavelet analysis. In particular, the general architecture of the approach consists of three components: feature analysis, modeling of normal network traffic based on wavelet approximation and prediction using ARX model, intrusion or non-intrusion decision making
 The result is evaluated using the DARPA intrusion detection dataset, which performs a comprehensive analysis of the intrusions in the dataset. Evaluation results show that this approach provides a high level of detection of both instances and types of attacks.

The article deals with detection of network anomalies. Network anomalies include everything that is quite different from the normal operation. For detection of anomalies were used machine learning systems. Machine learning can be considered as a support or a limited type of artificial intelligence. A machine learning system usually starts with some knowledge and a corresponding knowledge organization so that it can interpret, analyse, and test the knowledge acquired. There are several machine learning techniques available. We tested Decision tree learning and Bayesian networks. The open source data-mining framework WEKA was the tool we used for testing the classify, cluster, association algorithms and for visualization of our results. The WEKA is a collection of machine learning algorithms for data mining tasks.