Anomaly Identification Based on Flow Analysis

Traffic matrices are essential for many traffic engineering tasks such as network management and capacity planning. However, it is difficult and cost to measure them accurately. The estimation of traffic matrix has usually been treated as a pure statistical inference problem. For example, infer the traffic matrix from the link load measurements and routing information. In practice, however, Internet Service Providers (ISPs) could measure partial traffic flows directly nowadays. In this paper, we propose a method to combine partial direct measurements of traffic matrix with one of the noticing method, the Minimum Mutual Information (MMI) method, to estimate the traffic matrix. Evaluation on real network data have demonstrated that with few levels of direct measurements of traffic flows could largely improve the performance of estimation, also the experiments have shown that choosing some large traffic flows to measure directly could have a great improvement on performance.

Many research works address detection and identification of network anomalies using traffic analysis. This paper considers large topologies, such as those of an ISP, with traffic analysis performed on multiple links simultan-eously. This is made possible by using a combination of simple online traffic parameters and specific data from headers of selective packets. Even though large networks may have many network links and a lot of traffic, the analysis is simplified with the usage of Principal Component Analysis (PCA) subspace method. The proposed method proves that aggregation of such traffic profiles on large topologies allows identification of a certain set of anomalies with high level of certainty.

Power consumption control paradigm constitutes an incentive solution for the serious energy issues faced by operators. In this work, we investigate two mobile users assignment strategies for energy management in heterogeneous LTE networks. Both strategies are based on greedy algorithms which are of low complexity and allow on-line approaches suitable for femtocells management issue. Moreover, as the femtocells can be switched off and the macrocells have to remain always on, both strategies aim at selecting the femtocells to deactivate. That is, the mobile user may be moved from a femtocell to a macrocell or the other way around. The move is legitimate if the power consumption is lower and only if the QoS of the user is not sacrificed. Using simulation and real network data, we assess the performances of proposed strategies, in terms of energy consumption and efficiency of the network, the percentage of served mobile users, the real allocated capacity and the number of deactivated femtocells.

Accurate on-road vehicle speed prediction is important for many intelligent vehicular and transportation applications. It is also challenging because the individual vehicle speed is affected by many factors, e.g., traffic speed, vehicle type, and driver's behavior, in either deterministic or stochastic ways. This paper proposes a novel vehicle speed prediction method in the context of vehicular networks, where the real-time traffic information is accessible. Traffic speeds of following road segments are first predicted by Neural Networks (NNs) based on historical traffic data. Hidden Markov models (HMMs) are trained by the Baum-Welch algorithm with historical traffic and vehicle data to present the statistical relationship between vehicle speed and traffic speed. The forward-backward algorithm is applied on HMMs to extract vehicle's speed on each road segment along the driving route. Simulation is set up on the SUMO microscopic traffic simulator with the application of a real Luxembourg highway network and traffic count data. The vehicle speed prediction result shows that our proposed method outperforms other ones in terms of prediction accuracy.

Problem statement: The statistical modeling for predicting network traffic has now become a major tool used for network and is of significant interest in many domains: Adaptive application, congestion and admission control, wireless, network management and network anomalies. To comprehend the properties of IP-network traffic and system conditions, many kinds of reports based on measured network traffic data have been reported by several researchers. The goal of the present contribution was to complement these previous researches by predicting network traffic data. Approach: The Adaptive Neuro-Fuzzy Inference System (ANFIS) was realized by an appropriate combination of fuzzy systems and neural networks. It was applied in different applications which have been increased in recent years and have multidisciplinary in several domains with a high accuracy. For this reason, we used a set of input and output data of packet transmission over Internet Protocol (IP) networks as input and output of ANFIS to develop a model for predicting data. Results: ANFIS was compared with some existing model based on Volterra system with Laguerre functions. The obtained results demonstrate that the sequences of generated values have the same statistical characteristics as those really observed. Furthermore, the relative error using ANFIS model was better than this obtained by Volterra system model. Conclusion: The developed model fits well real data and can be used for predicting purpose with a high accuracy.

The complexity of the Internet and the volume of network traffic have dramatically increased in the last few years, making it more challenging to design scalable Network Traffic Monitoring and Analysis (NTMA) systems. Critical NTMA applications such as the detection of network attacks and anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. The high-dimensionality of network data provided by current network monitoring systems opens the door to the massive application of machine learning approaches to improve the detection and classification of network attacks and anomalies, but this higher dimensionality comes with an extra data processing overhead. In this paper we present Big-DAMA, a big data analytics framework (BDAF) for NTMA applications. Big-DAMA is a flexible BDAF, capable to analyze and store big amounts of both structured and unstructured heterogeneous data sources, with both stream and batch processing capabilities. Big-DAMA uses off-the-shelf big data storage and processing engines to offer both stream data processing and batch processing capabilities, decomposing separate engines for stream, batch and query, following a Data Stream Warehouse (DSW) paradigm. Big-DAMA implements several algorithms for anomaly detection and network security using supervised and unsupervised machine learning (ML) models, using off-the-shelf ML libraries. We apply Big-DAMA to the detection of different types of network attacks and anomalies, benchmarking multiple supervised ML models. Evaluations are conducted on top of real network measurements collected at the WIDE backbone network, using the well-known MAWILab dataset for attacks labeling. Big-DAMA can speed up computations by a factor of 10 when compared to a standard Apache Spark cluster, and can be easily deployed in cloud environments, using hardware virtualization technology.

The paper proposes an approach to security information processing in order to detect computer attacks and network anomalies based on big data technologies. The main contribution of the work is in the development, implementation and investigation of the proposed combined framework for processing security data using parallel computing environment and measuring the performance of the implemented system for detection of network attacks and anomalies. The research goal is to increase the performance of attack detection (under the given requirements for accuracy of solutions) compared to the traditional IDS application. The implemented approach is built using the open source systems Snort and Spark. The paper discusses the capabilities and performance assessment of parallel data processing in order to detect computer attacks and network anomalies, as well as key principles of working with big data. The presented main results of an experimental performance evaluation of the applied approach confirm its high efficiency for analyzing network traffic and security events.

This article discusses an important problem in network management and security—accurate and efficient classification of internet traffic. The need for real-time processing, as well as the complexity and everchanging nature of the network traffic, calls for both high accuracy and computational efficiency. To meet these demands, this study proposes a new approach that incorporates the Online Sequential Extreme Learning Machine (OSELM) and Wavelet Transform, which has been optimized by the Artificial Bee Cloning Algorithm. The proposed OSELM-Wavelet method uses Daubechies 4 wavelet transform to capture relevant frequency-domain features and, at the same time, preserve raw time-domain signals. This method improves the feature set by capturing time and frequency domain features simultaneously, boosting the input data by 800 features. Classification is carried out via the OSELM framework which supports efficient online training and inference and is thus suitable for real-time applications. The Artificial Bee Clony Algorithm is used to optimize the OSELM model parameters and improve the classification accuracy. This algorithm is inspired by the intelligent foraging behavior of bees and can effectively explore and exploit the parameter space for optimal solutions. The experiments bee algorithm-optimized OSELM-Wavelet conducted for internet traffic classification tasks demonstrated high accuracy and robustness.It outperforms conventional statistical and machine learning methods, particularly in situations demanding rapid adaptation and online learning. To summarize, the article combines OSELM-Wavelet with Artificial Bee Cloning Algorithm for Internet traffic classification, presenting a new solution. It aids in precisely and swiftly classifying traffic with minimal computation, thereby improving network security and management.

Traffic flow analysis was considered as one of the main steps in transportation planning and engineering. Various methods have been developed for this purpose. Most of these methods are based on available equations in traffic engineering. Mathematical methods used for analyzing traffic flow has always been very complicated and resource consuming. In 1995, using finite elements method for analyzing traffic flow was first introduced and developed over the few years. Hence it is a new look at traffic flow analysis which still needs lots of complex equations to be solved. In 2007, for the first time, it was suggested to simulate traffic flow as a fluid; using finite elements equations in fluid mechanics (i.e. model the vehicles in the road as a fluid in a canal). Results were considerably matching the real world case studies data. In this paper which is a part of study on the finite elements methods in transportation modeling and planning, ten different routes (e.g. bridge, intersection, roundabout) in Shiraz city (one of large scale cities of Iran) have been modeled in one of fluid mechanics modeling platform. Outputs compared with real world data by some adjustment and calibration which have been made to make it possible to match data and calculate the errors. Results show acceptable errors regarding modeling traffic flow by fluid mechanics simulation platforms. Additionally, a sensitivity analysis has been performed for acquisition of each element’s role (e.g. number of lanes, maximum allowed speed and AADT) in model accuracy in Neural Network Algorithm platform.

in the past decade DDoS attacks detection solutions have been one hot area in the research of cyberspace. Different techniques including machine learning and recent complex deep learning models were used and improved. However, only a few have used real network data and multiclass classification. In this paper, we challenged an Optimized K-Nearest Neighbor (OKNN) on a recent public dataset of the real network containing labeled classes of normal network flow and DDoS attacks. While performing minimum preprocessing to keep data original, OKNN with a tune of hyper-parameters such as; n_neighbors, metric, weights, n_jobs, has identified a normal traffic flow and DDoS attacks with high accuracy. The results of the experiment show that our model would perform better than its counterparts if only they are trained in the same conditions.

People flow dynamics from the basis of various applications such as traffic flow analysis, surveillance, business building security, and crowd motion prediction. With the development of sensing technology, diverse sensors have accumulated sufficient data for a proper understanding of pedestrian movement. As the amount of data increases, the automatic acquisition of crowd behavior and walking information has become more imperative. However, despite large developments in sensing technology, detecting and tracking pedestrians over a relatively large area is costly and high-maintenance. In contrast to traditional individual-based analysis, we consider the movement of pedestrians as a whole entity by incorporating dynamic continuum flow theory and demonstrating how it is applied in our kernel-function-based model of people flow density. In order to reconstruct people flow in areas that are partially invisible to sensors, we assess data assimilation methods to predict the whole areas people flow. The experiments which involve 1D/2D simulation and real tracking data demonstrate the validity of our proposed method. Experimental results of real tracking data show that the estimated density in the invisible area is acceptably close to the true value.

A variety of Web-based applications, mobile apps, and other over the top data services with affordable 3G/4G enabled smart devices are major factors for enormous increase in heterogeneous data traffic at enterprise and mobile networks. This creates challenges regarding traffic management and requires traffic-aware intelligent network management to deliver sustained quality of experience for subscribers. Deep packet inspection and analysis (DPIA) provides base platform for development of traffic-aware intelligent network management and security systems. However, computationally complex DPIA-related packet processing for high speed data traffic makes these systems expensive. Furthermore, conventionally these traffic-aware network management and security systems are deployed in enterprise networks with independent and dedicated DPIA-related processing resources and require multiple copies of passively provisioned high speed data from network, while performing similar DPIA operations over the same data again and again. This duplicate deployment of expensive software and hardware resources for DPIA processing eventually results in higher capital expenditures as well as operational expenditures for network operators. We have proposed a novel service-oriented framework for heterogeneous deep packet inspection and analysis (SoDPI) that simultaneously provides diversified DPIA services to multiple client applications for network management and security operations in high-speed networks. Proposed framework provides flexible and comprehensive API-based service interface for client applications to register required DPIA services. SoDPI framework implementation is based on commodity hardware and deploys shared set of DPIA-related packet processing components, requiring only single copy of passive data provisioned from network. Experimental evaluations show that novel SoDPI framework requires considerably reduced amount of software and hardware resources to fulfill heterogeneous DPIA packet processing requirements for multiple client applications in comparison with conventional network management and security applications with dedicated DPIA components. This results in lower cost impacts for network operators with more network manageability.

With the emergence of connected and automated vehicle (CAV) technologies, research on traffic flow modeling and analysis will play a very important role in improving our understanding of the fundamental characteristics of traffic flow. The frontier of studies on CAV systems have examined the impacts of CAVs on freeway bottleneck capacity, and macroscopic traffic flow, CAV applications on optimization of individual vehicle trajectories, potentials of CAV in traffic signal control, and applications of CAV in network routing. For current and future research initiatives, the greatest challenge lies in the potential inconsistencies between user, operator, and manufacturer goals. Specific research needs were identified on data collection and analysis on CAV behavior and applications. This paper summarizes the presentations and discussions during the Automated Vehicles Symposium 2015 (AVS15) held in Ypsilanti, Michigan, on July 20–23, 2015.

Critical industrial infrastructures are currently facing increasing cyberspace threats in their underlying information and communication systems. The advanced monitoring, control, and management functionalities of the industrial systems firmly rely on the reliable and secure operations of the industrial control system (ICS) network. This article characterizes the ICS network traffic and presents a scalable and efficient solution for real-time ICS network traffic anomaly detection, considering various forms of ICS anomaly events. The events due to the cyberattacks, malicious operating behaviors, and network anomalies can be effectively detected without sophisticated computational requirements and retrieval of communication protocols. The proposed hybrid statistical-machine learning model integrates a seasonal autoregressive integration moving average (SARIMA)-based dynamic threshold model and a long short-term memory (LSTM) model to jointly identify the abnormal traffic patterns with low false omission rates. The proposed solution is extensively evaluated at a realistic ICS cyber–physical system (CPS) testbed, and the numerical results confirm its high detection accuracy and low computational complexity. Note to Practitioners—This article was motivated by the challenge of real-time anomaly detection in industrial cyber–physical systems (CPSs). The existing industrial control system (ICS) network anomaly detection solutions are generally carried out based on a single model based on the historian database and cannot dynamically classify the abnormal conditions in a real-time fashion. A novel hybrid statistical-machine learning model is developed that integrates a seasonal autoregressive integration moving average (SARIMA)-based dynamic threshold model and a long short-term memory (LSTM) model to jointly identify the anomalous events through traffic pattern analysis. The proposed anomaly detection solution can efficiently provide accurate detection for cyberattacks, malicious operating behaviors, and network anomalies while meeting the real-time requirements of ICS networks. The proposed solution can be deployed in the realistic ICS CPSs, e.g., the power generation system, gas pipeline systems, and urban railway transportation systems. The preliminary numerical results obtained from the ICS-CPS testbed suggested that it can provide high detection accuracy with low computational complexity and, hence, can be adopted with minimal deployment hurdles.

To accommodate the increasing data rate demand, the fifth-generation (5G) cellular network came up with new technological advancements including massive multiple-input multiple-output (massive MIMO) and hyperdensification which can significantly boost network capacity. On the other hand, the introduction of these technologies along with their heterogeneity brings a challenge in terms of network operators’ need to identify a cost-effective optimal deployment approach which is hardly entertained by the legacy planning and optimization method. Hence, to leverage the core benefits of those technologies in a cost-effective manner, we need a holistic planning framework that takes into account their coverage, capacity and cost impact, and realistic spatiotemporal distribution of users. In this work, we present a data-driven multiobjective optimization planning framework that can be used not only for small cells but also for massive MIMO. The planning framework is illustrated using a 5G planning case study for a service area in Addis Ababa, Ethiopia, considering its realistic network data that is collected from the network management system and different potential identified deployment options. Ray tracing is employed to compute propagation, and users and demands are distributed based on the realistic network data. A two-stage optimization and a joint optimization are applied to identify points that provide optimal network performance. Simulation results reveal that the planning method provides Pareto points for different deployment options that can significantly improve the performance of the existing network while reducing the total network cost.