Network traffic data analysis

With the rapid increasing of network scale, the size of traffic data also expands a lot. In traditional traffic data analysis, there are some problems, such as high computation complexity, low analysis efficiency, long learning period, and difficulty of development. To address these problems, we design and implement a GPU-accelerated parallel analysis scheme for network traffic - EasyAnalyze. In EasyAnalyze, we introduce GPU parallel computing, Map/Reduce architecture into network traffic analysis, which greatly improves the efficiency but does not increase the difficulty in programming. In the experiments, EasyAnalyze shows very promising results: 1 the speed is 6-17 times faster than conventional serial analysis in network traffic data analysis; and 2 the size of code is only 2% of the mainstream GPU Map/Reduce.

Optical Packet Switching (OPS) has emerged as a promising candidate for the future all-optical core network. A crucial issue in such networks is how the packet length distribution (PLD) influences the performance under different traffic scenarios. In this paper we investigate how a deterministic, empirical and exponential PLD influences the Packet Loss Rate (PLR) in an optical packet switch in the case of a Poisson arrival process and a bursty Hyperexponential arrival process. We consider optical packet switches with and without Fibre Delay Line (FDL) contention buffers. Our results show that in the case of OPS without FDL buffering and with a Hyperexponential arrival process, the empirical PLD yields the lowest PLR. However, in the case of OPS with FDL buffering, the deterministic PLD yields the lowest PLR.

Using relational graphs for exploratory analysis of network traffic data

With the growth of the internet of things (IoT) smart objects, managing these objects becomes a very important challenge, to know the total number of interconnected objects on a heterogeneous network, and if they are functioning correctly; the use of IoT objects can have advantages in terms of comfort, efficiency, and cost. In this context, the identification of IoT objects is the first step to help owners manage them and ensure the security of their IoT environments such as smart homes, smart buildings, or smart cities. In this paper, to meet the need for IoT object identification, we have deployed an intelligent environment to collect all network traffic traces based on a diverse list of IoT in real-time conditions. In the exploratory phase of this traffic, we have developed learning models capable of identifying and classifying connected IoT objects in our environment. We have applied the six supervised machine learning algorithms: support vector machine, decision tree (DT), random forest (RF), k-nearest neighbors, naive Bayes, and stochastic gradient descent classifier. Finally, the experimental results indicate that the DT and RF models proved to be the most effective and demonstrate an accuracy of 97.72% on the analysis of network traffic data and more particularly information contained in network protocols. Most IoT objects are identified and classified with an accuracy of 99.21%.

The main data-driven techniques for detecting cybersecurity attacks are based on the analysis of network traffic data and/or of application/system logs (stored in a host or in some other kind of device). A wide range of machine-learning techniques (and possible alternative configurations of them) have been proposed in the literature so far, for this purpose, but none of them has been proven to consistently overcome the others across different datasets. In order to ensure better accuracy and stability, the ensemble paradigm can be exploited as an effective solution for combining such techniques. However, as attack detection problems are hard to cope with and, usually, entail the analysis of large and fast streams of data, different types of ensemble (and of base algorithms composing the ensemble) should have experimented, exploiting distributed architecture to suitably reduce the high-execution times necessary to run them. In order to handle all these issues, a p2p environment to validate ensemble-based approaches in the cybersecurity domain is proposed in this paper. Two case studies are analyzed by using this framework, which concern the detection of intrusions in network-traffic data and of deviant process instances. Preliminary scalability results demonstrate that the framework is a viable solution for these challenging kind of problems.

Intrusion detection systems, also known as IDSs, are widely regarded as one of the most essential components of an organization’s network security. This is because IDSs serve as the organization’s first line of defense against several cyberattacks and are accountable for accurately detecting any possible network intrusions. Several implementations of IDSs accomplish the detection of potential threats throughout flow-based network traffic analysis. Traditional IDSs frequently struggle to provide accurate real-time intrusion detection while keeping up with the changing landscape of threat. Innovative methods used to improve IDSs’ performance in network traffic analysis are urgently needed to overcome these drawbacks. In this study, we introduced a model called a deep neural decision forest (DNDF), which allows the enhancement of classification trees with the power of deep networks to learn data representations. We essentially utilized the CICIDS 2017 dataset for network traffic analysis and extended our experiments to evaluate the DNDF model’s performance on two additional datasets: CICIDS 2018 and a custom network traffic dataset. Our findings showed that DNDF, a combination of deep neural networks and decision forests, outperformed reference approaches with a remarkable precision of 99.96% by using the CICIDS 2017 dataset while creating latent representations in deep layers. This success can be attributed to improved feature representation, model optimization, and resilience to noisy and unbalanced input data, emphasizing DNDF’s capabilities in intrusion detection and network security solutions.

Packet sampling has become an attractive and scalable means to measure flow data on high-speed links. Passive traffic measurement increasingly employs sampling at the packet level and makes inferences from sampled network traffic. This paper proposes a maximum probability method that estimates the length of the corresponding original flow from the length of a sampled flow. We construct the probability models of the original flow length distributions of a sampled flow under the assumptions of various flow length distributions, respectively. Through recovery analyzing with different parameters, we obtain a consistent linear expression that reflects the relationship between the length of sampled flow and that of the corresponding original flow. Furthermore, after using publicly available traces and traces collected from CERNET to do recovery experiments and comparing the experiment outcomes and theoretic values calculated with Pareto distributions, we may conclude that the maximum probability method calculated by using the Pareto distribution with 1.0 can be used to estimate original flow length in the concerned network.

Big data analysis has become much popular in the present day scenario and the manipulation of big data has gained the keen attention of researchers in the field of data analytics. Analysis of big data is currently considered as an integral part of many computational and statistical departments. As a result, novel approaches in data analysis are evolving on a daily basis. Thousands of transaction requests are handled and processed everyday by different websites associated with e-commerce, e-banking, e-shopping carts etc. The network traffic and weblog analysis comes to play a crucial role in such situations where Hadoop can be suggested as an efficient solution for processing the Netflow data collected from switches as well as website access-logs during fixed intervals.

Recently, data hiding by modifying network parameters like packet header, payload, and packet length has become popular among researchers. Different algorithms have been proposed during the last few years which have altered the network packets in different ways to embed the data bits. Some of these algorithms modify the network packet length for embedding. Although most of the packet length based embedding schemes try to imitate the normal network traffic distribution, they have altered the statistical distribution of network packet lengths during embedding. These statistical anomalies can be exploited to detect such schemes. In this paper, a second order detection scheme for packet length based steganography has been proposed. A comprehensive set of experiments have been carried out to show that the proposed detection scheme can detect network packet length based steganography with a considerably high accuracy.

Packet sampling measurement is a key technique in high-speed network traffic monitoring and management. In this paper, packet sampling techniques are introduced and analyzed after the presentation of related background researches. Then time-trigger and event-trigger Poisson sampling techniques are evaluated separately on several aspects such as packet length distribution, average packet length and the comparison between sampled volume and the whole traffic volume. The experiment results support that event-trigger Poisson sampling technique shows better performance than time-trigger Poisson sampling technique. At last, a heuristic explanation is briefly discussed and the prospect of packet sampling is depicted.

Output process characterization for fast token ring networks supporting two priorities

The identification of network applications through observation of associated packet traffic flows is vital to the areas of network management and surveillance. Currently popular methods such as port number and payload-based identification exhibit a number of shortfalls. An alternative is to use machine learning (ML) techniques and identify network applications based on per-flow statistics, derived from payload-independent features such as packet length and inter-arrival time distributions. The performance impact of feature set reduction, using Consistency-based and Correlation-based feature selection, is demonstrated on Naïve Bayes, C4.5, Bayesian Network and Naïve Bayes Tree algorithms. We then show that it is useful to differentiate algorithms based on computational performance rather than classification accuracy alone, as although classification accuracy between the algorithms is similar, computational performance can differ significantly.

In this paper, we consider the problem of non-parametric estimation of network flow characteristics, namely packet lengths and byte sizes, based on sampled flow data. We propose two different approaches to deal with the problem at hand. The first one is based on single stage Bernoulli sampling of packets and their corresponding byte sizes. Subsequently, the flow length distribution is estimated by an adaptive expectation- maximization (EM) algorithm that in addition provides an estimate for the number of active flows. The estimation of the flow sizes (in bytes) is accomplished through a random effects regression model that utilizes the flow length information previously obtained. A variation of this approach, particularly suited for mixture distributions that appear in real network traces, is also considered. The second approach relies on a two-stage sampling procedure, which in the first stage samples flows amongst the active ones, while in the second stage samples packets from the sampled flows. Subsequently, the flow length distribution is estimated using another EM algorithm and the flow byte sizes based on a regression model. The proposed approaches are illustrated and compared on a number of synthetic and real data sets.

This work focuses on the study of mechanical and structural properties of glass fiber‐reinforced polypropylene (PPGF) composites compatiblized with maleic anhydride (MAH) grafted PP (PP‐g‐MAH). Thus, effects of the compatibilizer on the flow capacity, fiber orientation and length distribution, thermal and mechanical properties were evaluated. Combining fiber length and melt flow index (MFI) results, it can be speculated that when the compatibilizer is added, the level of fiber matrix interaction increases and the fluidity of the molten composite material improves, resulting in a better orientation and a lower level of fiber length degradation. Moreover, the mechanical analysis based on the micromechanical model was conducted to estimate the effect of the compatibilizer on the interfacial shear stress, orientation factor, and the tensile strength. The main results indicate that the experimental data have confirmed the validity of the micromechanical model. POLYM. COMPOS., 39:3497–3508, 2018. © 2017 Society of Plastics Engineers

The article discusses the key factors for the organization of train traffic. The analysis of statistical parameters of the intervals of departure of freight trains from railway stations and the length of the freight train flow is carried out. Based on these train schedules, graphical dependences of the intervals of departure of trains from separate points on the volume of train traffic and the distribution of train lengths in even and odd directions are constructed. Polynomial dependencies with coefficients of variation are derived. The curve of the second-order Erlang distribution function for the departure intervals is constructed. It is determined that the average departure intensity in one hour is equal to three trains, and the average departure interval lies between the 25th and 35th minutes with a probability of 0.793. There is a correlation between the coefficient of variation (mean square deviation) of the departure interval and its average daily deviation. The power function of the analytical ratio of the coefficient of variation from the average interval of departure of freight trains from the technical stations of the Krasnoyarsk railway during the day is determined using the least squares method. The degrees of correspondence of theoretical and statistical distributions of the number of cars in trains are studied using the Kolmogorov criterion with the calculation of the probability value and confirmation that the distribution of the number of cars in trains obeys the binomial law. Using the Pearson agreement criterion, it is proved that the frequency of departure of freight trains from railway stations obeys the exponential distribution law.