A Framework For Concept Drifting P2P Traffic Identification

Unlabeled training examples are readily available in many applications, but labeled examples are fairly expensive to obtain. For instance, in our previous works on classification of peer-to-peer (P2P) Internet traffics, we observed that only about 25% of examples can be labeled as “P2P”or “NonP2P” using a port-based heuristic rule. We also expect that even fewer examples can be labeled in the future as more and more P2P applications use dynamic ports. This fact motivates us to investigate the techniques which enhance the accuracy of P2P traffic classification by exploiting the unlabeled examples. In addition, the Internet data flows dynamically in large volumes (streaming data). In P2P applications, new communities of peers often join and old communities of peers often leave, requiring the classifiers to be capable of updating the model incrementally, and dealing with concept drift. Based on these requirements, this paper proposes an incremental Tri-Training (iTT) algorithm. We tested our approach on a real data stream with 7.2 Mega labeled examples and 20.4 Mega unlabeled examples. The results show that iTT algorithm can enhance accuracy of P2P traffic classification by exploiting unlabeled examples. In addition, it can effectively deal with dynamic nature of streaming data to detect the changes in communities of peers. We extracted attributes only from the IP layer, eliminating the privacy concern associated with the techniques that use deep packet inspection.

Realistic network environments are commonly emulated by Internet application developers to test both the performance and the correctness of their applications. In this paper, we argue that emulating just realistic network environments may be inadequate for detecting hard-to-detect faults that have very low occurrence probabilities but potentially high impact if exposed in the real world. Specifically, we conduct extensive experiments to study the correctness testing capabilities of realistic and unlikely network environments. Our results show that small investments in emulating unlikely network environments may help in quickly detecting otherwise hard-to-detect faults.

Rucas over privacy of domain name registration

Accurate network traffic classification is vital to the areas of network security. There are many applications using dynamic ports and encryption to avoid detection, so previous methods such as port number and payload-based classification exist some shortfalls. An alternative approach is to use Machine learning (ML) techniques. Here we will present three clustering algorithms, K-Means, FarthestFirst and Canopy, based on flow statistic features of applications. The performance impact of the data set processed by the PCA dimension reduction algorithm on the above three algorithms will be an important topic for our discussion. Our results show that the classification accuracy and computational performance all have been significantly improved after dimension reduction.

With the rapid development of Internet, many network applications (e.g., P2P) use dynamic ports and encryption technology, which makes the traditional port and payload-based classification methods ineffective. Hence, it is important and necessary to find the more effective ones. Currently the machine learning (ML) techniques provide a promising alternative one for IP traffic classification. In this work, we use the ML-based classification method to identify the classes of the unknown flows using the payload-independent statistical features such as packet-length and arrival-interval. In order to improve the efficiency of the classification methods, the feature reduction techniques are further adopted to refine the selected features for attaining a best group of features. Finally we compare and evaluate the ML classification algorithms based on the BRASIL data source in terms of the three metrics such as overall accuracy, average precision and average recall. Our experiments show that the decision-tree algorithm is the best ML one for IP traffic classification and is able to construct the real-time classification system.

Objectives: To present a model which can detect malicious attacks using the URL of the Social Communication Platform using the data imbalance machine learning algorithm. The main objective is to detect the attack and prevent it from happening. Methods: This study presents an efficient feature extraction and selection method addressing feature imbalance problems; and also presents an improved concept drift and machine learning-based classification. This paper extracts the URL of the undesired tweets, identifies them, and filters them for classification. Findings: The experiments have been conducted using the drifted twitter spam dataset. Our model DIA-XGBoost extracts the URL of the undesired tweets, identifies them, and filters them for classification. Further, the attack pattern varies with respect to time. Furthermore, the results show that our DIA-XGBoost attains higher accuracy performance by 1.254%, URL recall performance by 0.14%, and increased Fmeasure performance by 10% when compared with the existing ML techniques (Random Forest, K-Nearest Neighbour, XGBoost). Thus, the existing ML-based classification model achieves poor classification accuracy whereas our model solves this issue. Novelty: Various Machine Learning (ML) techniques have been applied for the classification of URL redirection attacks. However, the spam data generally exhibit feature imbalance. Further, the attack pattern varies with respect to time. Thus, the existing ML-based classification model achieves poor classification accuracy. Hence, our model solves the issue using the DIA-XGBoost algorithm, detects and prevents URL malicious attacks. Keywords: Data Imbalance; Feature Extraction; Concept Drift; URL; Machine Learning; URL Redirection Attack

In evolving data stream, when its concept undergoes a change it is known as concept drift. Detecting concept drift and handling it is a challenging task in data stream mining. If an algorithm is not adapted to concept drift, then it directly affects its performance. A number of algorithms have been developed to handle concept drift, but they are not suited for both sudden concept drift and gradual concept drift. Thus, there is a demand for an algorithm that can react to both the types of concept drift as well as incur less computational cost. A new approach hybrid early drift detection method (HEDDM) has been proposed for drift detection, which works with an ensemble method to improve the performance.

Machine Learning (ML) has been widely applied to cybersecurity and is considered state-of-the-art for solving many of the open issues in that field. However, it is very difficult to evaluate how good the produced solutions are, since the challenges faced in security may not appear in other areas. One of these challenges is the concept drift, which increases the existing arms race between attackers and defenders: malicious actors can always create novel threats to overcome the defense solutions, which may not consider them in some approaches. Due to this, it is essential to know how to properly build and evaluate an ML-based security solution. In this article, we identify, detail, and discuss the main challenges in the correct application of ML techniques to cybersecurity data. We evaluate how concept drift, evolution, delayed labels, and adversarial ML impact the existing solutions. Moreover, we address how issues related to data collection affect the quality of the results presented in the security literature, showing that new strategies are needed to improve current solutions. Finally, we present how existing solutions may fail under certain circumstances and propose mitigations to them, presenting a novel checklist to help the development of future ML solutions for cybersecurity.

Radio wave data gathered by pulsar finding telescopes are required to be classified while being streamed. The reason for that is the practical constraints of traditional machine learning algorithms on streaming datasets. Traditional machine learning algorithms would take considerable compute power, memory and time to give pragmatic results.(recent surveys collect data at the rate of 0.5 – 1 terabyte per second) Stream classification algorithms are specifically developed to address the above limitations and can classify data streams without taking up a lot of memory or training time. They relate with characteristics of data streams such as concept drift and limited memory. Extremely Fast Decision Tree is one of the stream classification algorithms that can learn incrementally when it sees new data. However, data from pulsar detecting datastreams are highly imbalanced (there are less examples of pulsars in the data than non-pulsar objects). Learning incrementally from such a datastream would be a destructive interference for the model’s precision (of detecting pulsars). In this research, we introduce an improved version of the Extremely Fast Decision Tree, that is able to learn imbalanced data streams. Our approach is fast, accurate, and avoids the pitfalls of class imbalance and concept drift.

Concept drift referring to the changes of data distributions has been one critical challenge typically associated with mining data streams. Current drift detection and adaptation methods focus on how to immediately detect the distribution changes once the concept drift occurs and swiftly update the model to be applicable to the newly arrived data instances. Most of those methods assume the data does not have noise or the noise is too weak to affect the modeling procedure. However, realworld data are normally contaminated, and denoise techniques are highly preferred as a necessary preprocess. This issue is more complex for a data stream with concept drift because the noise is very likely to be confused with drift. Motivated by that, this paper proposes a Noise-tolerant Fuzzy c-means based drift Adaptation method (NFA) which can adapt to the changing distributions and is suitable for noisy data streams. The concept drift problem is solved by using a fuzzy c-means based regression model to continuously include the most relevant data instances to the latest pattern in the training set. In addition, a denoise technique is designed in NFA to remove noise, and the ability of incremental updating enables it to be embedded in the incremental drift adaptation process, and therefore NFA can solve concept drift and noise problems at the same time. Experimental evaluation results also show good performance of our method on handling data streams with concept drift and noise.

In data stream mining, one of the biggest challenges is to develop algorithms that deal with the changing data. As data evolve over time, static models become outdated. This phenomenon is called concept drift, and it is investigated extensively in the literature. Detecting and subsequently adapting to concept drifts yield more robust and better performing models. In this study, we present an unsupervised method called D3 which uses a discriminative classifier with a sliding window to detect concept drift by monitoring changes in the feature space. It is a simple method that can be used along with any existing classifier that does not intrinsically have a drift adaptation mechanism. We experiment on the most prevalent concept drift detectors using 8 datasets. The results demonstrate that D3 outperforms the baselines, yielding models with higher performances on both real-world and synthetic datasets.

A frequent problem in data stream mining is concept drift, meaning the data distribution changes over time. A common issue when dealing with concept drift is insufficient data. Real-world applications of data stream mining often involve multiple data streams. However, most concept drift methods handle these data streams separately. This study uses data from other data streams to handle the problem of insufficient data. We propose a novel Multistream Concept Drift Handling Framework via data sharing, containing a fuzzy membership-based drift detection (FMDD) component and a fuzzy membership-based drift adaptation (FMDA) component, to train the new learning model for drifting streams by sharing weighted data from other nondrifting streams. A stream fuzzy set is defined with membership functions that measure the degree to which samples belong to a data stream. Our Concept Drift Handling Framework can detect when and in which stream concept drift occurs, and therefore the insufficient data issue can be solved by adding the weighted data from nondrifting streams to train new learning models. Synthetic and real-world experimental results show that our method can help avoid the insufficient data issue and thereby significantly improve the prediction performance.

Self-similarity is one of the most important characteristics of real network traffic. Constructing an accurate model can reflect the real network environment precisely. It is also an effective measure to predict the upcoming network traffic, thus ensuring the quality and reliability of network service. In this paper, we present a cluster-based distributed system, which is based on FGN(Fraction Gaussian Noise) algorithm. With the purpose to generate large-scale traffic and simulate the real network environment. We offer a detailed description for this system in the following three aspects: algorithm, architecture and implementation. Experiments have proved that the cluster-based system can generate large-scale similar network traffic with designated Hurst parameter

Recently different capacity boosting technologies, <span>including 3D beamforming and expanding operating bandwidth have been investigated and included in the enhanced fourth generation (4G) and fifth-generation standards. For mobile operators to enhance their network performance, applying these advanced technologies is vital. To achieve that, planning and optimization work need to consider the spatiotemporal distribution of users. Although the capacity impact of advanced antenna technologies is investigated well, deployment options to exploit their benefits in a cost-effective manner considering a realistic network environment are seldom reported. This work presents a data driven and multi-objective based optimal vertical beams placement for an enhanced 4G mobile network with the newly introduced C-band. The method is utilized for an area located in Addis Ababa, Ethiopia, taking into account the existing 4G mobile network as a reference configuration. Users are distributed based on the data collected by ethio-telecom network management system. Findings indicate that optimal vertical beams placement achieved for gradual deployment with consideration cell edge and aggregate throughput performances while reducing network cost. While being cost-effective, up to 69.2% and 73.8% cell edge performance gain is achieved at 20 and 30 pareto points relative to an existing macro mobile network.</span>

6G networks are envisioned to be trustworthy systems that can self-maintain their reliability and service availability. The ability to automatically recognize and predict any faults or failures that occur while delivering services is a step towards realizing such systems. Real-time analytics based on Artificial Intelligence (AI)/Machine Learning (ML) techniques provides the required functionality. However, in the highly dynamic and complex environment of 6G services, the distribution of data feeding to the ML models is subject to change over time (i.e., concept drift), which makes persistent model accuracy challenging without any frequent model retraining. To address this issue, in this paper, we propose a system that predicts the faults in an edge cloud environment, where the prediction models are trained and automatically adapted to the concept drifts via Transfer Learning (TL). To present the effectiveness of our system, we implemented an edge cloud testbed and introduced CPU over-utilization and network congestion fault to it in the presence of concept drift. We used Convolutional Neural Network (CNN), Long-Short Term Memory (LSTM), and a combination of them as fault prediction models. For detecting the drifts, we have implemented and compared four drift detection methods, and we utilized three TL scenarios for adapting the model to the drift. Our results indicate the superiority of our system in maintaining a persistent accuracy in the presence of concept drift compared to a prediction system without a drift handling entity.