Network Traffic Measurements, Applications to Internet Services and Security

The advent of Cyber-Physical Systems (CPS) has brought a revolutionary change coined as a mixture of information, communication, computation, and control. With applications in smart grid, health monitoring, automatic avionics, distributed robotics, etc., CPS is currently an area of attention among the academia and industry. The advancement of mobile communications and embedded technology has made it possible to build large scale CPS consisting of the interconnection of mobile phones. These devices collect information about the surrounding environment at any time anywhere basis through real-time video capture. Video streaming has proven to be a massive industry that is growing rapidly playing an important role in everyday life. Customer-driven approach wanting best experience with quality has to be the core offering of contemporary scenario. Video streaming is categorized into Video-On-Demand Streaming (VoDS) and Live Video Streaming (LVS) showing the current state-of-art opportunities. Many diverse applications of video streaming are military video surveillance using drones, live sports match player face recognition, on-demand video characters recognition, movie summarization like identifying parts of the movie which are viewed many times by different users, movie and series recognition, motion detection, gesture recognition, image segmentation, etc. This paper introduces an approach to develop video analysis on VoD and LVS using cloud-based services and analyzes the impact of Quality of Experience (QoE), cost, and bandwidth on the cloud. To achieve the best user experience for video streaming and video analysis, Content Delivery Network (CDN) offers the best QoE at various analyzed locations using various cloud providers like Amazon Web Services (AWS) CloudFront, Google Cloud CDN, Azure CDN, Akamai CDN, etc.

Internet-scale services like YouTube are provisioned by large Content Delivery Networks (CDNs), which push content as close as possible to the end-users to improve their Quality of Experience (QoE) and to pursue their own optimization goals. Adopting space and time variant traffic delivery policies, CDNs serve users' requests from multiple servers/caches at different physical locations and different times. CDNs traffic distribution policies can have a relevant impact on the traffic routed through the Internet Service Provider (ISP), as well as unexpected negative effects on the end-user QoE. In the event of poor QoE due to faulty CDN server selection, a major problem for the ISP is to avoid being blamed by its customers. In this paper we show a real case study in which Google CDN server selection policies negatively impact the QoE of the customers of a major European ISP watching YouTube. We argue that it is extremely important for the ISP to rapidly and automatically detect such events to increase its visibility on the overall operation of the network, as well as to promptly answer possible customer complaints. We therefore present an Anomaly Detection (AD) system for detecting unexpected cache-selection changes in the traffic delivered by CDNs. The proposed algorithm improves over traditional AD approaches by analyzing the complete probability distribution of the monitored features, as well as by self-adapting its functioning to dynamic environments, providing better detection capabilities.

Over the past decade, video streaming on the Internet has become the primary source of our media consumption. Billions of users stream online video on multiple devices with an increasing expectation that video will be delivered at high quality without any rebuffering or other events that affect their Quality of Experience (QoE). Video streaming platforms leverage Content Delivery Networks (CDNs) to achieve this at scale. However, there is a gap in how the quality of video streams is monitored. Current solutions rely on client-side beacons that are issued actively by video players. While such approaches may be feasible for streaming platforms that deploy their own CDN, they are less applicable for third-party CDNs with multiple tenants and diverse video players.In this paper, we present a characterization of video workload from a global multi-tenant CDN and develop SSQoE: a methodology deployed on the server side which estimates rebuffering experienced by video clients using passive measurements. Using this approach, we calculate a QoE score which represents the health of a video stream across multiple consumers. We present our findings using this QoE score for various scenarios and compare it to traditional server and network monitoring metrics. We also demonstrate the QoE score’s efficacy during large streaming events such as the 2020 Superbowl LIV. We show that this server-side QoE estimation methodology is able to track video performance at an AS or user agent level and can easily pinpoint regional issues at the CDN, making it an attractive solution to be explored by researchers and other CDNs.

Mobile video traffic is dominant in cellular and enterprise wireless networks. With the advent of myriads of applications from video telephony and streaming to virtual reality, network administrators face the challenge to provide high quality of experience (QoE) in the face of diverse wireless conditions and application contents. Yet, state-of-the-art networks lack analytics for QoE, as this requires support from the application or user feedback. While there are existing techniques to map quality of service (QoS) to QoE by training machine learning (ML) models without requiring user feedback, these techniques are limited to only few applications (e.g., Skype), due to insufficient QoE ground-truth annotation for ML. To address these limitations, we focus on video telephony applications and model key artefacts of spatial and temporal video QoE. Our key contribution is designing content- and device-independent metrics and training across diverse WiFi conditions. We show that our metrics achieve a median 90% accuracy by comparing with mean-opinion-score (MOS) from more than 200 users and 800 video samples. Our content-independent metrics significantly reduce the MOS prediction error of previous works and are validated over three popular video telephony applications – Skype, FaceTime and Google Hangouts.

Video traffic has witnessed exponential growth in recent years. As the cost optimization space of Content Delivery Network (CDN) has reached a plateau, content providers are expanding their network infrastructure to accommodate this surge. To address the cost issue, content providers have turned to cost-effective Peer-to-Peer Content Delivery Network (PCDN) solutions. However, the utilization of limited-capability PCDN nodes may lead to a decline in overall streaming performance compared to powerful CDN servers. To bridge this performance gap, we present Twist. Twist is a receiver-driven multi-site transport that leverages multiple cost-effective PCDN nodes to achieve performance equivalent to traditional CDN solutions. It incorporates a joint flow control to prevent throughput drops that can occur when using multiple content sources, and a proactive retransmission algorithm to handle frequent packet loss. Our evaluations demonstrate that Twist improves download speed on PCDN nodes by 1.71x-2.09x compared to traditional PCDN streaming solutions. Furthermore, extensive A/B testing verifies that Twist enables PCDN to achieve comparable download performance and Quality of Experience (QoE) as CDN. Over three years of commercial deployment, Twist has already served over 300 million users and has handled 35% of the video traffic, resulting in substantial cost savings for a world-leading content provider.

We analyze competition between Internet Service Providers (ISPs) where consumers demand heterogeneous content within two Quality-of-Service (QoS) regimes, Net Neutrality and Paid Prioritization, and show that paid prioritization increases the static efficiency compared to a neutral network. We also consider paid prioritization intermediated by Content Delivery Networks (CDNs). While the use of CDNs is welfare neutral, it results in higher consumer prices for internet access. Regarding incentives to invest in network capacity we show that discriminatory regimes lead to higher incentives than the neutral regime as long as capacity is scarce, while investment is highest in the presence of CDNs.

Quality of Experience (QoE) support becomes more demanding due to the increasing popularity of multimedia services. However, QoE-based multimedia provisioning will impose a high pressure on server capacity and network bandwidth, which becomes one of the major concerns for both content providers and network operators. A common approach for efficient and scalable multimedia service provisioning in content distribution networks (CDN) is to replicate certain “hot” service or content from the original server to multiple replica servers, usually closer to end users. However, how to direct distributed user requests to an appropriate replica server to fundamentally improve the user perceived service quality largely remains unknown. In this paper, we present a user-oriented QoE-driven server selection scheme to solve the aforementioned problem. Experiments carried out on a real-world network have been performed to evaluate the proposed scheme, and the results show that the proposed scheme can improve QoE perceived by end users at least 24.6%.

To deal with the massive traffic produced by video applications, mobile operators rely on offloading technologies such as Small Cells, Content Delivery Networks and, shortly, Cloud Edge and 5G Device to Device communications. Although these techniques are fundamental for improving network efficiency, they produce a multitude of paths onto which the user traffic can be forwarded. Thus, a critical problem arises about how to handle the increasing video traffic while managing the interplay between infrastructure optimization and the user's Quality of Experience (QoE). Solving this problem is remarkably difficult, and recent investigations do not consider the large-scale context of mobile operator networks. To address this issue, we present a novel QoE-aware path deployment scheme for large-scale SDN-based mobile networks. The scheme relies on both a polynomial-time algorithm for composing multiple QoS metrics and a scalable QoS to QoE translation strategy. Considering real mobile operator network and video traffic traces, we show that the proposed algorithm outperformed state-of-the-art approaches by reducing impaired videos in aggregate MOS by at least 37% and lowering accumulated video stall length four times.

The development of Quality of Experience (QoE) models using Machine Learning (ML) is challenging, since it can be difficult to share datasets between research entities to protect the intellectual property of the ML model and the confidentiality of user studies in compliance with data protection regulations such as General Data Protection Regulation (GDPR). This makes distributed machine learning techniques that do not necessitate sharing of data or attribute names appealing. One suitable use case in the scope of QoE can be the task of mapping QoE indicators for the perception of quality such as Mean Opinion Scores (MOS), in a distributed manner. In this article, we present Distributed Ensemble Learning (DEL), and Vertical Federated Learning (vFL) to address this context. Both approaches can be applied to datasets that have different feature sets, i.e., split features. The DEL approach is ML model-agnostic and achieves up to 12% accuracy improvement of ensembling various generic and specific models. The vFL approach is based on neural networks and achieves on-par accuracy with a conventional Fully Centralized machine learning model, while exhibiting statistically significant performance that is superior to that of the Isolated local models with an average accuracy improvement of 26%. Moreover, energy-efficient vFL with reduced network footprint and training time is obtained by further tuning the model hyper-parameters.

Due to the widespread use of encryption in Over-The-Top video streaming traffic, network operators generally lack insight into application-level quality indicators (e.g., video quality levels, buffer underruns, stalling duration). They are thus faced with the challenge of finding solutions for monitoring service performance and estimating customer Quality of Experience (QoE) degradations based solely on passive monitoring solutions deployed within their network. We address this challenge by considering the concrete case of YouTube, whereby we present a methodology for the classification of end users' QoE when watching YouTube videos, based only on statistical properties of encrypted network traffic. We have developed a system called YouQ which includes tools for monitoring and analysis of application-level quality indicators and corresponding traffic traces. Collected data is then used for the development of machine learning models for QoE classification based on computed traffic features per video session. To test the YouQ system and methodology, we collected a dataset corresponding to 1060 different YouTube videos streamed across 39 different bandwidth scenarios, and tested various classification models. Classification accuracy was found to be up to 84% when using three QoE classes ("low", "medium" or "high") and up to 91% when using binary classification (classes "low" and "high"). To improve the models in the future, we discuss why and when prediction errors occur. Moreover, we have analysed YouTube's adaptation algorithm, thus providing valuable insight into the logic behind the quality level selection strategy, which may also be of interest in improving future QoE estimation algorithms.

Cloud computing is an emerging technology, which provides services over internet such as software, hardware, network and storage. The key role for cloud computing is virtualization which reduces the total cost and gives reliable, flexible and secured services. However compute service are chosen between the providers located in multiple data centres. One of the major security concerns related to the virtualization and the Storage where the outside attackers can use the files in the storage and the data owners are not capable of knowing attacks. In this paper we proposed a high level authentication for the cloud user and remote monitor controlled of your cloud storage. Here our model provides the dual authentication for the cloud and to get the runtime record of the logs and the secured application controls, the logs are remotely accessed and controlled by the owner of the data.

The widespread use of encryption in the delivery of Over-The-Top video streaming services poses challenges for network operators looking to monitor service performance and detect potential customer perceived Quality of Experience (QoE) degradations. While monitoring solutions deployed on client devices provide insight into application layer KPIs (e.g., video quality levels, buffer underruns, stalling duration) which can be further mapped to user QoE, network providers commonly rely primarily on passive traffic monitoring solutions deployed within their network to obtain insight into user perceived degradations and their potential causes. In this paper we present a methodology for the estimation of end users' QoE when watching YouTube videos which is based only on statistical properties of encrypted network traffic. We have developed a system called YouQ which includes tools for monitoring and analysis of application-layer KPIs and corresponding traffic traces, and the subsequent use of this data for the development of machine learning models for QoE estimation based on traffic features. To test this approach, we have collected a dataset of 1060 different YouTube video traces using 39 different bandwidth scenarios. All video traces are annotated with application-layer KPIs and classified into one of three QoE classes. The dataset was used to test various machine learning algorithms, and results showed that up to 84\% QoE classification accuracy could be achieved using only features extracted from encrypted traffic.

Quality of experience (QoE) serves as a direct evaluation of users' experience in mobile video transmission and is critical to ensure good network service. Although many efforts have been made to predict QoE based on network parameters of the user terminal equipment, it is difficult to predict QoE based on Quality of Service (QoS) offered by the network servers. In this paper, a machine learning based QoE evaluation method is proposed to evaluate user QoE in real-time by analyzing the QoS characteristics for mobile video transmission. For this purpose, we construct a large-scale dataset by collecting more than 300 thousand pieces of metrics data with two kinds of key quality indicators (KQIs) describing the QoE and 91 key performance indicators (KPIs) describing the QoS. A two-process feature subset selection (FSS) method consisting of single parameter pre-FSS and multi-parameter FSS is then proposed to find the KPIs related to KQIs. An Extra-Trees model is finally developed to learn the relationships between the KPIs and KQIs. By employing machine learning and data analytics on network data with the data-driven framework, the proposed method can predict the user QoE according to the QoS of network servers. The results prove that our proposed method can outperform other state-of-the-art approaches.

Since the cloud storage services are increasingly used by cybercriminals, the law enforcement community has faced difficulties on how best to tackle the complex and dynamic developments on cloud services. This creates difficulties in the consistency of handling a digital crime scene. This study aims to explore the date-time stamp and timeline reconstruction of file operation in the cloud. It can be applied for incident response specialists who identify crime clues, characteristics or evidences in a data breach. The results demonstrate that the proposed findings are very effective for investigators to clear the sober and to find the fact behind digital date-time stamp.

Cloud Storage Forensics