Network Traffic Analysis with Cloud Platform

SUMMARYNetwork traffic monitoring and analysis are essential for effective network operation and resource management. In particular, multidimensional analysis for long‐term traffic data is necessary for comprehensive understanding of the traffic trend and effective quality‐of‐service provision considering the extremely dynamic behavior of the current Internet, where various types of traffic occur from high‐speed network links and greatly increasing number of applications. However, only limited analysis results are provided, as the existing network traffic analysis tools and systems are developed and deployed focusing on their own specialized analysis purposes. Consequently, it is difficult to understand the network comprehensively and deeply, which increases the necessity for multilateral analysis of long‐term traffic data. In this paper, we propose a novel traffic analysis model for large volumes of Internet traffic accumulated over a long period of time. TheNetCube, the proposed network traffic analysis model using online analytical processing (OLAP) on a multidimensional data cube, provides an easy and fast way to construct a multidimensional traffic analysis system for comprehensive and detailed analysis of long‐term traffic data by utilizing simple OLAP operations and powerful data‐mining techniques on various abstraction levels of traffic data to complete the analysis purpose. We validate the feasibility and applicability of the proposedNetCubetraffic analysis model by implementing a traffic analysis system and applying it to our campus network. Copyright © 2012 John Wiley & Sons, Ltd.

Given the current expansion of cloud computing, the expected advent of the Internet of Things, and the requirements of future fifth‐generation network infrastructures, significantly larger pools of computational and storage resources will soon be required. This emphasizes the need for more scalable data centers that are capable of providing such an amount of resources in a cost‐effective way. A quick look into today's commercial data centers shows that they tend to rely on variations of well‐defined leaf‐spine/Clos data center network (DCN) topologies, offering low latency, ultrahigh bisectional bandwidth, and enhanced reliability against concurrent failures. However, DCNs are typically restricted by the use of the Transmission Control Protocol/Internet Protocol (TCP/IP) suite, thus suffering limited routing scalability. In this work, we study the benefits that replacing TCP/IP with the recursive internetwork architecture (RINA) can bring into commercial DCNs, focusing on forwarding and routing scalability. We quantitatively evaluate the benefits that RINA solutions can yield against those based on TCP/IP and highlight how, by deploying RINA, topological routing solutions can improve even more the efficiency of the network. To this goal, we propose a rule‐and‐exception forwarding policy tailored to the characteristics of several DCN variants, enabling fast forwarding decisions with merely neighbors' information. Upon failures, few exceptions are necessary, whose computation can also profit from the known topology. Extensive numerical results show that the proposed policy requirements depend mainly on the number of neighbors and concurrent failures in the DCN rather than its size, dramatically reducing the amount of forwarding and routing information stored at DCN nodes.

Today's data centers use substantial amounts of the world's electrical supply. However, in line with circular economy concepts, much of this energy can be reused. Such reuse includes the heating of buildings, but also commodity dehydration, electricity production and energy storage. This multi-disciplinary paper presents several novel applications for data center waste heat. Next, the paper accounts for three case studies, taken from three different societal contexts: urban Malaysia, rural Costa Rica and semi-urban Sweden. A discussion on data center energy metrics leads to the development of a new metric, Datacenter Energy Sustainability Score (DESS), which is evaluated within the three use cases. Last, it is shown how a material social view on metrics provides a way past a problem that has haunted the data center industry for the last 15 years, whilst benefitting both data center owners who want to compete through sustainability as well as stakeholders from governments on local, regional and national levels. The paper makes clear that a sustainability strategy should be based on a material social view and stretch beyond the building itself. In fact, and as demonstrated by the relevance of DESS, modern data centers are so energy-efficient that data center sustainability is no longer mainly an engineering issue, but a matter requiring multi-disciplinary insights, approaches and collaboration.

With the rapid increasing of network scale, the size of traffic data also expands a lot. In traditional traffic data analysis, there are some problems, such as high computation complexity, low analysis efficiency, long learning period, and difficulty of development. To address these problems, we design and implement a GPU-accelerated parallel analysis scheme for network traffic - EasyAnalyze. In EasyAnalyze, we introduce GPU parallel computing, Map/Reduce architecture into network traffic analysis, which greatly improves the efficiency but does not increase the difficulty in programming. In the experiments, EasyAnalyze shows very promising results: 1 the speed is 6-17 times faster than conventional serial analysis in network traffic data analysis; and 2 the size of code is only 2% of the mainstream GPU Map/Reduce.

Today's data centers tend to have tens to hundreds of thousands of servers to provide massive and sophisticated services. Statistically, data center and data center networks (DCNs) remain highly underutilized which can be exploited for energy-saving. In this paper, we have studied energy-saving problem for network and server sides of the data center. The problem was formulated as a Mixed Integer Linear Program (MILP) that is solvable by an optimization software to jointly minimize the energy consumed by the servers and DCN. To overcome the optimization software high computational time, a heuristic algorithm to provide practical and efficient solution is introduced. The heuristic algorithm has two stages: first, it uses the virtual machines (VM) and the predicted servers resource utilization to provide VM consolidation algorithm and turn-off unused servers. The second stage uses an abstract performance-aware network flow consolidation that focused the traffic on subset of the network and turn-off the unused network devices. Simulation experiments using CloudsimSDN were conducted to validate the heuristic using real traces from Wikipedia in terms of energy consumption and average response time. The results show that the heuristic can save servers and network energy while maintaining performance.

Energy consumption is a growing concern for data centers. Advances in server equipment technologies and increased demand for computing power have increased load densities in the computer room, which in turn has caused corresponding increases in data center power consumption. Energy efficiency measures are thus of high importance for data center designers, operators, and owners. This article outlines a software-based approach to the data gathering, trending, and analysis that are necessary to apply successful energy efficiency measures in data center environments.

Today's data centers offer IT services mostly hosted on dedicated physical servers. Server virtualization provides a technical means for server consolidation. Thus, multiple virtual servers can be hosted on a single server. Server consolidation describes the process of combining the workloads of several different servers on a set of target servers. We focus on server consolidation with dozens or hundreds of servers, which can be regularly found in enterprise data centers. Cost saving is among the key drivers for such projects. This paper presents decision models to optimally allocate source servers to physical target servers while considering real-world constraints. Our central model is proven to be an NP-hard problem. Therefore, besides an exact solution method, a heuristic is presented to address large-scale server consolidation projects. In addition, a preprocessing method for server load data is introduced allowing for the consideration of quality-of-service levels. Extensive experiments were conducted based on a large set of server load data from a data center provider focusing on managerial concerns over what types of problems can be solved. Results show that, on average, server savings of 31 percent can be achieved only by taking cycles in the server workload into account.

Netflow-based network traffic analysis is one of today's mainstream network traffic monitoring and analysis techniques. However, port-based traffic identification that employed in Netflow-based protocol analysis is inaccurate. Netflow cannot provide enough details for analysis of site access behaviour. In order to overcome the above limitation, we propose an enhanced Netflow data collection method combining packet capturing and flow technique for accurate application identification. Then we design and implement an enhanced Netflow data collection system, which lays a data foundation for traffic analysis. Finally, the feasibility and effectiveness of the system are verified.

NTLFlowLyzer: Towards generating an intrusion detection dataset and intruders behavior profiling through network and transport layers traffic analysis and pattern extraction

Big data analysis has become much popular in the present day scenario and the manipulation of big data has gained the keen attention of researchers in the field of data analytics. Analysis of big data is currently considered as an integral part of many computational and statistical departments. As a result, novel approaches in data analysis are evolving on a daily basis. Thousands of transaction requests are handled and processed everyday by different websites associated with e-commerce, e-banking, e-shopping carts etc. The network traffic and weblog analysis comes to play a crucial role in such situations where Hadoop can be suggested as an efficient solution for processing the Netflow data collected from switches as well as website access-logs during fixed intervals.

Address Resolution Protocol (ARP) is a communication protocol to map the computer's addresses to the Media Access Control (MAC) address. In its implementation, ARP is abused, known as ARP poisoning Attack. The impact of ARP poisoning attacks is a deadlock to communicate on the network, identity fraud from addressing a computer through illegal access to steal important and confidential information. Several ARP poisoning attack detection models have been introduced. Still, they depend on application tools requiring complex configuration and mostly state ARP poisoning attacks as normal activity. In this paper, a model for detecting ARP poisoning attacks is proposed using the K-NN classification. The proposed model has a contribution to the feature extraction process based on network traffic flows analysis. The results show that the proposed model can detect ARP poisoning attacks more accurately than some classification algorithms with a TPR value of 97.67% and a detection accuracy of 98.7%.

In today's modern cloud supported applications, the traffic relies on both congestion-responsive transport protocols like TCP and congestion-oblivious transport like UDP including all their variations. As bandwidth is not totally virtualized in today's data centers, the diverse responses of these protocols to congestion lead to inefficiencies and service disruption to some applications. In this paper we present HyGenICC, a simple, distributed and practical congestion control mechanism that puts congestion control back where it belongs, in the network layer, albeit, without modifying the network layer behaviour. To this end IP ECN marking in the switches to indicate congestion, combined with additional packet processing in the hypervisor enable HyGenICC to partition bandwidth among competing virtual machines (VMs) in datacenters effectively. To enable easy deployment in existing data center, HyGenICC design is subjected to several constraints such as: i) freedom of changes to the guest VMs' congestion control mechanism, and ii) reliance only on switch capabilities that are already available in today's commodity switches. We evaluate HyGenICC via extensive simulation in the ns2 network simulator.

Today's data centers rely on scale-out architectures like fat-tree, BCube, VL2, etc. to connect a large number of commodity servers. It's important to balance the traffic load across the available links. Since the traditional electrical network cannot perfectly respond to the traffic variations in data centers, a growing trend is to introduce converters with adjustable optical links instead of adding more wiring links. However, little is known today about how to fully exploit the potential of the flexibility from the converters: the joint optimization on adjusting the optical links inside the converters and the routing in the whole network remains algorithmically challenging. In this paper, we initiate the study of dynamic load balancing problem (DLBP) in hybrid switching data center networks with converters. We design a set of specific converters for Diamond, VL2, BCube topologies to introduce more flexibility. Based on it, the connections of the optical links inside the converter and the route for each flow needs to be jointly optimized to minimize the maximum link utilization in the whole network. We formulate DLBP as an optimization program and prove that it's not only NP-hard, but also ρ-inapproximation. Further, we design a greedy algorithm to solve it. Extensive experiments show that our algorithm can reduce the traffic congestion by 12% on average.

As bandwidth is shared in a best-effort way in today's data centers, traffic generated between a set of VMs (virtual machines) affect the traffic between another set of VMs (possibly belonging to another tenant) sharing the same physical links, leading to unpredictable performance of applications running on these VMs. This article addresses the problem of allocation of not only server resources (computational and storage) but also network bandwidth, to provide performance guarantees in multi-tenant data centers. Bandwidth being a critical shared-resource, we formulate the problem as an optimization problem that minimizes bandwidth demand between clusters of VMs of a tenant; and we prove it as NP-hard. We develop fast online heuristics as an integrated resource allocator (IRA) that decides on the admission of dynamically arriving requests, and allocates resources for the accepted ones. We also present a modified version of IRA, called B-IRA that bounds the cost of bandwidth allocation, while exploring smaller search space for solution. We demonstrate that, IRA accommodates significantly higher number of requests in comparison to a load-balancing resource allocator (LBRA) that does not consider reducing bandwidth between clusters of VMs. IRA also outperforms B-IRA when traffic demands of VMs in an input are not localized.

Cloud computing is next generation of Internet technology and data centers. As such, Information security (IS) has become one of the most essential issues of data center security on the cloud computing, but we cannot rely solely on single solution to support all of the necessary features of IS. Therefore, we need to a new information security service model, called Security Operation Center (SOC). It can support online interactions and remote monitoring via network operations in data centers. The SOC is required to integrate related IS technologies based on security policies. In this paper, we propose the service level agreement (SLA) strategy for designing the SOC platform to meet the IS requirements of any organization in cloud computing. Therefore, the main purposes of this paper are to propose the framework of SLA strategy and the integration technology for designing an SOC platform. The SOC platform with SLA is to handle and supervise all the security service processes under different levels of cloud security enforcements according to data center scales, business properties, and existing IS functions. Consequently, the SLA strategy and integrating technology of IS can be the reference model for designing a proposed SOC platform in data centers on the cloud computing.