Capacity planning for IaaS cloud providers offering multiple service classes

Resource under-utilization is prevalent in data centers as a result of substantial stochastic workload variations and resource provision guarantee for the requirements to provide high-level service availability. To avoid resource under-utilization, recently, the Economy cloud computing service class (Economy class in short) is proposed that sells allocated but unused (i.e., reclaimed) physical resources with long-term service level objectives (SLOs). Though the Economy class would lead to revenue benefit, how much of the reclaimed resources should be used as the Economy class (i.e., capacity planning) and how to set its price and SLAs to maximize revenue benefit have not been studied. To address this problem, in this paper, we propose a multi-level SLA for Economy class, which has multiple levels of SLOs and penalty. We then present a rigorous theoretical analysis on the capacity planning and pricing of the new Economy class with an objective to maximize the revenue of the cloud computing service provider. Theoretical results are validated using resource utilization data derived from a publicly available Google cluster workload trace dataset.

Latency-critical workloads such as web search engines, social networks and finance market applications are sensitive to tail latencies for meeting Service Level Objectives (SLOs). Since unexpected tail latencies are caused by sharing hardware resources with other co-executing workloads, a service provider executes the latency-critical workload alone. Thus, the data center for the latency-critical workloads has exceedingly low hardware resource utilization. For improving hardware resource utilization, the service provider has to co-locate the latency-critical workloads and other batch processing workloads. However, because memory bandwidth cannot be provided in isolation unlike cores and cache memory, the latency-critical workloads experience poor performance isolation even though the core and the cache memory are allocated in isolation to the workloads. To solve this problem, we propose an optimized memory bandwidth management approach for ensuring Quality of Service (QoS) and high server utilization. By providing isolated shared resources including memory bandwidth to the latencycritical workload and co-executing batch processing ones, our proposed approach guarantees SLOs and improves hardware resource utilization. Firstly, we predict the size of the memory bandwidth to meet the SLO for all Queries Per Seconds (QPSs) while executing the latency-critical workload with minimal preprofiling. Then, our approach allocates the amount of the isolated memory bandwidth that guarantees the SLO to the latencycritical workload and the rest of the memory bandwidth to coexecuting batch processing workloads. As a result, our proposed approach can achieve up to 99% SLO assurance and improve server utilization up to 6.5x.

The current large selection of cloud instances that are functionally equivalent makes selecting the right cloud service a challenging decision. We envision a model driven engineering (MDE) approach to raise the level of abstraction for cloud service selection. One way to achieve this is through a domain specific language (DSL) for modelling the service level objectives (SLOs) and a brokerage system that utilises the SLO model to select services. However, this demands an understanding of the provider SLAs and the capabilities of the current cloud modelling languages (CMLs). This paper investigates the state-of-the-art for SLO support in both cloud providers SLAs and CMLs in order to identify the gaps for SLO support. We then outline research directions towards achieving the MDE-based cloud brokerage.

The rapid growth of cloud computing in cloud datacenters in recent decades greatly increases the brown energy consumption in datacenters, and hence significant increase of carbon emission that negatively impacts on the environment as well as the monetary cost. More and more cloud service providers are adopting renewable energy as the energy supply to offset the consumption of brown energy. Meanwhile, an increasing number of renewable energy generators have been built to meet the needs. However, the instability of the renewable energy cannot guarantee the support to the datacenter and the energy competition of different datacenters may lead to datacenter energy outage. In this paper, we focus on the problem of how to match different renewable energy generators to the datacenters from different cloud providers to minimize the carbon emission, monetary cost, and service level objective (SLO) violation due to renewable energy shortage. The challenges here are that the datacenters may compete in energy requesting, the renewable energy generation is not stable and the decision should be made quickly. There have been no previous efforts devoting to this problem. To solve the problem, we first test several machine learning techniques on long-term prediction accuracy on renewable energy generation and energy demand using real traces and identify SARIMA for the prediction. We then propose a multi-agent reinforcement learning based method (MARL) for each datacenter to determine how much renewable energy to request from each generator based on the predicted results. We also propose a deadline guaranteed job postponement method (DGJP) to postpone executing unurgent jobs upon insufficient renewable energy supply. The trace-driven experiments show that MARL outperforms other methods by increasing up to 35% SLO satisfaction ratio, and reducing up to 19% (0.33 billion dollars in 90 days) total monetary cost and 33% total carbon emission, and DGJP further improves the performance.

We develop three auction-based pricing and allocation solution methods for the case where a capacity-constrained online service provider offers multiple classes of unique, one-time services with differentiated quality. Consumers desire exactly one of the many service classes offered. We call such a setting a vertically integrated online services market. Examples of these services are webcasting of special events over the Internet, provision of video-on-demand, and allocation of grid computing resources. We model the pricing and allocation decision faced by firms in such a setting as a knapsack problem with an added preference elicitation dimension. We present a variety of computational solution approaches based on adaptations of the traditional greedy heuristic for knapsack problems. The solution approaches vary in efficacy depending on whether bidders are restricted to bid in one service class or allowed to bid in multiple service classes, as well as on the overall variability of the demand. In the case bidders can bid in multiple classes but are interested in consuming only one service class, a direct application of the heuristics developed for the single service case results in a nonfair allocation. We develop a novel data structure to eliminate the unfair allocation while maintaining the original computation complexity of the simpler setting. The paper contributes by presenting a menu of auction clearing mechanisms for selling vertically integrated online services.

Analyzing Hadoop power consumption and impact on application QoS

Cloud service is being adopted as a utility for large numbers of tenants by renting Virtual Machines (VMs). But for cloud storage, unpredictable IO characteristics make accurate Service-Level-Objective (SLO) enforcement challenging. As a result, it has been very difficult to support simple-to-use and technology-agnostic SLO specifying a particular value for a specific metric (e.g., storage bandwidth). This is because the quality of SLO enforcement depends on performance error and fluctuation that measure the precision of SLO enforcement . High precision of SLO enforcement is critical for user-oriented performance customization and user experiences. To address this challenge, this article presents V-Cup, a framework for VM-oriented customizable SLO and its near-precise enforcement. It consists of multiple auto-tuners, each of which exports an interface for a tenant to customize the desired storage bandwidth for a VM and enable the storage bandwidth of the VM to converge on the target value with a predictable precision. We design and implement V-Cup in the Xen hypervisor based on the fair sharing scheduler for VM-level resource management. Our V-Cup prototype evaluation shows that it achieves satisfying performance guarantees through near-precise SLO enforcement.

Traditionally, any capacity planning problem is modeled with deterministic workloads by considering the peak workload for resource allocation. In the context of businesses using cloud service, cloud provider could allocate resources for peak workload which could lead to under utilization of resource and charging users for unused yet provisioned resources. Hence we came up with a better capacity planning algorithm which could ensure that we plan for peak usage but do not provision for it.In our approach, we modeled the problem as a stochastic optimization problem with the objective of minimizing the number of servers considering two important constraints a) stochastic nature of workloads and b) minimizing the application SLA violations. We implemented the model using genetic algorithm and to address the stochastic nature of work loads, we reserved a free pool of resources in each server by the quantity determined by our algorithm. We evaluated the solution with real sever utilization data from a datacenter seeking consolidation. We did comparative analysis on the number of servers required suggested by our solution vs. peak work loads based solutions for various service levels. Our results illustrate that reserving certain amount of resources in servers for addressing variability of workloads gives better results in terms of lesser number of servers compared to packing resources based on peak workloads for the same service levels.

Elastic threshold-based admission control for QoS satisfaction with reward optimization for servicing multiple priority classes in wireless networks

By choosing to use cloud services, organizations seek to reduce costs and maximize efficiency. For mission critical systems that must satisfy security constraints, this push to the cloud introduces risks associated with cloud service providers not implementing organizationally selected security controls or policies. As internal system details are abstracted away as part of the cloud architecture, the organization must rely on contractual obligations embedded in service level agreements (SLAs) to assess service offerings. Current SLAs focus on quality of service metrics and lack the semantics needed to express security constraints that could be used to measure risk. We create a framework, called SecAgreement (SecAg), that extends the current SLA negotiation standard, WS-Agreement, to allow security metrics to be expressed on service description terms and service level objectives. The framework enables cloud service providers to include security in their SLA offerings, increasing the likelihood that their services will be used. We define and exemplify a cloud service matchmaking algorithm to assess and rank SecAg enhanced WS-Agreements by their risk, allowing organizations to quantify risk, identify any policy compliance gaps that might exist, and as a result select the cloud services that best meet their security needs.

Cost-effective cloud storage services attract users with their convenience; there is a trade-off between the service availability and the usage cost. We develop two cloud provider selection models for cloud storage services to minimize the total cost of usage. The models select multiple cloud providers with considering unavailability to meet the user requirements. The first model, called a user-copy (UC) model, allows to select multiple cloud providers, where a user itself makes copies of its data for multiple providers. In addition to the user copy function of the UC model, the second model, which is called a user and cloud-provider copy (UCC) model, allows cloud providers to make copies of the data to deliver them to other cloud providers. The cloud service is available if at least one cloud provider is available. We formulate both models as integer linear programming (ILP) problems. Our performance evaluation observes that both models reduce the total cost of usage, compared to the single cloud provider selection. As the cost of bandwidth usage between a user and a cloud provider increases, the UCC model becomes more beneficial than the UC model. We implement the prototype for cloud storage services.

Many Cloud Service Providers (CSPs) provide data storage services with datacenters distributed worldwide. These datacenters provide different Get/Put latencies and unit prices for resource utilization and reservation. Thus, when selecting different CSPs' datacenters, cloud customers of globally distributed applications (e.g., online social networks) face two challenges: i) how to allocate data to worldwide datacenters to satisfy application SLO (service level objective) requirements including both data retrieval latency and availability, and ii) how to allocate data and reserve resources in datacenters belonging to different CSPs to minimize the payment cost. To handle these challenges, we first model the cost minimization problem under SLO constraints using integer programming. Due to its NP-hardness, we then introduce our heuristic solution, including a dominant-cost based data allocation algorithm and an optimal resource reservation algorithm. We finally introduce an infrastructure to enable the conduction of the algorithms. Our trace-driven experiments on a supercomputing cluster and on real clouds (i.e., Amazon S3, Windows Azure Storage and Google Cloud Storage) show the effectiveness of our algorithms for SLO guaranteed services and customer cost minimization.

Recent advancements in the domain of Network Function Virtualization (NFV), and rollout of next-generation networks have necessitated the requirement for the upkeep of latency-critical application architectures in future networks and communications. While Cloud service providers recognize the evolving mission-critical requirements in latency sensitive verticals such as autonomous driving, multimedia, gaming, telecommunications, and virtual reality, there is a wide gap to bridge the Quality of Service (QoS) constraints for the end-user experience. Most latency-critical services are over-provisioned on all fronts to offer reliability, which is inefficient towards scalability in the long run. To address this, we propose a strategy to model frequent violations on the application level as a multi-output target to enable more complex decision-making in the management of virtualised communication networks. In this work, we utilize data from a real-world deployment to configure and draft a realistic set of Service Level Objectives (SLOs) for a voice based NFV application, and develop a deep neural network based multi-label classification methodology to identify and predict multiple categories of SLO breaches associated with an application state. With this, we aim to gain granular SLA and SLO violation insights, enabling us to study and mitigate their impact and inform precision in drafting proactive scaling policies. We further compare the performance against a set of multi-label compatible machine learning classifiers, and address class imbalance in a multi-label setup. We perform a comprehensive evaluation to assess the performance on example-based, label-based and ranking-based measures, and demonstrate the suitability of deep learning in such a use-case.

The diversity of cloud offerings motivated the proposition of cloud modelling languages (CMLs) to abstract complexities related to selection of cloud services. However, current CMLs lack the support for modelling service level objectives (SLOs) that are required for the customer applications. Consequently, we propose an application- and provider-independent SLO modelling language (SLO-ML) to enable customers to specify the required SLOs. We also sketch the architecture to realise SLO-ML.

Infrastructure as a Service (IaaS) providers, such as Amazon Web Services, offer on-demand access to computing resources at pay-as-you-go prices. The key benefit of IaaS is elasticity, i.e., the ability to provision and de-provision resources at will. This feature makes IaaS infrastructure as the best platform for hosting web applications, e.g. e-business, that are subjected to highly-variable request patterns. However, elasticity can be triggered either on the basis of resource utilization or for meeting service level objectives (SLOs). In this paper, we extensively evaluate these two types of elasticity rules using the TPC-W benchmark on Amazon IaaS infrastructure. From this experimental data, we evaluate the performance of these rules against the primary metric of service level satisfaction for web applications, and secondary metrics such as resource utilization and cost. Through our inferences, we present a number of recommendations that would enable practitioners and cloud consumers using Amazon to define appropriate elasticity rules to meet their SLOs and other metrics.KeywordsAverage Response TimeSmall InstanceMedium InstanceElasticity ApproachCloud ConsumerThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.