An energy, performance efficient resource consolidation scheme for heterogeneous cloud datacenters

For the rapid increase in resource requirements in large scale Data Centers (DCs), enterprises have brought hyperconverged architecture where the storage pool is built up by the individual storage components associated with different servers, and it is shared among all the Virtual Machines (VMs) or containers through a common network infrastructure. Due to the sharing of network bandwidth among the application generated network traffic and the storage traffic from shared storage infrastructure, quality of service (QoS) performances of networking and storage intensive applications are affected, which further impacts VM or container migrations with dynamic workload scenarios. In this article, we propose NetStor to ensure QoS for various collocated network and storage intensive workloads over a hyperconverged architecture and ensure QoS during VM or container migration. NetStor uses a workload estimation strategy for VMs and containers, and applies the strategic decisions for resource allocation and migration based on environmental learning and workload characterization. NetStor supports a dynamic QoS provisioning that is workload-agnostic catering to both VMs and containers, and is unique to the best of our knowledge. We have implemented NetStor over a hyperconverged DC architecture on a testbed and found that NetStor can enhance network performance significantly compared to other related mechanisms discussed in the literature.

Cloud computing forms the backbone of the era of automation and the Internet of Things (IoT). It offers computing and storage-based services on consumption-based pricing. Large-scale datacenters are used to provide these service and consumes enormous electricity. Datacenters contribute a large portion of the carbon footprint in the environment. Through virtual machine (VM) consolidation, datacenter energy consumption can be reduced via efficient resource management. VM selection policy is used to choose the VM that needs migration. In this research, we have proposed PbV mSp: A priority-based VM selection policy for VM consolidation. The PbV mSp is implemented in cloudsim and evaluated compared with well-known VM selection policies like gpa, gpammt, mimt, mums, and mxu. The results show that the proposed PbV mSp selection policy has outperformed the exisitng policies in terms of energy consumption and other metrics.

Backfilling refers to the practice of allowing small jobs to be completed ahead of schedule as long as they do not cause the first job in the line to wait. Users are expected to offer estimates of how long jobs will take to complete in order to make these decisions possible, and these projections are often based on historical data. However, predictions are very hard and may not be accurate, particularly in cloud computing scenarios where jobs or applications run on Virtual Machines (VMs). In addition, scheduling and consolidation techniques can improve the energy efficiency and performance of applications. Consolidation involves VM migrations that can have a negative impact on workload performance and users’ costs. Backfilling can be used as an alternative technique for consolidation (short-term) and/or can be used along with consolidation (long-term). Backfilling methods are well-utilised in single computing systems, but are relatively unexplored in cloud resource allocation. A backfilling-based resource allocation and consolidation technique is proposed. Using real workloads from the Google cluster traces, we investigate the impact of backfilling on infrastructure energy efficiency and performance. For 12583 heterogeneous servers and approximately three million jobs that belong to three different applications, we observed that approximately 19% energy savings and 6% workload performance improvements are achievable using the backfilling approach. Furthermore, our evaluation suggests that using VM runtime as a criterion for the backfilling approach is approximately 3.56%–7.78% more energy and 1.91%–3.38% more performance efficient than using priority as a backfilling criterion.

Cloud providers generally run one or more Virtual Machine (VM) instances on the same physical machine. Though it increases data center utilization, it exposes VM to a co-location attack. VM placement and migration are the two strategies adopted for mitigating co-locations. Current methods for VM placement or VM migration consider only security as decision criteria and do not consider other factors like Quality-of-Service degradation, data center utilization, etc. This work proposes a placement and migration strategy for mitigation of co-location attacks with joint consideration of multi objectives like QoS, data center utilization, energy consumption, and security risks. A security-driven multi-criteria optimization -based VM placement policy is proposed. A joint consideration of multiobjective performance optimization along with co-location security risk minimization is done to design a novel VM placement policy based on user categorization. The policy can reduce the likelihood of co-location target VM with attacker VM without much degradation to the performance of VM and data center utilization. The solution mitigates co-location risks without much compromise to the performance of VM and data center resource utilization. The co-residence risk is mitigated by the categorization of users into three levels i.e. unlabeled, risky, and safe, and physical machines into two groups as safe and unsafe. The PMs available in data center is grouped into three different VM placement policies, they are undecided pool, safe pool and unsafe pool.

Cloud computing is currently playing an essential role in supporting emerging sectors such as smart energy, intelligent transportation, and large-scale distributed systems. The scalability and adaptability features of cloud computing are efficiently enable resource utilization and continuous data exchange in dynamic and heterogeneous environments. However, the increasing demand for cloud services has raised the energy consumption of cloud data centers, which has a critical environmental impact. Therefore, sustainable resource management tactics have become crucial in today's world. Dynamic Virtual Machine (VM) consolidation is one of the major tactics for sustainable resource management in the green cloud computing environment. Herein, a novel dynamic Energy-Aware Cosine Similarity Learning Network (ECSLN) is proposed to predict the overutilized host. Further, an Impact Factor-Based VM Selection (IFBVMS) method is proposed to select VMs for migration, and an ECSLN-Packed Placement method for VM placement into hosts. The main aim of the proposed VM consolidation is to maximize energy efficiency while preserving compliance with Service Level Agreements (SLAs) for sustainable environments. The experimental evaluation using real-world traces like PlanetLab, Bitbrains, and Alibaba Cluster 2020 workload validates that the proposed VM consolidation methods significantly reduce energy consumption and SLA violation compared to the existing methods for sustainable environments. The evaluation shows that the proposed ECSLN-based consolidation framework enables green cloud infrastructure by intelligently balancing energy consumption, SLA violation, and performance of cloud data centers.

The rapid growth in cloud computing encourages the cloud provider to create more reliable, scalable and efficient cloud environment. For providing such a scalable environment, cloud provider develops large-scale Cloud Data Centers (CDC). Such CDCs consume a large amount of electrical energy for operational purpose and cooling purpose to keep the machines operating at a right temperature, but it also emits a large amount of carbon-dioxide. Thus to decrease the energy consumption, Virtual Machines (VMs) are consolidated to fewer numbers of active servers through VM migration techniques, this leads to Service Level Agreement (SLA) violation and performance degradation. Therefore to retain the energy-performance tradeoff, we should perform an optimum number of VM migrations. There are two cases when the VM migration performed: First when a hot spot is found and Second when a cold spot is found. But in both the cases we do not directly migrate the VM instead first we check whether it is required to migrate the VM. After that, the algorithm finds an appropriate destination host on which the VM will be migrated. In this paper, we implement three approaches to select destination host: First-fit, Best-fit, and Worst-fit. Finally compare the result of a number of VM migration performed, energy consumption and SLA violations with existing one.

Increasing demand for Cloud infrastructure and services leads to the challenges for management and maintenance of large data Center. Data center is fully equipped with huge number of resources. Those resources consumes energy in spite of their partial or full utilization. As a result data center consumes lots of energy, which in turn increases the total cost of operation and carbon footprint in environment. These concern leads to “Green Computing”, i.e. to reduce total operational cost, Carbon Footprint in environment and efficient usage of the computing resources. In data center main processing element is virtual machine (VM), which is an instance of computing and storage resources, handles computational processes. Hence, it is important to reduce energy consumed by VM. As the workload distribution is varying in data Center as per the need, the number of VMs configured in the host are uneven, but host consumes maximum energy every time, irrespective of the workload. This leads to wastage of computational resources. This paper is intended to analyze such issues and specifically prove an algorithm which, significantly reduces energy consumption in data center, while ensuring SLA, when VM is in migration from one host to another in the data center.

High energy consumption in data center has become a key problem, and the Carbon (CO $$_2$$ ) emissions from data center have serious impacts on environment. To improve traditional performance-energy model and minimize energy consumption in cloud data center, an energy-efficient and QoS-aware (EEQoS) model is developed for virtual resources consolidation. The EEQoS model is designed to investigate the trade-off between energy and QoS. The user’s satisfaction degree and Euclidean distance of resources are also respectively applied to the QoS model and power model. In addition, this EEQoS model is combined with the particle swarm optimization algorithm by setting the power consumption per QoS value as the objective function to consolidate virtual resources in data center. Compared with traditional power model, the proposed model reduces response time by 27.2%, cost by 31.4%, SLA violations by 40.5% and improves throughput by 13.5% on average while increasing less than 3.8% energy consumption during the whole test period.

Security-aware dynamic VM consolidation

Virtual machine (VM) consolidation is an effective way to improve resource utilization and reduce energy consumption in cloud data centers. Most existing studies have considered VM consolidation as a bin-packing problem, but the current schemes commonly ignore the long-term relationship between VMs and hosts. In addition, there is a lack of long-term consideration for resource optimization in the VM consolidation, which results in unnecessary VM migration and increased energy consumption. To address these limitations, a VM consolidation method based on multi-step prediction and affinity-aware technique for energy-efficient cloud data centers (MPaAFVMC) is proposed. The proposed method uses an improved linear regression prediction algorithm to predict the next-moment resource utilization of hosts and VMs, and obtains the stage demand of resources in the future period through multi-step prediction, which is realized by iterative prediction. Then, based on the multi-step prediction, an affinity model between the VM and host is designed using the first-order correlation coefficient and Euclidean distance. During the VM consolidation, the affinity value is used to select the migration VM and placement host. The proposed method is compared with the existing consolidation algorithms on the PlanetLab and Google cluster real workload data using the CloudSim simulation platform. Experimental results show that the proposed method can achieve significant improvement in reducing energy consumption, VM migration costs, and service level agreement (SLA) violations.

[Purpose] Server virtualization technology is the basic form for the cloud data center to serve customers. While the cloud data center reduces the SLA violations in the early stage, the increase in the utilization of resources such as virtual machine servers and network equipment and decrease in energy consumption have become an important issue for reduction of operating costs. Therefore, on the background of the cloud data center network based on SDN technology, this paper studies a cloud data center resource utilization and energy consumption optimization strategy through virtual machine migration and integration. [Method] The technology based on SDN is firstly introduced, the approximate optimal solution FFD algorithm for the multidimensional vector bin packing problem is then presented, the Partheno-Genetic Algorithm (PGA) is then recommended, finally a Hybrid Partheno genetic Algorithm (HPGA) based on the both is established and a mathematic model is created. [Results] By combining the advantages of FFD and PGA, this paper improves the utilization of the resources of the entire cloud data center, reduces energy consumption, and prevents the occurrence of computing “hot spots”. [Conclusion] Experiments in CloudSim prove the effectiveness of the algorithm. Compared with FFD and PGA algorithms, it is able to enhance the efficiency of the Data Center effectively, and obtain the optimal solution more quickly, thereby balancing the physical resources of the server.

Reducing the energy consumption of data centers and the Cloud is very important in order to lower CO_2 footprint and operational cost (OPEX) of a Cloud operator. To this extent, it becomes crucial to minimise the energy consumption by consolidating the number of powered-on physical servers that host the given virtual machines (VMs). In this work, we propose a novel approach to the energy efficient VM consolidation problem by applying Robust Optimisation Theory. We develop a mathematical model as a robust Mixed Integer Linear Program under the assumption that the input to the problem (e.g. resource demands of the VMs) is not known precisely, but varies within given bounds. A numerical evaluation shows that our model allows the Cloud Operator to tradeoff between the power consumption and the protection from more severe and unlikely deviations of the uncertain input.

Virtualization, one of the most actively adopted technologies today in computer systems, is increasingly widening its range of applications. As multiple virtual machines are concurrently executed on a physical machine, they compete for physical resources such as CPU, DRAM, HDD and NIC. In large scale data centers or cluster systems, therefore, it is required to balance each physical machine's load by migrating a virtual machine to another physical machine. Since system performance is significantly hindered during migration, a technique that can reduce the migration overheads is required. Interestingly, we notice that virtual machines' memory contain numerous dispensable data. Therefore, in this paper, we propose an efficient virtual machine migration technique called D3 (Discarding Dispensable Data). The basic idea is simple but effective: finding indispensable page frames, and then, transferring only those page frames. In D3, simple metadata is handed over for the dispensable page frames, instead of actual data transmission, and the discarded page frames are reconstructed appropriately after the migration process. One issue is that checking whether a page frame is dispensable or not based on comparing the entire content yields excessive overhead. To overcome this problem, we devise a novel sampling based dispensable page frame detection technique. D3 have implemented in Xen virtualization software and evaluated on a realistic environment with relevant workloads. We show that D3 can efficiently reduce migration time up to 64.2% and size of memory required for completing a migration up to 94.8%. We also present that D3 is very useful to provide a certain level of QoS and better performance.

DataABC: A fast ABC based energy-efficient live VM consolidation policy with data-intensive energy evaluation model

Cloud computing is a highly scalable and cost - effective infrastructure for running High Performance Computing, enterprise and Web applications. However, the growing demand of Cloud infrastructure has drastically increased the energy consumption of data centers, which has become a critical issue. Hence, energy efficient solutions are required to minimize this energy consumption. The energy efficient solutions aim at lowering the energy usage of data centers because computing applications and data are growing so quickly that increasingly larger servers and disks are needed to process them fast enough within the required time period so here we reduce the energy consumption by an average of 40% over previously introduced methods. So in datacenters, the number of physical machines can be reduced using virtualization by consolidating virtual machines onto shared servers and enabling them to migrate according to migration policy. This paper presents virtual machines migration and selection policies to boost Cloud Computing Environment energy efficiency and performance.