Virtual Machine Consolidation with Multi-Step Prediction and Affinity-Aware Technique for Energy-Efficient Cloud Data Centers

Energy consumption in data center is currently the main focus of many large-scale enterprises and cloud service providers. Dynamic virtual machine (VM) consolidation technologies are widely used to improve resource utilization of data centers and reduce energy. They try to identify the poorly utilized physical hosts and make the most of the resources. Then idle hosts can be switched to sleep or active mode, meanwhile considering the real-time fluctuation of service workload. In this paper, we propose a Reinforcement Learning (RL) based Virtual Machine Consolidation (RL-VMC) framework with an application to the cloud data center operation. The RL-VMC method uses an agent corresponding to a VM planner to interact with the environment, which encapsulates the data center running state, and learns the optimal policy to determine the migration mapping from VMs to physical hosts. Specifically, we adopt the on-policy method State-Action-Reward-State-Action (SARSA) in RL-VMC, which learns from experience to get the optimal VMs migration strategy and manage the host power mode as well. Additionally, we use the PBRS technology to speed up the convergence of the RL-VMC. Experimental results show that the RL-VMC method can adapt to the dynamic workload maintaining an improved balance between energy consumption and Service Level Agreements (SLA).

Virtual machine (VM) consolidation is an emerging solution for energy saving in cloud data centers. An appropriate VM consolidation policy must change accordingly, depending on the situation. To easily deploy a VM consolidation solution, this paper proposes a VM consolidation framework for Apache CloudStack, which is a popular open source cloud platform software suite. We separate the VM consolidation mechanism from the policy used to decide which VMs should be moved to somewhere more appropriate to run. This framework is a modular architecture that combines various resource monitoring systems, power control mechanisms, and VM packing algorithms. Using the proposed framework, we have demonstrated that a Web application can consolidate and deconsolidate VMs so as to balance the load of CPU utilization among physical machines in response to the number of physical machines in operation. Experimental results reveal the proposed framework is preliminary, but provides the required functionality.

DataABC: A fast ABC based energy-efficient live VM consolidation policy with data-intensive energy evaluation model

Improving the energy efficiency while guaranteeing quality of services (QoS) is one of the main challenges of efficient resource management of large-scale data centers. Dynamic virtual machine (VM) consolidation is a promising approach that aims to reduce the energy consumption by reallocating VMs to hosts dynamically. Previous works mostly have considered only the current utilization of resources in the dynamic VM consolidation procedure, which imposes unnecessary migrations and host power mode transitions. Moreover, they select the destinations of VM migrations with conservative approaches to keep the service-level agreements , which is not in line with packing VMs on fewer physical hosts. In this paper, we propose a regression-based approach that predicts the resource utilization of the VMs and hosts based on their historical data and uses the predictions in different problems of the whole process. Predicting future utilization provides the opportunity of selecting the host with higher utilization for the destination of a VM migration, which leads to a better VMs placement from the viewpoint of VM consolidation. Results show that our proposed approach reduces the energy consumption of the modeled data center by up to 38% compared to other works in the area, guaranteeing the same QoS. Moreover, the results show a better scalability than all other approaches. Our proposed approach improves the energy efficiency even for the largest simulated benchmarks and takes less than 5% time overhead to execute for a data center with 7600 physical hosts.

Virtual Machine (VM) consolidation provides a promising approach to save energy and improve resource utilization in cloud data centers. However, aggressive consolidation of VMs may lead to performance degradation of data centers and even Service Level Agreement (SLA) violation. Therefore, it is very important to solve the tradeoff between reduction of SLA violation level and energy costs. In this paper, we propose a Host State 3-order Markov Chain prediction model for SLA-aware and energy efficient consolidation of VMs in cloud data centers. We propose two adaptive utilization thresholds based on a robust statistical method for host state estimate, and our model uses them to build a 3-order Markov Chain Transition Matrix manually for host state detection. We implement our algorithms on Cloudsim simulator for comparative study. The experimental results show that our model can significantly reduce SLA violation rates while keeping energy cost efficient, it can reduce the metric of SLA Violation by at most 98.69% and the metric of Energy by at most 21.88% for real world workload.

Server consolidation technique plays an important role in energy management and load-balancing of cloud computing systems. Dynamic virtual machine (VM) consolidation is a promising consolidation approach in this direction, which aims at using least active physical machines (PMs) through appropriately migrating VMs to reduce resource consumption. The resulting optimization problem is well-acknowledged to be NP-hard optimization problems. In this paper, we propose a novel merge-and-split-based coalitional game-theoretic approach for VM consolidation in heterogeneous clouds. The proposed approach first partitions PMs into different groups based on their load levels, then employs a coalitional-game-based VM consolidation algorithm (CGMS) in choosing members from such groups to form effective coalitions, performs VM migrations among the coalition members to maximize the payoff of every coalition, and close PMs with low energy-efficiency. Experimental results based on multiple cases clearly demonstrate that our proposed approach outperforms traditional ones in terms of energy-saving and level of load fairness.

The demand for cloud computing has increased manifold in the recent past. More specifically, on-demand computing has seen a rapid rise as organizations rely mostly on cloud service providers for their day-to-day computing needs. The cloud service provider fulfills different user requirements using virtualization - where a single physical machine can host multiple Virtual Machines. Each virtual machine potentially represents a different user environment such as operating system, programming environment, and applications. However, these cloud services use a large amount of electrical energy and produce greenhouse gases. To reduce the electricity cost and greenhouse gases, energy efficient algorithms must be designed. One specific area where energy efficient algorithms are required is virtual machine consolidation. With virtual machine consolidation, the objective is to utilize the minimum possible number of hosts to accommodate the required virtual machines, keeping in mind the service level agreement requirements. This research work formulates the virtual machine migration as an online problem and develops optimal offline and online algorithms for the single host virtual machine migration problem under a service level agreement constraint for an over-utilized host. The online algorithm is analyzed using a competitive analysis approach. In addition, an experimental analysis of the proposed algorithm on real-world data is conducted to showcase the improved performance of the proposed algorithm against the benchmark algorithms. Our proposed online algorithm consumed 25% less energy and performed 43% fewer migrations than the benchmark algorithms.

Cloud computing is currently playing an essential role in supporting emerging sectors such as smart energy, intelligent transportation, and large-scale distributed systems. The scalability and adaptability features of cloud computing are efficiently enable resource utilization and continuous data exchange in dynamic and heterogeneous environments. However, the increasing demand for cloud services has raised the energy consumption of cloud data centers, which has a critical environmental impact. Therefore, sustainable resource management tactics have become crucial in today's world. Dynamic Virtual Machine (VM) consolidation is one of the major tactics for sustainable resource management in the green cloud computing environment. Herein, a novel dynamic Energy-Aware Cosine Similarity Learning Network (ECSLN) is proposed to predict the overutilized host. Further, an Impact Factor-Based VM Selection (IFBVMS) method is proposed to select VMs for migration, and an ECSLN-Packed Placement method for VM placement into hosts. The main aim of the proposed VM consolidation is to maximize energy efficiency while preserving compliance with Service Level Agreements (SLAs) for sustainable environments. The experimental evaluation using real-world traces like PlanetLab, Bitbrains, and Alibaba Cluster 2020 workload validates that the proposed VM consolidation methods significantly reduce energy consumption and SLA violation compared to the existing methods for sustainable environments. The evaluation shows that the proposed ECSLN-based consolidation framework enables green cloud infrastructure by intelligently balancing energy consumption, SLA violation, and performance of cloud data centers.

Cloud datacenters form the underlying infrastructure of cloud computing, containing a large number of physical machines (servers).The servers are virtualized to host many virtual machines (VMs) that support a wide range of services for cloud users.Efficient management of VMs is crucial to achieve optimum resource utilization, better energy efficiency, and a high degree of quality of service (QoS).Virtual machines consolidation is one of the main mechanisms to achieve such objectives.Consolidation techniques involve dynamically re-allocation of VMs across physical machines to balance workloads, reduce energy consumption, and maintain system reliability.However, VM consolidation is a complex process that can introduce significant overhead in terms of power consumption, network bandwidth, and service disruption.This makes its optimization a critical challenge for cloud service providers.This paper proposes DHLT-ACO (Dynamic Host Load Thresholding with Ant Colony Optimization) to enhance VM consolidation through two key mechanisms.At first, an adaptive host load detection method based on exponentially weighted moving average (EWMA) and a confidence interval-based dynamic thresholds to enhance the accuracy of detecting underloaded and overloaded hosts.Second, an improved Ant Colony Optimization (ACO) algorithm to optimize VMs placement by dynamically adjusting exploration and exploitation parameters, ensuring faster convergence and better resource allocation.Experimental results demonstrate that DHLT-ACO outperforms existing methods in terms of power consumption, number of VMs migration, and minimizing service level agreement (SLA) violations.DHLT-ACO achieved energy savings of 21.3%, reduced the number of migrations by 47%, and minimized SLA violations by 45.1% over the utilization-aware VM placement (UAVMP), hybrid VM allocation and placement (HVMAP), normalization-based VM consolidation (NVMC), and modified genetic-based VM consolidation (MGVMC) on average.These improvements verify the efficiency of DHLT-ACO in achieving energy efficiency, reducing the number of migrations and ensuring compliance with SLA in cloud computing environments.

With the rapid development of cloud computing, the demand for infrastructure resources in cloud data centers has further increased, which has already led to enormous amounts of energy costs. Virtual machine (VM) consolidation as one of the important techniques in Infrastructure as a Service clouds (IaaS) can help resolve energy consumption by reducing the number of active physical machines (PMs). However, the necessity of considering energy-efficiency and the obligation of providing high quality of service (QoS) to customers is a trade-off, as aggressive consolidation may lead to performance degradation. Moreover, most of the existing works of threshold-based VM consolidation strategy are mainly focused on single CPU utilization, although the resource request on different VMs are very diverse. This paper proposes a novel self-adaptive VM consolidation strategy based on dynamic multi-thresholds (DMT) for PM selection, which can be dynamically adjusted by considering future utilization on multi-dimensional resources of CPU, RAM and Bandwidth. Besides, the VM selection and placement algorithm of VM consolidation are also improved by utilizing each multi-dimensional parameter in DMT. The experiments show that our proposed strategy has a better performance than other strategies, not only in high QoS but also in less energy consumption. In addition, the advantage of its reduction on the number of active hosts is much more obvious, especially when it is under extreme workloads.

Virtual machine consolidation has been a widely explored topic in recent years due to Cloud Data Centers’ effect on global energy consumption. Thus, academia and companies made efforts to achieve green computing, reducing energy consumption to minimize environmental impact. By consolidating Virtual Machines into a fewer number of Physical Machines, resource provisioning mechanisms can shutdown idle Physical Machines to reduce energy consumption and improve resource utilization. However, there is a tradeoff between reducing energy consumption while assuring the Quality of Service established on the Service Level Agreement. This work introduces a Systematic Literature Review of one year of advances in virtual machine consolidation. It provides a discussion on methods used in each step of the virtual machine consolidation, a classification of papers according to their contribution, and a quantitative and qualitative analysis of datasets, scenarios, and metrics.

Nowadays cloud computing provides an effective way of implementing infrastructure as a service (IaaS). However virtualized data centers still face many challenges, such as low resource utilization of physical machines (PMs) and imbalanced server loads. Virtual machine (VM) consolidation based on live migration allows administrator to dynamically redeploy VMs into PMs for better resource utilization. Common VM consolidation methods usually focus on one challenge, and pay little attention to others or just ignore them, while effective VM redeployment should make tradeoffs between these challenges, and more importantly, should not let other challenges become worse. On the other hand, since VM live migration leads to performance degradation of applications, consolidation work should control migration cost. In this paper, we provide a manner to comprehensively consider power consumption, load balancing, communication delay and migration cost during VM redeployment. And we formalize VM consolidation as a multiobjective optimization problem, then solve this problem with an improved genetic algorithm. Simulation experiments based on real world workload trace show that compared with single objective optimization approaches our method effectively make tradeoffs between optimized objectives and has better overall performance, which is more practical in real data centers.

Virtual machine (VM) consolidation is necessary for increasing the server utilization; however, it also leads to VM performance degradation. This work presents a method to predict the consolidated VMs performance from the critical system events data. Experiments are designed to demonstrate the effect of system events like interrupts, page faults, mutex operations, and context switching on the consolidated VMs. Results show that the host server counters are not reliable for such predictions. On the other hand, the coupling of the task execution time with the VM system events is an effective way to predict the consolidation performance. Results further show that the VM memory allocation plays an important part in the consolidated tasks performance. The system event data is also used to train an Artificial Neural Network (ANN) for performance prediction on three hypervisors; ESXi, Xen, and XenServer and similar results are observed in all three.

Cloud data centers present a challenge to environmental sustainability because of their significant energy consumption. Additionally, performance degradation resulting from energy management solutions, such as virtual machine (VM) consolidation, impacts service level agreements (SLAs) between cloud service providers and users. Thus, to achieve a balance between efficient energy consumption and avoiding SLA violations, we propose a novel VM consolidation algorithm. Conventional algorithms result in unnecessary migrations when improperly selecting VMs. Therefore, our proposed E2SVM algorithm addresses this issue by selecting VMs with high load fluctuations and minimal resource usage from overloaded servers. These selected VMs are then placed on normally loaded servers, considering their stability index. Moreover, our approach prevents server underutilization by either applying all or no VM migrations. Simulation results demonstrate a 12.9% decrease in maximum energy consumption compared with the minimum migration time VM selection policy. In addition, a 47% reduction in SLA violations was observed when using the medium absolute deviation as the overload detection policy. Therefore, this approach holds promise for real-world data centers because it minimizes energy waste and maintains low SLA violations.

Cloud Computing has emerged as a computing paradigm where services are provided through the internet in recent years. Offering on-demand services has transformed the IT companies' working environment, leading to a linearly increasing trend of its usage. The provisioning of the Computing infrastructure is achieved with the help of virtual machines. A great figure of physical devices is required to satisfy the users' resource requirements. To meet the requirements of the submitted workloads that are usually dynamic, the cloud data centers cause the over-provisioning of cloud resources. The result of this over-provisioning is the resource wastage with an increase in the levels of energy consumption, causing a raised operational cost. High CO2 emissions result from this huge energy consumption by data centers, posing a threat to environmental stability. The environmental concern demands for the controlled energy consumption, which can be attained by optimal usage of resources to achieve in the server load, by minimizing the number of active nodes, and by minimizing the frequency of switching between active and de-active server mode in the data center. Motivated by these actualities, we discuss numerous statistical, deterministic, probabilistic, machine learning and optimization based computational solutions for the cloud computing environment. A comparative analysis of the computational methods, on the basis of architecture, consolidation step involved, objectives achieved, simulators involved and resources utilized, has also been presented. A taxonomy for virtual machine (VM) consolidation has also been derived in this research article followed by emerging challenges and research gaps in the field of VM consolidation in cloud computing environment.