ENTICE VM Image Analysis and Optimised Fragmentation

A virtual machine (VM) has been serving as a crucial component in cloud computing with its rich set of convenient features. The high overhead of a VM has been well addressed by hardware support such as Intel virtualization technology (VT), and by improvement in recent hypervisor implementation such as Xen, KVM, etc. However, the high demand on VM image storage remains a challenging problem. Existing systems have made efforts to reduce VM image storage consumption by means of deduplication within a storage area network (SAN) cluster. Nevertheless, an SAN cannot satisfy the increasing demand of large-scale VM hosting for cloud computing because of its cost limitation. In this paper, we propose Liquid, a scalable deduplication file system that has been particularly designed for large-scale VM deployment. Its design provides fast VM deployment with peer-to-peer (P2P) data transfer and low storage consumption by means of deduplication on VM images. It also provides a comprehensive set of storage features including instant cloning for VM images, on-demand fetching through a network, and caching with local disks by copy-on-read techniques. Experiments show that Liquid's features perform well and introduce minor performance overhead.

Virtual machine (VM) management is one of the key problems in cloud data centers. Many efforts have been put on finding an (approximate) optimal placement of VMs at physical machines (PMs) to achieve high resource utility. In this paper, we consider to reduce the startup delay of VMs, which is a key issue for improving user experience and resource utility. The basic idea of our work is to cache VM images at PMs so as to reduce the time of VM image retrieval. By handling VM placement and image placement jointly and de-centrally, we design a joint placement algorithm and an associated layered transport VM selection algorithm, so as to start VMs quickly in the meanwhile save some calculation resources. The key point of our algorithms lies in how to accelerate the startup of VM with limited information in a distributed fat-tree network. The proposed algorithms are evaluated by extensive simulations using a distributed system simulator SimGrid. The results show that our algorithms just lost a little performance in most cases, compared with the centralized algorithm and a classical cache placement strategy.

Infrastructure-as-a-service (IaaS) Clouds concurrently accommodate diverse sets of user requests, requiring an efficient strategy for storing and retrieving virtual machine images (VMIs) at a large scale. The VMI storage management require dealing with multiple VMIs, typically in the magnitude of gigabytes, which entails VMI sprawl issues hindering the elastic resource management and provisioning. Nevertheless, existing techniques to facilitate VMI management overlook VMI semantics (i.e at the level of base image and software packages) with either restricted possibility to identify and extract reusable functionalities or with higher VMI publish and retrieval overheads. In this paper, we design, implement and evaluate Expelliarmus, a novel VMI management system that helps to minimize storage, publish and retrieval overheads. To achieve this goal, Expelliarmus incorporates three complementary features. First, it makes use of VMIs modelled as semantic graphs to expedite the similarity computation between multiple VMIs. Second, Expelliarmus provides a semantic aware VMI decomposition and base image selection to extract and store non-redundant base image and software packages. Third, Expelliarmus can also assemble VMIs based on the required software packages upon user request. We evaluate Expelliarmus through a representative set of synthetic Cloud VMIs on the real test-bed. Experimental results show that our semantic-centric approach is able to optimize repository size by 2.2 - 16 times compared to state-of-the-art systems (e.g. IBM's Mirage and Hemera) with significant VMI publish and slight retrieval performance improvement.

Virtual Machine Image (VMI) provisioning is an important process of Infrastructure as a Service delivery model to provide virtual images in Cloud Computing. The power consumption and energy efficiency of VMI provisioning process depend not only on the hardware infrastructure, but also on the VMI\,'s configuration, which helps to compose, configure and deploy VMIs in Cloud Computing environments. The major issue of improving the energy efficiency of VMI provisioning process is how to reduce the power consumption while ensuring the compatibility of software components installed in a virtual machine image. This paper describes a model-driven approach to improve the energy efficiency of VMI provisioning in Cloud Computing. This approach considers virtual images as product lines and uses feature models to represent their configurations. It uses model-based techniques to handle VMI specialization, automatic deployment and reconfiguration. The approach aims at minimizing the amount of unneeded software installed in VMIs, and thus to reduce the power consumption of VMI provisioning as well as the data transfer through the network.

Deduplication is widely adopted in virtual machine(VM) backup to save storage space. However, the deduplication storage could cause serious fragmentation, which severely affects the performance of restoring VMs. Current studies mainly focus on backups from a single data source, whereas the backup of VM images is usually a group of behaviors. Exploiting the block reference helps to defragment deduplication storage. In this paper, we propose RefCRE, a framework for accelerating VM image restoration. The framework implements two reference count based methods (RCR and ATL) to reduce the dispersion degree of blocks and reduce the replacement frequency. Experimental results show that the framework can efficiently reduce the dispersion degree of blocks and effectively improve the restoration performance, when the cache size is properly set.

Hypervisor-enabled virtualization is a promising technology to deploy applications and implement Virtual Machine (VM) migration on cloud data centers. However, one critical aspect in hypervisor-based solution is that to deploy applications each VM is installed with a dedicated guest OS along with the requirement of binaries and library files and substantially large VM's image size. Therefore, it is a significant concern for data center administrators because it causes performance overheads. To solve these issues, a new paradigm of virtualization technology that has attracted considerable attention is containerization. Although, progress has been made to address container-based virtualization, there has been less attention to explore the issue of performance evaluation of container and VM in a migration environment. The main contribution of this research study is to develop an experimental setup to implement and evaluate the performance of LXD/CR container-based migration technique and compared it with the pre-copy VM migration scheme. To conduct this performance evaluation, first we modified Linux kernel and implemented LXD which uses liblxc to launch container, then migration of the running container is facilitated using a checkpoint/restore mechanism of CRIU technique. To ensure the applicability and validity for real-time cloud infrastructure, we executed a wide variety of benchmarks such as web server, CPU-intensive, disk I/O and database. From the results obtained, it is found that compared to pre-copy VM migration scheme, LXD/CR container migration technique reduces the downtime, migration time, amount of data transferred and the number of pages transferred by 75.66%, 65.55%, 76.63%, and 76.78%, respectively. In addition, the migration overhead caused is reduced by 55.89% w.r.t. CPU utilization and 76.52% w.r.t. RAM utilization. This being an important contribution for migrating the running applications efficiently and consequently increase the business value of the cloud infrastructure providers.

Cloud computing centers face the key challenge of provisioning diverse virtual machine instances in an elastic and scalable manner. To address this challenge, we have performed an analysis of VM instance traces collected at six production data centers during four months. One key finding is that the number of instances created from the same VM image is relatively small at a given time and thus conventional file-based p2p sharing approaches may not be effective. Based on the understanding that different VM image files often have many common chunks of data, we propose a chunk-level Virtual machine image Distribution Network (VDN). Our distribution scheme takes advantage of the hierarchical network topology of data centers to reduce the VM instance provisioning time and also to minimize the overhead of maintaining chunk location information. Evaluation shows that VDN achieves as much as 30-80× speed up for large VM images under heavy traffic.

Cloud computing is challenging conventional wisdom and practices in the management of IT resources and capabilities; at the same time, the technical underpinnings of cloud computing offer unique opportunities and levers for addressing these challenges. In this talk, I will talk about a number of new research thrusts brought forth by the cloud computing paradigm. First, the large magnitude of a typical virtual machine (VM) image and the large volume of common VM libraries have major performance implications for many cloud operations. Fortunately, content similarity that often exists across distinct VM images may be exploited to optimize the transfer and storage of VM images. Second, resource virtualization in the cloud is at odds with the assumptions in traditional software license models, which must be adapted or extended for virtualized environments. There is also a critical need for new license management techniques to help cloud-based software users achieve cost-effectiveness and compliance. Third, cloud infrastructure is evolving towards heterogeneity and is becoming programmable and composable. The so-called Software-Defined Infrastructure promises agility, flexibility, efficiency, and resilience through continuous and proactive optimization of heterogeneous resources to meet the changing requirements of a wide variety of workloads. For each of these research thrusts, I will discuss the specific technical issues and some initial work conducted at IBM Research.

Infrastructure as a Service (IaaS) cloud has been attracting more and more customers as it provides the highest level of flexibility by offering configurable virtual machines (VMs) and computing infrastructures. Public VM images are usually available for customers to customize and launch. However, the 1 to N mapping between VM images and running instances in IaaS makes vulnerabilities propagate rapidly across the entire public cloud. Besides, IaaS cloud naturally comes with a larger and more stable attack surface and more concentrated target resources than traditional surroundings. In this paper, we first identify the threat of exploiting prevalent vulnerabilities over public IaaS cloud with an empirical study in Amazon EC2. We find that attackers can compromise a considerable number of VMs with trivial cost. We then do a qualitative cost-effectiveness analysis of this threat. Our main result is a two-fold observation: in IaaS cloud, exploiting prevalent vulnerabilities is much more cost-effective than traditional in-house computing environment, therefore attackers have stronger incentive; Fortunately, on the other hand, cloud defenders (cloud providers and customers) also have much lower cost-loss ratio than in traditional environment, therefore they can be more effective for defending attacks. We then build a game-theoretic model and conduct a risk-gain analysis to compare exploiting and patching strategies under cloud and traditional computing environments. Our modeling indicates that under cloud environment, both attack and defense become less cost-effective as time goes by, and the earlier actioner can be more rewarding. We propose countermeasures against such threat in order to bridge the gap between current security situation and defending mechanisms. To our best knowledge, we are the first to analyze and model the threat with prevalent known-vulnerabilities in public cloud.

Grid computing provides mechanisms for making large-scale computing environments available to the masses. In recent times, with the advent of Cloud computing, the concepts of Software as a Service (SaaS), where vendors provide key software products as services over the internet that can be accessed by users to perform complex tasks, and Service as Software (SaS), where customizable and repeatable services are packaged as software products that dynamically meet the demands of individual users, have become increasingly popular. Both SaaS and SaS models are highly applicable to scientific software and users alike. Opal2 is a toolkit for wrapping scientific applications as Web services on Grid and cloud computing resources. It provides a mechanism for scientific application developers to expose the functionality of their codes via simple Web service APIs, abstracting out the details of the back-end infrastructure. Services may be combined via customized workflows for specific research areas and distributed as virtual machine images. In this paper, we describe the overall philosophy and architecture of the Opal2 framework, including its new plug-in architecture and data handling capabilities. We analyze its performance in typical cluster and Grid settings, and in a cloud computing environment within virtual machines, using Amazon's Elastic Computing Cloud (EC2).

14 - DedupCloud: an optimized efficient virtual machine deduplication algorithm in cloud computing environment

Checkpoint-recovery based Virtual Machine (VM) replication is an emerging approach towards accommodating VM installations with high availability. However, it comes with the price of significant performance degradation of the application executed in the VM due to the large amount of state that needs to be synchronized between the primary and the backup machines. It is therefore critical to find new ways for attaining good performance, and at the same time, maintaining fault tolerant execution. In this paper, we present a novel approach to improve the performance of services deployed over replicated virtual machines by exploiting data similarity within the VM's memory image to reduce the network traffic during synchronization. For identifying similar memory areas, we propose a bit density based hash function, upon which, we build a content addressable hash table. We present a quantitative analysis on the degree of similarity we found in various workloads, and introduce a lightweight compression method, which, compared to existing replication techniques, reduces network traffic by up to 80% and yields a performance improvement over 90% for certain latency sensitive applications.

This demo presents a Machine-to-Machine (M2M) service platform running on an OpenStack cloud environment. Access to this platform is secured with authentication based on 3GPP standardized Generic Bootstrapping Architecture (GBA). The M2M network in this testbed consists of Raspberry Pi gateways and resource-constrained devices such as Arduinos. These devices have associated sensors, that report physical data such as temperature, and actuators such as LEDs, that respond to actuation commands. In this demo, the sensor data is authenticated and integrity protected end-to-end with public-key elliptic curve digital signature algorithm (ECDSA). Lastly, the demo testbed includes a web-based cloud management interface referred to as the IoT Portal. This portal allows a system administrator to specify pre-configured Virtual Machine (VM) images with custom applications. As an example, a M2M Web service is installed into the VMs. This web service collects and displays authenticated and verified sensor data and can also be used to send actuator commands.

The success of cloud computing leads to large, centralized collections of virtual machine (VM) images. The ability to retrospect (examine the historical state of) these images at a high semantic level can be valuable in many aspects of IT management such as debugging and troubleshooting, software quality control, legal establishment of data or code provenance, and cyber forensics such as malware tracking and licensing violations. In this paper, we explore the privacy implications of VM retrospection. We argue that retrospection will worsen current concerns about privacy in cloud computing. We develop privacy-sensitive requirements for the design of a retrospection mechanism, and then show how they can be met in a functional prototype.

With the recent rapid development of cloud computing technology, how to reduce the costs of a cloud data center effectively has become an important issue. The study on virtual machine deployment mainly aims at deploying virtual machine resources required by users on a physical server rationally and effectively. This paper proposes a multi-population ant colony algorithm to solve problems of virtual machine deployment. With resource wastage and energy consumption as optimization objectives, this algorithm uses multiple ant colonies for the solution and determines strategies for information exchange among ant colonies according to the information entropy of each population to guarantee the balance of its convergence and diversity. The simulation results show that this algorithm has better performance than the single-population ant colony algorithm and can reduce resource wastage and energy consumption effectively for high-demand virtual machine deployment.