Privacy-preserving Virtual Machine

Virtual Machine (VM) checkpointing enables a user to capture a snapshot of a running VM on persistent storage. VM checkpoints can be used to roll back the VM to a previous good state in order to recover from a VM crash or to undo a previous VM activity. Although VM checkpointing eases systems administration and improves usability, it can also increase the risks of exposing sensitive information. This is because the checkpoint may store VM's physical memory pages that contain confidential information such as clear text passwords, credit card numbers, patients' health records, tax returns, etc.This paper presents the design and implementation of SPARC, a security and privacy aware checkpointing mechanism. SPARC enables users to selectively exclude processes and terminal applications that contain sensitive data from being checkpointed. Selective exclusion is performed by the hypervisor by sanitizing memory pages in the checkpoint file that belong to the excluded applications. We describe the design challenges in effectively tracking and excluding process-specific memory contents from the checkpoint file in a VM running the commodity Linux operating system. Our preliminary results show that SPARC imposes only 1% - 5.3% of overhead if most pages are dirty before checkpointing is performed.

In conventional virtualized systems, a hypervisor can access the memory pages of guest virtual machines without any restriction, as the hypervisor has a full control over the address translation mechanism. In this paper, we propose Secure MMU, a hardware-based mechanism to isolate the memory of guest virtual machines from unauthorized accesses even from the hypervisor. The proposed mechanism extends the current nested paging support for virtualization with a small hardware cost. With Secure MMU, the hypervisor can flexibly allocate physical memory pages to virtual machines for resource management, but update nested page tables only through the secure hardware mechanism, which verifies each mapping change. With the hardware-rooted memory isolation among virtual machines, the memory of a virtual machine in cloud computing can be securely protected from a compromised hypervisor or co-tenant virtual machines.

Virtual machine (VM) level checkpoints bring several advantages which process-level checkpoint implementation can hardly provide: compatibility, transparence, flexibility, and simplicity. However, the size of VM-level checkpoint may be very large and even in the order of gigabytes. This disadvantage causes the VM checkpointing and restart time become very long. To reduce the size of VM checkpoint, this paper proposes a memory exclusion scheme using ballooning mechanism, which omits saving unnecessary free pages in the VM. We implement our prototype in Xen environment. Experimental measurements show our approach can significantly reduce the size of VM checkpoint with minimal runtime overhead, thereby greatly improve the checkpoint performance.

Once compromising the hypervisor, remote or local adversaries can easily access other customers' sensitive data in the memory and context of guest virtual machines (VMs). VM isolation is an efficient mechanism for protecting the memory of guest VMs from unauthorized access. However, previous VM isolation systems either modify hardware architecture or introduce a software module without being protected, and most of them focus on the x86 architecture. This paper proposes HA-VMSI, a lightweight hardware-assisted VM isolation approach for ARM, to provide runtime protection of guest VMs, even with a compromised hypervisor. In the ARM TrustZone secure world, a thin security monitor is introduced as HA-VMSI's entire TCB. Hence, the security monitor is much less vulnerable and safe from attacks that can compromise the hypervisor. The key of HA-VMSI is decoupling the functions of memory isolation among VMs from the hypervisor into the security monitor. As a result, the hypervisor can only update the Stage-2 page tables of VMs via the security monitor, which inspects and approves each new mapping. It is worth noting that HA-VMSI is more secure and effective than current software approaches, and more flexible and compatible than hardware approaches. We have implemented a prototype for KVM hypervisor with multiple Linux as guest OSes on Juno board. The security assessment and performance evaluation show that HA-VMSI is effective, efficient and practical.

Live migration of Virtual Machines (VMs) is a key technique to quickly migrate workloads in response to events such as impending failure or load changes. Despite extensive research, state-of-the-art live migration approaches take a long time to migrate a VM, which in turn negatively impacts the application performance during migration. We present, Quick Eviction, a new approach to significantly speed up the eviction of a VM from the source host with low impact on VM's performance during migration. Before migration, Quick Eviction regularly snapshots the VM's memory to a destination or a failover node. During the actual migration, Quick Eviction has to transfer only a small amount of dirtied memory resulting in a very short time to completely evict the VM out of the source. Our experimental results show that Quick Eviction in the KVM/QEMU platform significantly reduces the eviction time.

Hardware virtualization is a major component of large scale server and data center deployments due to their facilitation of server consolidation and scalability. Virtualization, however, comes at a high cost in terms of system main memory utilization. Current virtual machine (VM) memory management solutions impose a high performance penalty and are oblivious to the operating regime of the system. Therefore, there is a great need for low-impact VM memory management techniques which are aware of and reactive to current system state, to drive down the overheads of virtualization. We observe that the host machine operates under different memory pressure regimes, as the memory demand from guest VMs changes dynamically at runtime. Adapting to this runtime system state is critical to reduce the performance cost of VM memory management. In this paper, we propose a novel dynamic memory management policy called Memory Pressure Aware (MPA) ballooning. MPA ballooning dynamically allocates memory resources to each VM based on the current memory pressure regime. Moreover, MPA ballooning proactively reacts and adapts to sudden changes in memory demand from guest VMs. MPA ballooning requires neither additional hardware support, nor incurs extra minor page faults in its memory pressure estimation. We show that MPA ballooning provides an 13.2% geomean speed-up versus the current ballooning techniques across a set of application mixes running in guest VMs; often yielding performance nearly identical to that of a non-memory constrained system.

Checkpoint-recovery based Virtual Machine (VM) replication is an emerging approach towards accommodating VM installations with high availability. However, it comes with the price of significant performance degradation of the application executed in the VM due to the large amount of state that needs to be synchronized between the primary and the backup machines. It is therefore critical to find new ways for attaining good performance, and at the same time, maintaining fault tolerant execution. In this paper, we present a novel approach to improve the performance of services deployed over replicated virtual machines by exploiting data similarity within the VM's memory image to reduce the network traffic during synchronization. For identifying similar memory areas, we propose a bit density based hash function, upon which, we build a content addressable hash table. We present a quantitative analysis on the degree of similarity we found in various workloads, and introduce a lightweight compression method, which, compared to existing replication techniques, reduces network traffic by up to 80% and yields a performance improvement over 90% for certain latency sensitive applications.

Virtual machine technology and the ease with which VMs can be migrated within the LAN, has changed the scope of resource management from allocating resources on a single server to manipulating pools of resources within a data center. We expect WAN migration of virtual machines to likewise transform the scope of provisioning compute resources from a single data center to multiple data centers spread across the country or around the world. In this paper we present the CloudNet architecure as a cloud framework consisting of cloud computing platforms linked with a VPN based network infrastructure to provide seamless and secure connectivity between enterprise and cloud data center sites. To realize our vision of efficiently pooling geographically distributed data center resources, CloudNet provides optimized support for live WAN migration of virtual machines. Specifically, we present a set of optimizations that minimize the cost of transferring storage and virtual machine memory during migrations over low bandwidth and high latency Internet links. We evaluate our system on an operational cloud platform distributed across the continental US. During simultaneous migrations of four VMs between data centers in Texas and Illinois, CloudNet's optimizations reduce memory migration time by 65% and lower bandwidth consumption for the storage and memory transfer by 19GB, a 50% reduction.

Workload hotspot detection is a key component of virtual machine (VM) management in virtualized environment. One of its challenges is how to effectively collect the resource usage of VMs. Also, since data centers usually have hundreds or even thousands of nodes, workload hotspot detection must be able to handle a large amount of monitoring data. In this paper, we address these two challenges. We first present a novel approach to VM memory monitoring. This approach collects memory usage data by walking through the page tables of VMs and by checking the present bit of page table entry. Second, we present a MapReduce-based approach to efficiently analyze a large amount of resource usage data of VMs and nodes. Leveraging the power of parallelism and robustness of MapReduce can significantly accelerate the detection of hotspots. Extensive simulations have been performed to evaluate the proposed approaches. The simulation results show that our approach can achieve effective estimation of memory usage with low overhead and can quickly detect workload hotspots.

Virtualization is fundamental to cloud computing because it allows multiple operating systems to run simultaneously on a physical machine. However, it also brings a range of security/privacy problems. One particularly challenging and important problem is: how can we protect the Virtual Machines (VMs) from being attacked by Virtual Machine Monitors (VMMs) and/or by the cloud vendors when they are not trusted? In this paper, we propose an architectural solution to the above problem in multi-processor cloud environments. Our key idea is to exploit hardware mechanisms to enforce access control over the shared resources (e.g., memory spaces), while protecting VM memory integrity as well as inter-processor communications and data sharing. We evaluate the solution using full-system emulation and cycle-based architecture models. Experiments based on 20 benchmark applications show that the performance overhead is 1.5%--10% when access control is enforced, and 9%--19% when VM memory is encrypted.

The workload of a data center has the characteristics of complexity and requirement variability. However, in reality, the attributes of network workloads are rarely used by resource schedulers. Failure to dynamically schedule network resources according to workload changes inevitably leads to the inability to achieve optimal throughput and performance when allocating network resources. Therefore, there is an urgent need to design a scheduling framework that can be workload-aware and allocate network resources on demand based on network I/O virtualization. However, in the current mainstream I/O virtualization methods, there is no way to provide workload-aware functions while meeting the performance requirements of virtual machines (VMs). Therefore, we propose a method that can dynamically sense the VM workload to allocate network resources on demand, and can ensure the scalability of the VM while improving the performance of the system. We combine the advantages of I/O para-virtualization and SR-IOV technology, and use a limited number of virtual functions (VFs) to ensure the performance of network-intensive VMs, thereby improving the overall network performance of the system. For non-network-intensive VMs, the scalability of the system is guaranteed by using para-virtualized Network Interface Cards (NICs) which are not limited in number. Furthermore, to be able to allocate the corresponding bandwidth according to the VM’s network workload, we hierarchically divide the VF’s network bandwidth, and dynamically switch between VF and para-virtualized NICs through the active backup strategy of Bonding Drive and ACPI Hotplug technology to ensure the dynamic allocation of VF. Experiments show that the allocation framework can effectively improve system network performance, in which the average request delay can be reduced by more than 26%, and the system bandwidth throughput rate can be improved by about 5%.

Virtual machine (VM) consolidation is necessary for increasing the server utilization; however, it also leads to VM performance degradation. This work presents a method to predict the consolidated VMs performance from the critical system events data. Experiments are designed to demonstrate the effect of system events like interrupts, page faults, mutex operations, and context switching on the consolidated VMs. Results show that the host server counters are not reliable for such predictions. On the other hand, the coupling of the task execution time with the VM system events is an effective way to predict the consolidation performance. Results further show that the VM memory allocation plays an important part in the consolidated tasks performance. The system event data is also used to train an Artificial Neural Network (ANN) for performance prediction on three hypervisors; ESXi, Xen, and XenServer and similar results are observed in all three.

As server consolidations based on the virtualization techniques become popular and cloud services continue to grow rapidly, more and more data centers are being built to accommodate a number of virtual clusters running various workloads. Since these virtual clusters often share the resources provided by physical machines (PMs), it is more likely that the interferences between virtual machines (VMs) affect the performance of applications running on top of the virtual clusters. While a lot of studies have proposed different virtual machine placement algorithms to investigate this issue, the problem caused by network performance variability still remains as a challenging issue. Since they usually ignore the CPU overhead to process the communications between VMs, the network bandwidth allocated to a VM cannot be fully utilized when a PM has not enough CPU resources to cover the CPU overhead for VM networking functions. This results in unpredictable application performance running on the virtual clusters. This paper proposes a virtual machine placement algorithm that considers the CPU overhead incurred to reserve network bandwidth in a virtualized data center environment. In order to decide the CPU overhead necessary to guarantee the network bandwidth allocated to a VM, a performance model based on standard linear regression using the data collected from a real environment is used. By comparing the amount of CPU resource available in the driver domain with the CPU overhead obtained from the performance model, the proposed algorithm decides whether the network bandwidth for the VM can be provided or not and selects an appropriate location for VM placement. The benchmarking results show that the proposed algorithm guarantees the network bandwidth allocated to each VM without violations when the CPU resources are shared by multiple VMs.

Since a virtual machine independently uses its own caching policy, redundant disk operations exacerbate the I/O virtualization overhead when virtual machines access large amounts of data on shared storage. This paper presents XHive, an efficient cooperative caching system that is implemented at the virtualization layer, for consolidated environments. Our proposed scheme globally manages buffer caches of consolidated virtual machines in order to accommodate a shared working set in machine memory. A singlet, which is a block cached solely by a virtual machine, is preferentially given more chances to be cached in machine memory by XHive, when it is evicted by a guest operating system. For efficient use of limited memory, singlets are cached in memory that is collaboratively donated from idle memory of virtual machines. Our evaluation shows that XHive significantly reduces disk I/O operations for shared working sets, thereby achieving high read performance and scalability. Improved scalability enables a high degree of workload consolidation with respect to virtual machines that have shared working sets.

As more and more data centers embrace end host virtualization and virtual machine (VM) mobility becomes commonplace, we explore its implications on data center networks. Live VM migrations are considered expensive operations because of the additional network traffic they generate, which can impact the network performance of other applications in the network, and because of the downtime that applications running on a migrating VM may experience. Most virtualization vendors currently recommend a separate network for VM mobility. However, setting up an alternate network just for VM migrations can be extremely costly and thus presents a barrier to seamless VM mobility. Therefore, it is apparent that VM migrations should be orchestrated in a network-aware manner with appropriate QoS controls such that they do not degrade network performance of other flows in the network while still being allocated the bandwidth they require for successful completion within the specified time lines. In this context, we present VMPatrol - a QoS framework for VM migrations. VMPatrol uses a cost of migration model to allocate a minimal bandwidth for a migration flow such that it completes within the specified time limit while causing minimal interference to other flows in the network. Our implementation and experimental evaluation of VMPatrol on real and virtual software testbeds demonstrates that automated bandwidth reservation can reduce the impact of migrations on other flows in the network to a negligible level.