Dynamic Memory Pressure Aware Ballooning

In conventional virtualized systems, a hypervisor can access the memory pages of guest virtual machines without any restriction, as the hypervisor has a full control over the address translation mechanism. In this paper, we propose Secure MMU, a hardware-based mechanism to isolate the memory of guest virtual machines from unauthorized accesses even from the hypervisor. The proposed mechanism extends the current nested paging support for virtualization with a small hardware cost. With Secure MMU, the hypervisor can flexibly allocate physical memory pages to virtual machines for resource management, but update nested page tables only through the secure hardware mechanism, which verifies each mapping change. With the hardware-rooted memory isolation among virtual machines, the memory of a virtual machine in cloud computing can be securely protected from a compromised hypervisor or co-tenant virtual machines.

Virtual Infrastructure-as-a-Service (IaaS) clouds are emerging for secondary cloud service providers to manage their own IaaS clouds on top of existing IaaS clouds. In virtual IaaS clouds, guest virtual machines (VMs) run inside cloud VMs provided by existing IaaS clouds. Unlike traditional IaaS clouds, they can be migrated between cloud VMs co-located at the same host. However, the performance of such VM migration is low due to slow virtual networks and doubled system loads. To optimize VM migration between co-located cloud VMs, we propose zero-copy migration for virtual IaaS clouds. Zero-copy migration just relocates the memory image of a guest VM without any copy. To enable live migration with negligible downtime, it first makes the memory of a guest VM share with the destination cloud VM and thereafter completes memory relocation. We have implemented a system called VMBeam for enabling zero-copy migration in Xen. According to our experimental results, zero-copy migration could achieve high migration performance and low system loads.

Obtaining high flexibility to performance-loss ratio is a key challenge of today's HPC virtual environment landscape. And while extensive research has been targeted at extracting more performance from virtual machines, the idea that whether novel virtual machine usage scenarios could lead to high flexibility Vs performance trade-off has received less attention.

With increasing demands on cloud computing, protecting guest virtual machines (VMs) from malicious attackers has become critical to provide secure services. The current cloud security model with software-based virtualization relies on the invulnerability of the software hypervisor and its trustworthy administrator with the root permission. However, compromising the hypervisor with remote attacks or root permission grants the attackers with a full access capability to the memory and context of a guest VM. This paper proposes a HW-based approach to protect guest VMs even under an untrusted hypervisor. With the proposed mechanism, memory isolation is provided by the secure hardware, which is much less vulnerable than the software hypervisor. The proposed mechanism extends the current hardware support for memory virtualization based on nested paging with a small extra hardware cost. The hypervisor can still flexibly allocate physical memory pages to virtual machines for efficient resource management. In addition to the system design for secure virtualization, this paper presents a prototype implementation using system management mode. Although the current system management mode is not intended for security functions and thus limits the performance and complete protection, the prototype implementation proves the feasibility of the proposed design.

This paper presents EXTERIOR, a dual-VM architecture based external shell that can be used for trusted, timely out-of-VM management of guest-OS such as introspection, configuration, and recovery. Inspired by recent advances in virtual machine introspection (VMI), EXTERIOR leverages an isolated, secure virtual machine (SVM) to introspect the kernel state of a guest virtual machine (GVM). However, it goes far beyond the read-only capability of the traditional VMI, and can perform automatic, fine-grained guest-OS writable operations. The key idea of EXTERIOR is to use a dual-VM architecture in which a SVM runs a kernel identical to that of the GVM to create the necessary environment for a running process (e.g., rmmod, kill), and dynamically and transparently redirect and update the memory state at the VMM layer from SVM to GVM, thereby achieving the same effect in terms of kernel state updates of running the same trusted in-VM program inside the shell of GVM. A proof-of-concept EXTERIOR has been implemented. The experimental results show that EXTERIOR can be used for a timely administration of guest-OS, including introspection and (re)configuration of the guest-OS state and timely response of kernel malware intrusions, without any user account in the guest-OS.

Once compromising the hypervisor, remote or local adversaries can easily access other customers' sensitive data in the memory and context of guest virtual machines (VMs). VM isolation is an efficient mechanism for protecting the memory of guest VMs from unauthorized access. However, previous VM isolation systems either modify hardware architecture or introduce a software module without being protected, and most of them focus on the x86 architecture. This paper proposes HA-VMSI, a lightweight hardware-assisted VM isolation approach for ARM, to provide runtime protection of guest VMs, even with a compromised hypervisor. In the ARM TrustZone secure world, a thin security monitor is introduced as HA-VMSI's entire TCB. Hence, the security monitor is much less vulnerable and safe from attacks that can compromise the hypervisor. The key of HA-VMSI is decoupling the functions of memory isolation among VMs from the hypervisor into the security monitor. As a result, the hypervisor can only update the Stage-2 page tables of VMs via the security monitor, which inspects and approves each new mapping. It is worth noting that HA-VMSI is more secure and effective than current software approaches, and more flexible and compatible than hardware approaches. We have implemented a prototype for KVM hypervisor with multiple Linux as guest OSes on Juno board. The security assessment and performance evaluation show that HA-VMSI is effective, efficient and practical.

Cloud computing systems routinely process users' confidential data, but the underlying virtualization software in use today is not constructed to minimize the exposure of such data. For instance, virtual machine (VM) checkpointing can drastically prolong the lifetime and vulnerability of confidential data without users' knowledge by storing such data as part of a persistent snapshot. A key requirement for minimizing the exposure of any data is the ability to cleanly isolate such data for either exclusion or processing. Traditional mechanisms for memory taint tracking are expensive whereas those for isolating application footprint in VM-based sandboxes are not transparent. In this paper, we propose a transparent and lightweight mechanism for isolating a confidential application's memory footprint in a VM. The key idea is for a parent VM to spawn a child VM, called a Privacy-preserving Virtual Machine (PPVM) within which the confidential application executes. Hypervisor features, such as VM checkpointing, that need to exclude the memory of a confidential application can safely ignore the child VM's memory footprint. Alternatively, features such as checkpoint encryption or malware tracking can operate only on the child VM's memory. We implement memory isolation for PPVM through a lightweight VM fork operation that uses copy-on-write to reduce the memory and filesystem overhead of the PPVM. Transparency is achieved through a confidential shell that allows the parent VM to spawn the confidential application in the PPVM and exercise control over it during runtime. We demonstrate the effectiveness of PPVM through its use with VM checkpointing, which can safely checkpoint the parent VM while excluding or encrypting the associated PPVM. We show that our PPVM implementation achieves effective memory isolation with low overheads on memory, CPU, and network performance.

Virtualization, one of the most actively adopted technologies today in computer systems, is increasingly widening its range of applications. As multiple virtual machines are concurrently executed on a physical machine, they compete for physical resources such as CPU, DRAM, HDD and NIC. In large scale data centers or cluster systems, therefore, it is required to balance each physical machine's load by migrating a virtual machine to another physical machine. Since system performance is significantly hindered during migration, a technique that can reduce the migration overheads is required. Interestingly, we notice that virtual machines' memory contain numerous dispensable data. Therefore, in this paper, we propose an efficient virtual machine migration technique called D3 (Discarding Dispensable Data). The basic idea is simple but effective: finding indispensable page frames, and then, transferring only those page frames. In D3, simple metadata is handed over for the dispensable page frames, instead of actual data transmission, and the discarded page frames are reconstructed appropriately after the migration process. One issue is that checking whether a page frame is dispensable or not based on comparing the entire content yields excessive overhead. To overcome this problem, we devise a novel sampling based dispensable page frame detection technique. D3 have implemented in Xen virtualization software and evaluated on a realistic environment with relevant workloads. We show that D3 can efficiently reduce migration time up to 64.2% and size of memory required for completing a migration up to 94.8%. We also present that D3 is very useful to provide a certain level of QoS and better performance.

As more and more virtual machines (VM) are packed into a physical machine, refactoring common kernel components shared by the virtual machines running on the same physical machine significantly reduces the overall resource consumption. A refactored kernel component typically runs on a special VM called a virtual appliance. Because of the semantics gap in Hardware Abstraction Layer (HAL)-based virtualization, a physical machine's virtual appliance requires the support of per-VM in-guest agents to perform VM-specific operations such as kernel data structure access and modification. To simplify deployment, these agents must be injected into guest virtual machines without requiring any manual installation. Moreover, it is essential to protect the integrity of in-guest agents at run time, especially when the underlying refactored kernel service is security-related. This paper describes the design, implementation and evaluation of a surreptitious kernel agent deployment and execution mechanism called SADE that requires zero installation effort and effectively hides the execution of agent code. To demonstrate the efficacy of SADE, we describe a signature-based memory scanning virtual appliance that uses SADE to inject its in-guest kernel agents without any support from the injected virtual machine, and show that both the start-up overhead and the run-time performance penalty of SADE are quite modest in practice.

For the rapid increase in resource requirements in large scale Data Centers (DCs), enterprises have brought hyperconverged architecture where the storage pool is built up by the individual storage components associated with different servers, and it is shared among all the Virtual Machines (VMs) or containers through a common network infrastructure. Due to the sharing of network bandwidth among the application generated network traffic and the storage traffic from shared storage infrastructure, quality of service (QoS) performances of networking and storage intensive applications are affected, which further impacts VM or container migrations with dynamic workload scenarios. In this article, we propose NetStor to ensure QoS for various collocated network and storage intensive workloads over a hyperconverged architecture and ensure QoS during VM or container migration. NetStor uses a workload estimation strategy for VMs and containers, and applies the strategic decisions for resource allocation and migration based on environmental learning and workload characterization. NetStor supports a dynamic QoS provisioning that is workload-agnostic catering to both VMs and containers, and is unique to the best of our knowledge. We have implemented NetStor over a hyperconverged DC architecture on a testbed and found that NetStor can enhance network performance significantly compared to other related mechanisms discussed in the literature.

Advances in virtualization technology have focused mainly on strengthening the isolation barrier between virtual machines (VMs) that are co-resident within a single physical machine. At the same time, a large category of communication intensive distributed applications and software components exist, such as web services, high performance grid applications, transaction processing, and graphics rendering, that often wish to communicate across this isolation barrier with other endpoints on co-resident VMs. State of the art inter-VM communication mechanisms do not adequately address the requirements of such applications. TCP/UDP based network communication tends to perform poorly when used between co-resident VMs, but has the advantage of being transparent to user applications. Other solutions exploit inter-domain shared memory mechanisms to improve communication latency and bandwidth, but require applications or user libraries to be rewritten against customized APIs - something not practical for a large majority of distributed applications. In this paper, we present the design and implementation of a fully transparent and high performance inter-VM network loopback channel, called XenLoop, in the Xen virtual machine environment. XenLoop does not sacrifice user-level transparency and yet achieves high communication performance between co-resident guest VMs. XenLoop intercepts outgoing network packets beneath the network layer and shepherds the packets destined to co-resident VMs through a high-speed inter-VM shared memory channel that bypasses the virtualized network interface. Guest VMs using XenLoop can migrate transparently across machines without disrupting ongoing network communications, and seamlessly switch between the standard network path and the XenLoop channel. In our evaluation using a number of unmodified benchmarks, we observe that XenLoop can reduce the inter-VM round trip latency by up to a factor of 5 and increase bandwidth by a up to a factor of 6.

Dynamic Virtual Machine (VM) consolidation dynamically adapts VM resource allocation to resource demands promising significant cost benefits for highly variable enterprise workloads. In this work, we analyze large enterprise workloads with the goal of understanding how effective are the VM consolidation variants in real world. We observe that burstiness in memory demand is much lower than the burstiness in CPU demand. Further, memory is the more constrained resource in virtualized servers, significantly reducing the potential gains due to dynamic consolidation. We study consolidation planning in four very large data centers and observe that the savings in facilities cost due to dynamic consolidation over static consolidation is not as large as estimated by past studies. Further, the savings over intelligent semi-static consolidation are surprisingly modest in most cases, putting a question mark over the applicability of dynamic consolidation in real world.

Live migration of Virtual Machines (VMs) is a key technique to quickly migrate workloads in response to events such as impending failure or load changes. Despite extensive research, state-of-the-art live migration approaches take a long time to migrate a VM, which in turn negatively impacts the application performance during migration. We present, Quick Eviction, a new approach to significantly speed up the eviction of a VM from the source host with low impact on VM's performance during migration. Before migration, Quick Eviction regularly snapshots the VM's memory to a destination or a failover node. During the actual migration, Quick Eviction has to transfer only a small amount of dirtied memory resulting in a very short time to completely evict the VM out of the source. Our experimental results show that Quick Eviction in the KVM/QEMU platform significantly reduces the eviction time.

Servers have different energy efficiency and energy proportionality (EP) due to their hardware configuration (i.e., CPU generation and memory installation) and workload. However, current virtual machine (VM) scheduling in virtualized environments will saturate servers without considering their energy efficiency and EP differences. This article will discuss EASE, the energy efficiency and proportionality aware VM scheduling approach. EASE first executes customized computing intensive, memory intensive, and hybrid benchmarks to calculate a server's energy efficiency and EP. Then it schedules VMs to servers to keep them working at their peak energy efficiency point (or optimal working range). This step improves the overall energy efficiency of the cluster and the data center. For performance guarantee, EASE migrates VMs from servers under highly contending conditions. The experimental results on real clusters show that power consumption can be saved 37.07% ~ 49.98% in the homogeneous cluster. The average completion time of the computing intensive VMs increases only 0.31 % ~ 8.49%. In the heterogeneous nodes, the power consumption of the computing intensive VMs can be reduced by 44.22 %. The job completion time can be saved by 53.80%.

One key challenge for data owners to host their databases in the cloud is data privacy. In this paper, we first demonstrate that even with the most recent hardware-based security technology such as Intel SGX, a hypervisor can still sniff key database operations running in its guest virtual machine (VM) such as the frequency and type of SQL queries, by monitoring the access pattern of this VM’s main and secondary memory. To ensure security against such access pattern monitoring attacks, we then propose ProDB, a minimal adaptation of a conventional DBMS with both hardware enclave and Oblivious RAM protocol. To enhance its performance for practical use, we also design a SQL-aware Path ORAM protocol called SaP ORAM, which optimizes the classic Path ORAM protocol under practical database workload. Through security analysis and extensive experimental results, we prove and show ProDB achieves high security and throughput on commodity cloud hosting servers.