Draining the Swamp: Micro Virtual Machines as Solid Foundation for Language Development

Safepoints are a virtual machine mechanism that allows one thread to suspend other threads in a known state so that runtime actions can be performed without interruption and with data structures in a consistent state. Many virtual machines use safepoints as a mechanism to provide services such as stop-the-world garbage collection, debugging, and modification to running code such as installing or replacing classes. Languages implemented on these virtual machines may have access to these services, but not directly to the safepoint mechanism itself. We show that safepoints have many useful applications for the implementation of guest languages running on a virtual machine. We describe an API for using safepoints in languages that were implemented under the Truffle language implementation framework on the Java Virtual Machine and show several applications of the API to implement useful guest-language functionality. We present an efficient implementation of this API, when running in combination with the Graal dynamic compiler. We also demonstrate that our safepoints cause zero overhead with respect to peak performance and statistically insignificant overhead with respect to compilation time. We compare this to other techniques that could be used to implement the same functionality and demonstrate the large overhead that they incur.

In large-scale software applications, programmers often combine different programming languages because this allows them to use the most suitable language for a given problem, to gradually migrate existing projects from one language to another, or to reuse existing source code. However, different programming languages have fundamentally different implementations, which are hard to combine. The composition of language implementations often results in complex interfaces between languages, insufficient flexibility, or poor performance. We propose TruffleVM, a virtual machine (VM) that can execute different programming languages and is able to compose them in a seamless way. TruffleVM supports dynamically-typed languages (e.g., JavaScript and Ruby) as well as statically typed low-level languages (e.g., C). It consists of individual language implementations, which translate source code to an intermediate representation that is executed by a shared VM. TruffleVM composes these different language implementations via generic access . Generic access is a language-agnostic mechanism that language implementations use to access foreign data or call foreign functions. It features language-agnostic messages that the TruffleVM resolves to efficient foreign-language-specific operations at runtime. Generic access supports multiple languages, enables an efficient multi-language development, and ensures high performance. We evaluate generic access with two case studies. The first one explains the transparent composition of JavaScript, Ruby, and C. The second one shows an implementation of the C extensions application programming interface (API) for Ruby. We show that generic access guarantees good runtime performance. It avoids conversion or marshalling of foreign objects at the language boundary and allows the dynamic compiler to perform its optimizations across language boundaries.

The Programming Languages (PL) Track provides researchers and practitioners with a forum to present their ideas and experience in designing new programming concepts and implementing programming languages. It includes the topics of Compiling Techniques, Domain-Specific Languages, Formal Semantics and Syntax, Garbage Collection, Language Design and Implementation, Languages for Modeling, Model-Driven Development and Model Transformation, New Programming Language Ideas and Concepts, New Programming Paradigms, Practical Experiences with Programming Languages, Program Analysis and Verification, Program Generation and Transformation, Programming Languages from All Paradigms (Agent-Oriented, Aspect-Oriented, Functional, Logic, Object-Oriented, etc.), and Visual Programming Languages.

chapter 1 - Introducing C# and .NET

Virtual machines raise the abstraction level of the execution environment at the cost of restricting the set of supported languages. Moreover, the ability of a language implementation to integrate with other languages hosted on the same virtual machine typically constrains the features of the language. In this paper, we present a highly flexible yet efficient approach to hosting multiple programming languages on an objectoriented virtual machine. Our approach is based on extending the interface of each class with language-specific wrapper methods, offering each language a tailored view of a given class. This approach can be deployed both on a statically typed virtual machine, such as the JVM, and on a dynamic virtual machine, such as a Smalltalk virtual machine. We have implemented our approach to language interoperability on top of a prototype virtual machine for embedded systems based on the Smalltalk object model, which provides interoperability for embedded versions of the Smalltalk, Java, and BETA programming languages. (Less)

Recently, solid state drives (SSDs) are replacing hard disk drives (HDDs) in datacenter storage systems in order to reduce power consumption and improve I/O performance. Additionally, in order to mitigate the performance bottleneck at I/O interface between host and SSD, the PCIe-leveraging NVMe SSD is emerging for datacenter SSDs. The NVMe interface supports the I/O virtualization mechanism called single root I/O virtualization (SR-IOV), which is a device self-virtualization technique for supporting direct paths from virtual machines (VMs) to I/O devices. Multiple virtual machines can share an SR-IOV-supporting physical device without intervention of virtual machine monitor. SR-IOV-supporting SSD should provide a device-level scheduler which can schedule the requests from multiple VMs considering performance isolation and fairness. In this paper, we propose a workload-aware budget compensation scheduling algorithm for the device-level request scheduler. To guarantee the performance isolation, the device-level scheduler estimates the contribution on the garbage collection (GC) cost of each virtual machine in the SSD device. Based on the estimated GC contributions, the budget of each VM is compensated for performance isolation. We experimented the effects of the proposed technique with an SSD simulator. The experiments showed that the scheduler can guarantee the performance isolation when multiple VMs share an NVMe SSD with different workloads.

Tree-based overlay multicast is an effective group communication method for media streaming applications. However, a group member’s departure causes all of its descendants to be disconnected from the multicast tree for some time, which results in poor performance. The above problem is difficult to be addressed because overlay multicast tree is intrinsically instable. In this paper, we proposed a novel stability enhancing solution, VMCast, for tree-based overlay multicast. This solution uses two types of on-demand cloud virtual machines (VMs), i.e., multicast VMs (MVMs) and compensation VMs (CVMs). MVMs are used to disseminate the multicast data, whereas CVMs are used to offer streaming compensation. The used VMs in the same cloud datacenter constitute a VM cluster. Each VM cluster is responsible for a service domain (VMSD), and each group member belongs to a specific VMSD. The data source delivers the multicast data to MVMs through a reliable path, and MVMs further disseminate the data to group members along domain overlay multicast trees. The above approach structurally improves the stability of the overlay multicast tree. We further utilized CVM-based streaming compensation to enhance the stability of the data distribution in the VMSDs. VMCast can be used as an extension to existing tree-based overlay multicast solutions, to provide better services for media streaming applications. We applied VMCast to two application instances (i.e., HMTP and HCcast). The results show that it can obviously enhance the stability of the data distribution.

Both managed and native languages use memory safety techniques to ensure program correctness and as a security defense. A critical element of memory safety is to initialize newly allocated memory to zero before making it available to the program. In this thesis I explore the performance impact of zero initialization and show that it comes with a substantial overhead. I also show that this overhead can be largely removed with new designs that exploit both the language semantics of zero initialization and the hardware semantics of memory hierarchies Programmers increasingly choose managed languages to develop large scale applications. One of the reasons is that managed languages offer automatic memory management (garbage collection), which protects against memory leaks and dangling pointers. Garbage collection encourages programs to allocate large numbers of small and medium size objects, leading to significant overheads of zero initializing objects — on average the direct cost of zeroing is 4% to 6% and up to 50% of total application time on a variety of modern processors. Zeroing incurs indirect costs as well, which include memory bandwidth consumption and cache displacement. Existing virtual machines (VMs) either: a) minimize direct costs by zeroing in large blocks, or b) minimize indirect costs by integrating zeroing into the allocation sequence to reduce cache displacement. This thesis first describes and evaluates zero initialization costs and the two existing designs. The microarchitectural analysis of prior designs inspires two better designs that exploit concurrency and non-temporal cache-bypassing store instructions to reduce the direct and indirect costs simultaneously. The best strategy is to adaptively choose between the two new designs based on CPU utilization. This approach improves over widely used hot-path zeroing by 3% on average and up to 15% on the newest Intel i7-2600 processor, without slowing down any of the benchmarks. These results indicate that zero initialization is a surprisingly important source of overhead in existing VMs and that our new software strategies are effective at reducing this overhead. These findings also invite other optimizations, including software elision of zeroing and microarchitectural support.

The quality of the scheduling method for virtualization clusters directly affects the performance of the entire cluster. Improper scheduling can even eliminate the advantages of virtualization clusters and reduce the performance of the entire system. Aiming at the problems of low resource utilization and poor performance caused by improper resource scheduling in cluster mode, a comprehensive heuristic resource scheduling method based on virtualization cluster - Resource Type Priority List Scheduling (RTPLS) algorithm is proposed. The entire algorithm includes two stages: resource monitoring and virtual machine scheduling. On the basis of considering multiple threshold triggering conditions such as CPU, memory, and network in the host load judgment during virtual machine scheduling, the host’s resource oscillation granularity and resource weight are introduced to comprehensively evaluate the resource load situation of the entire host, avoiding frequent migration of virtual machines caused by the oscillation of host resource load. In the selection of target virtual machine migration, in order to minimize the number of virtual machine migrations as much as possible, the compatibility coefficient λ of virtual machines is introduced, and the migration is sorted according to the priority list of λ to reduce the server load and energy consumption caused by virtual machine migration with the minimum number of virtual machine migrations. When selecting the target server, in order to minimize the load on the target virtual machine, target server relevance is introduced. Finally, by testing the algorithm in practical environments, it was found that the RTPLS algorithm can significantly improve the efficiency of host load balancing adjustments and enhance resource utilization.

Chapter 21 - Building a VM

Distributed applications for data analytics have emerged to be a significant growth domain both in terms of technology as well as business relevance. With the advent of Hadoop MapReduce, several distributed execution frameworks have been developed for data analytics and machine learning applications. Many of these frameworks are written in Java, Scala or Python and run on top of the Java Virtual Machine (JVM), leaving them vulnerable to the performance impact of garbage collection (GC) operations. Application pauses induced by the Java GC are notorious in being one of the most important contributors of runtime performance bottleneck for a single process execution. These pauses are present in the distributed scenario as well, though in this case, it is significantly more complex to measure the impact of a GC pause for the overall execution runtime. Typically, in a distributed workload, a GC can result in delayed tasks impacting the overall completion time of the application. We present herein a novel way to preemptively trigger Java GC, stealing idle cycles, so as to prevent future tasks from being impacted by a GC pause during peak execution periods, resulting in improved overall application throughput.

High-level language virtual machines (HLL VMs) are now widely used to implement high-level programming languages. To a certain extent, their widespread adoption is due to the software engineering benefits provided by these managed execution environments, for example, garbage collection (GC) and cross-platform portability. Although HLL VMs are widely used, most research has concentrated on high-end optimizations such as dynamic compilation and advanced GC techniques. Few efforts have focused on introducing features that automate or facilitate certain software engineering activities, including software testing. This paper suggests that HLL VMs provide a reasonable basis for building an integrated software testing environment. As a proof-of-concept, we have augmented a Java virtual machine (JVM) to support weak mutation analysis. Our mutation-aware HLL VM capitalizes on the relationship between a program execution and the underlying managed execution environment, thereby speeding up the execution of the program under test and its associated mutants. To provide some evidence of the performance of our implementation, we conducted an experiment to compare the efficiency of our VM-based implementation with a strong mutation testing tool (muJava). Experimental results show that the VM-based implementation achieves speedups of as much as 89% in some cases.

Hundreds of programming languages are in use todayscripting languages for Internet commerce, user interface programming tools, spreadsheet macros, page format specification languages, and many others. Designing a programming language is a metaprogramming activity that bears certain similarities to programming in a regular language, with clarity and simplicity even more important than in ordinary programming. This comprehensive text uses a simple and concise framework to teach key ideas in programming language design and implementation. The book's unique approach is based on a family of syntactically simple pedagogical languages that allow students to explore programming language concepts systematically. It takes as its premise and starting point the idea that when language behaviors become incredibly complex, the description of the behaviors must be incredibly simple. The book presents a set of tools (a mathematical metalanguage, abstract syntax, operational and denotational semantics) and uses it to explore a comprehensive set of programming language design dimensions, including dynamic semantics (naming, state, control, data), static semantics (types, type reconstruction, polymporphism, effects), and pragmatics (compilation, garbage collection). The many examples and exercises offer students opportunities to apply the foundational ideas explained in the text. Specialized topics and code that implements many of the algorithms and compilation methods in the book can be found on the book's Web site, along with such additional material as a section on concurrency and proofs of the theorems in the text. The book is suitable as a text for an introductory graduate or advanced undergraduate programming languages course; it can also serve as a reference for researchers and practitioners.

Our everyday lives are becoming increasingly filled with mobile devices of varying capabilities. The common practice of creating multiple versions of the same application to cope with diverse device resource capabilities increases software development and maintenance costs. We discuss an offloading method to mask out the memory constraints on devices running a typical Java virtual machine. The method allows the garbage collector to selectively offload part of the object heap into a nearby wired server. In comparison with traditional virtual memory techniques, the garbage collector can make wiser offloading choices using information about object access patterns at a finer granularity. Our experiments show that our prototype introduces modest overhead in the JVM while allowing applications to execute on devices without enough physical memory. In addition, when running with the Linux virtual memory system under intense memory constraints, the prototype achieves an average improvement of 24% in run-time performance and 53% in energy savings.

Some programming languages and computer systems use dynamic memory allocation with garbage collection. It would be useful to understand how the utilization of memory depends on the stochastic parameters describing the size and life distributions of the cells. We consider a class of dynamic storage allocation systems which use a first-fit strategy to allocate cells and perform noncompacting garbage collections to recover free memory space when memory becomes fully occupied. A formula is derived for the expected number of holes (available cells) in memory immediately following a garbage collection which specializes to an analogue of Knuth's 'Fifty Percent' rule for nongarbage-collection systems. Simulations confirm the rule for exponentially distributed cell lifetimes. Other lifetime distributions are discussed. The memory-size requirements for noncompacting garbage collection are also analyzed.