Generalized just-in-time trace compilation using a parallel task farm in a dynamic binary translator

Dynamic binary translation (DBT) is a core technologyto many important applications such as system virtualization, dynamic binary instrumentation, and security. However, there are several factors that often impede its performance: 1) emulation overhead before translation; 2) translation and optimization overhead; and 3) translated code quality. The issues also include its retargetabilitythat supports guest applications from different instruction-set architectures (ISAs) to host machines also with different ISAs-an important feature to system virtualization. In this work, we take advantage of the ubiquitous multicore platforms, and use a multithreaded approach to implement DBT. By running the translator and the dynamic binary optimizer on different cores with different threads, it could off-load the overhead incurred by DBT on the target applications; thus, afford DBT of more sophisticated optimization techniques as well as its retargetability. Using QEMU (a popular retargetable DBT for system virtualization) and Low-Level Virtual Machine (LLVM) as our building blocks, we demonstrated in a multithreaded DBT prototype, called Hybrid-QEMU (HQEMU), that it could improve QEMU performance by a factor of 2.6x and 4.1x on the SPEC CPU2006 integer and floating point benchmarks, respectively, for dynamic translation of x86 code to run on x86-64 platforms. For ARM codes to x86-64 platforms, HQEMU can gain a factor of 2.5x speedup over QEMU for the SPEC CPU2006 integer benchmarks. We also address the performance scalability issue of multithreaded applications across ISAs. We identify two major impediments to performance scalability in QEMU: 1) coarse-grained locks used to protect shared data structures, and 2) inefficient emulation of atomic instructions across ISAs. We proposed two techniques to mitigate those problems: 1) using indirect branch translation caching (IBTC) to avoid frequent accesses to locks, and 2) using lightweight memory transactions to emulate atomic instructions across ISAs. Our experimental results show that for multithread applications, HQEMU achieves 25X speedups over QEMU for the PARSEC benchmarks.

This paper describes the design and implementation of a novel dynamic binary translation technique specialized for embedded systems. Virtual platforms have been widely used to develop embedded software and dynamic binary translation is essential to boost their speed in simulations. However, unlike application simulation, the code generated for systems simulation is still slow because the simulator must replicate all of the functions of the target hardware. Embedded systems, which focus on providing one or a few functions, utilize only a small portion of the processor's features most of the time. For example, they may use a Memory Management Unit (MMU) in a processor to map physical memory to effective addresses, but they may not need paged memory support as in an OS. We can exploit this to specialize the dynamically translated code for more performance. We built a specialization framework on top of a functional simulator with a dynamic binary translator. Using the framework, we implemented three specializers for an MMU, bi-endianness, and register banks. Experiments with the EEMBC1.1 benchmark showed that the speed of the specialized code was up to 39% faster than the unspecialized code.

Static binary translation (SBT) systems and dynamic binary translation (DBT) systems have their own merits and disadvantages. SBT can perform whole-program optimizations and do not incur run-time overheads. However, the code discovery and the code location problems caused by indirect branches make SBT systems hard to develop. On the other hand, DBT can perform optimizations based on program's runtime behaviors and can handle indirect branches easily. However, because the translation time accounts for a part of the execution time, DBT systems cannot perform aggressive optimizations. Therefore, quality of the code generated by DBT is not as good as that by SBT. In this paper, we present a hybrid binary translation (HBT) system which combines the merits of both SBT and DBT. It leverages the LLVM infrastructure to translate source binary code, optimize, and generate target binary code. It first translates binary statically. If a run-time exception happens, the HBT system switches to dynamic translation. On the EEMBC benchmark suite, our experimental result shows that the HBT system can run about 4 to 20 times faster than a LLVM-based DBT system.

A hardware/software co-designed processor transparently supports a ubiquitous ISA (e.g. ×86) with diversified and innovative microarchitectural implementations. It leverages co-designed HW features and dynamic binary translation (DBT) SW to morph existing binary programs to scale performance and save power. On such systems, the portable bytecode of modern dynamic languages (e.g. Java, JavaScript, etc.) is first translated into the code in the architecture ISA by the just-in-time (JIT) compilation in the bytecode virtual machine, and then into the code in the internal implementation ISA by the DBT. This not only incurs the translation overheads twice, but also brings significant emulation inefficiency as the DBT does not have the high level bytecode information. In this paper, we present AccelDroid, which accelerates the Android Dalvik bytecode execution on the HW/SW co-designed processor through direct bytecode translation in the DBT. Our experiments on a HW/SW co-designed Transmeta Efficeon machine show that AccelDroid can improve performance by 78% and save energy by 40% for the CaffeineMark 3.0 benchmark suite.

Transmeta's Efficeon microprocessor is a full, system-level implementation of the x86 architecture, comprising a native VLIW microprocessor with a software layer, the Code Morphing Software (CMS), that combines an interpreter, dynamic binary translator, optimizer, and run-time system. This software layer provides unique opportunities for studying the tradeoffs between interpretation and dynamic translation with various levels of optimization, while running arbitrary PC workloads. This talk presents some results from such a study, and provides insight into what mix of these emulation modes is most desirable over the range of tested workloads.

A Dynamic Binary Translation (DBT) system dynamically translates program binaries built for a guest platform into code for the host machine that runs the program, one basic block at a time. Even after optimizations, auxiliary tasks performed alongside program emulation by the DBT system introduce performance overheads as compared to executing the program on the native guest platform. In this work, we analyze the extent and causes for a DBT system's startup performance latency. We then focus on understanding and alleviating the program translation cost that is a significant contributor to and disproportionately impacts the startup overhead. We propose and assess the potential of a new technique that parallelizes program translations on multi-core machines to reduce its evident run-time costs. We explain the challenges in achieving such parallelization and discuss and evaluate solutions.

This paper describes a just-in-time (JIT) Java1 compiler for the Intel® Itanium® processor. The Itanium processor is an example of an Explicitly Parallel Instruction Computing (EPIC) architecture and thus relies on aggressive and expensive compiler optimizations for performance. Static compilers for Itanium use aggressive global scheduling algorithms to extract instruction-level parallelism. In a JIT compiler, however, the additional overhead of such expensive optimizations may offset any gains from the improved code.In this paper, we describe lightweight code generation techniques for generating efficient Itanium code. Our compiler relies on two basic methods to generate efficient code. First, the compiler uses inexpensive scheduling heuristics to model the Itanium microarchitecture. Second, the compiler uses the semantics of the Java virtual machine to extract instruction-level parallelism.

The ARMv8 architecture introduced AArch64, a 64-bit execution mode with a new instruction set, while retaining binary compatibility with previous versions of the ARM architecture through AArch32, a 32-bit execution mode. Most hardware implementations of ARMv8 processors support both AArch32 and AArch64, which comes at a cost in hardware complexity. We present MAMBO-X64, a dynamic binary translator for Linux which executes 32-bit ARM binaries using only the AArch64 instruction set. We have evaluated the performance of MAMBO-X64 on three existing ARMv8 processors which support both AArch32 and AArch64 instruction sets. The performance was measured by comparing the running time of 32-bit benchmarks running under MAMBO-X64 with the same benchmark running natively. On SPEC CPU2006, we achieve a geometric mean overhead of less than 7.5% on in-order Cortex-A53 processors and a performance improvement of 1% on out-of-order X-Gene 1 processors. MAMBO-X64 achieves such low overhead by novel optimizations to map AArch32 floating-point registers to AArch64 registers dynamically, handle overflowing address calculations efficiently, generate traces that harness hardware return address prediction, and handle operating system signals accurately.

The Java Virtual Machine (JVM) is the cornerstone of Java technology and its efficiency in executing the portable Java bytecodes is crucial for the success of this technology. Interpretation, Just-in-Time (JIT) compilation, and hardware realization are well-known solutions for a JVM and previous research has proposed optimizations for each of these techniques. However, each technique has its pros and cons and may not be uniformly attractive for all hardware platforms. Instead, an understanding of the architectural implications of JVM implementations with real applications can be crucial to the development of enabling technologies for efficient Java runtime system development on a wide range of platforms. Toward this goal, this paper examines architectural issues from both the hardware and JVM implementation perspectives. The paper starts by identifying the important execution characteristics of Java applications from a bytecode perspective. It then explores the potential of a smart JIT compiler strategy that can dynamically interpret or compile based on associated costs and investigates the CPU and cache architectural support that would benefit JVM implementations. We also study the available parallelism during the different execution modes using applications from the SPECjvm98 benchmarks. At the bytecode level, it is observed that less than 5 out of the 256 bytecodes constitute 90 percent of the dynamic bytecode stream. Method sizes fall into a trinodal distribution with peak of 1, 9, and 26 bytecodes across all benchmarks. The architectural issues explored in this study show that, when Java applications are executed with a JIT compiler, selective translation using good heuristics can improve performance, but the saving is only 10-15 percent at best. The instruction and data cache performance of Java applications are seen to be better than that of C/C/sub +/+ applications except in the case of data cache performance in the JIT mode. Write misses resulting from installation of JIT compiler output dominate the misses and deteriorate the data cache performance in JIT mode. A study on the available parallelism shows that Java programs executed using JIT compilers have parallelism comparable to C/C++ programs for small window sizes, but falls behind when the window size is increased. Java programs executed using the interpreter have very little parallelism due to the stack nature of the SVM instruction set, which is dominant in the interpreted execution mode. In addition, this work gives revealing insights and architectural proposals for designing an efficient Java runtime system.

Condition code optimization in dynamic binary translation

One important aspect of making Control-Flow Integrity (CFI) practical is to support modularity. In the context of CFI, modularity refers to the ability to perform instrumentation of modules of an application separately, without considering other modules, and to link instrumented modules into an executable on demand. Modularity support is of crucial importance for accommodating dynamic linking, which loads libraries at runtime and is frequently used in modern software systems since it allows different software vendors to work on or update modules independently. Just-In-Time (JIT) compilation also requires modularity support because a piece of newly generated code by a JIT compiler can be considered a module, which is to be with the JIT compiler. In both dynamic linking and JIT compilation, not all code is available statically. The code of a dynamically linked library is available only after the library has been loaded at runtime, which may happen during the middle of a program execution; in JIT compilation, the binary code of a function is available only after the function has been compiled on the fly. We call this kind of code dynamic code since it is available only after a runtime event. Unfortunately, many CFI systems require all modules of an application, including libraries, to be available at the static instrumentation time. They perform a global analysis on all code to construct a global Control-Flow Graph (CFG). Instrumentation schemes in many CFI systems also assume global code properties. For instance, instrumentation in the classic CFI [Abadi et al. 2005a] inserts identifiers (representing a class of indirect branches and targets) before branch targets and inserts checks before indirect branches to make sure the right identifiers according to the CFG are at the targets. The identifiers are embedded as instructions in code and cannot appear in the rest of the code. However, this property cannot be guaranteed without inspecting the whole program. Many other CFI instrumentation techniques also do not support modularity; the reasons for this are discussed in Section 2.5. In this chapter, we present a modular CFI system that extends CFI to support dynamic code.We start with an overview of the main challenges and solutions when dealing with dynamic code in Section 2.1. In Section 2.2, we present a compilerassisted, type-based scheme that allows efficient CFG construction in the presence of dynamic code. In Section 2.3, we present a modular CFI system that supports dynamic libraries by adopting the type-based CFG construction process and a technique for supporting multi-threading. In Section 2.4, we show that, with some adjustments, the modular CFI system can support JIT compilation. We discuss related work in Section 2.5 and conclude in Section 2.6. The main results of this chapter were published in previous conference papers [Niu and Tan 2014a, Niu and Tan 2014b] and in a Ph.D. dissertation [Niu 2015]. In this chapter, we streamline the presentation by integrating the previous papers and by highlighting the most important ideas and messages that are of interest to future CFI research. Certain details are omitted and interested readers are referred to the previous papers. Furthermore, experimental results included in this chapter may be different from those in the conference publications because we have made improvements since their publication.

We have developed a research compiler for Java class files. The compiler, which we call Briki, is designed to test new compilation techniques. We focus on optimizations which are only possible or which are much easier to perform on a high-level intermediate representation (IR). We have designed such a representation (JavaIR) and have written a front-end which recovers high-level structure from the information from the class file. Some of the high-level optimizations can be performed by the Java compiler which produces the class file. There is, however, a set of machine-dependent optimizations which have to be customized for the specific architecture and so can only be performed when the machine code is generated from the byte codes, e.g. in a just-in-time (JIT) compiler. We choose memory hierarchy optimizations as an example of machine-dependent techniques. We show that there is an intersection of the set of machine-dependent optimizations and the set of high-level optimizations. One such example is array remapping, which requires multi-dimensional array references which are not present in the byte codes and at the same time requires information about the memory organization. We develop a set of optimizations for accessing array elements and object fields and show their impact on set of benchmarks which we run on two machines with a JIT compiler. The execution times are reduced by as much as 50%, and we argue that the improvement could be even higher with a more mature JIT technology.

Reasonable trade-off between the ASIC performance and GPP flexibility is the main objective of reconfigurable computing systems. Dynamic Reconfigurable computing platform using embedded just-in-time (JIT) compilation is the most flexible platform among others. All complex computing kernels can be translated to bitstream to be executed on FPGA using an embedded processor of dedicated specialized hardware. The main challenge in these systems is FPGA design automation time. Executing the CAD algorithms on embedded processor is too time-consuming and normally is not feasible for real applications. Placement is the most computative part of CAD algorithm. Therefore, a new FPGA placement algorithm is proposed in this paper which makes reasonable trade-off between execution time and quality of placement. The proposed algorithm includes two stages: force-directed placement and simulated annealing placement. The proposed algorithm is very low execution time to be useful in JIT compilation without considerable degradation on the quality of placement. Experimental results show $$2.33\times $$2.33A— speedup in execution time in cost of 3Â % overhead in channel tracks numbers.

With predictions that in the near future tens of billions of devices will be added to the Internet of Things (IoT), new challenges arise, including the development and deployment of software for a wide variety of devices, architectures, instruction set architectures, and computational resources. Despite the fact that emulation techniques have been successful in the creation of virtual layers to ease software deployment in heterogeneous settings, the power consumption and memory code footprint restrictions of the IoT lead to new issues which still need to be further explored within the scope of software emulation. Therefore, we investigate how a distributed implementation of dynamic translation can be used to increase emulation performance in this context. To evidence this, we present a novel distributed dynamic translator simulator that can simulate environments with several devices running RISCV applications and offloading code compilation through the network. Using this simulator, we tested a scenario with 400 constrained IoT devices running lean virtual machines that are capable of offloading dynamic translation to four fog devices, which in turn have the option of distributing their workload to other fog devices or the cloud.The results show that we can improve the performance of constrained devices by 10 times with the use of fog-assisted translation, while at the same time guaranteeing no extra overhead in the emulation process of the fog devices themselves. These findings suggest that, in general, distributed dynamic translation can leverage fog computing to improve emulation in the IoT, even when it comes to devices with limited resources.

In recent years, as performance and capacity of main and external memory grow, performance of database management systems (DBMSes) on certain kinds of queries is more determined by raw CPU speed. Currently, PostgreSQL uses the interpreter to execute SQL queries. This yields an overhead caused by indirect calls to handler functions and runtime checks, which could be avoided if the query were compiled into native code "on-the-fly", i.e. just-in-time (JIT) compiled: at run time the specific table structure is known as well as data types and built-in functions used in the query as well as the query itself. This is especially important for complex queries, performance of which is CPU-bound. We’ve developed a PostgreSQL extension that implements SQL query JIT compilation using LLVM compiler infrastructure. In this paper we show how to implement JIT compilation to speed up sequential scan operator (SeqScan) as well as expressions in WHERE clauses. We describe some important optimizations that are possible only with dynamic compilation, such as precomputing tuple attributes offsets only for attributes used by the query. We also discuss the maintainability of our extension, i.e. the automation for translating PostgreSQL backend functions into LLVM IR, using the same source code both for our JIT compiler and the existing interpreter. Currently, with LLVM JIT we achieve up to 5x speedup on synthetic tests as compared to original PostgreSQL interpreter.