DTT

Static binary translation (SBT) systems and dynamic binary translation (DBT) systems have their own merits and disadvantages. SBT can perform whole-program optimizations and do not incur run-time overheads. However, the code discovery and the code location problems caused by indirect branches make SBT systems hard to develop. On the other hand, DBT can perform optimizations based on program's runtime behaviors and can handle indirect branches easily. However, because the translation time accounts for a part of the execution time, DBT systems cannot perform aggressive optimizations. Therefore, quality of the code generated by DBT is not as good as that by SBT. In this paper, we present a hybrid binary translation (HBT) system which combines the merits of both SBT and DBT. It leverages the LLVM infrastructure to translate source binary code, optimize, and generate target binary code. It first translates binary statically. If a run-time exception happens, the HBT system switches to dynamic translation. On the EEMBC benchmark suite, our experimental result shows that the HBT system can run about 4 to 20 times faster than a LLVM-based DBT system.

Dynamic binary translation system must perform an address translation for every execution of indirect branch instructions. The procedure to convert Source binary Program Counter (SPC) address to Translated Program Counter (TPC) address always takes more than 10 instructions, becoming a major source of performance overhead. This paper proposes a novel mechanism called SPc-Indexed REdirecting (SPIRE), which can significantly reduce the indirect branch handling overhead. SPIRE doesn't rely on hash lookup and address mapping table to perform address translation. It reuses the source binary code space to build a SPC-indexed redirecting table. This table can be indexed directly by SPC address without hashing. With SPIRE, the indirect branch can jump to the originally SPC address without address translation. The trampoline residing in the SPC address will redirect the control flow to related code cache. Only 2-6 instructions are needed to handle an indirect branch execution. As part of the source binary would be overwritten, a shadow page mechanism is explored to keep transparency of the corrupt source binary code page. Online profiling is adopted to reduce the memory overhead. We have implemented SPIRE on an x86 to x86 DBT system, and discussed the implementation issues on different guest and host architectures. The experiments show that, compared with hash lookup mechanism, SPIRE can reduce the performance overhead by 36.2% on average, up to 51.4%, while only 5.6% extra memory is needed. SPIRE can cooperate with other indirect branch handling mechanisms easily, and we believe the idea of SPIRE can also be applied on other occasions that need address translation.

A Dynamic Binary Translation (DBT) system dynamically translates program binaries built for a guest platform into code for the host machine that runs the program, one basic block at a time. Even after optimizations, auxiliary tasks performed alongside program emulation by the DBT system introduce performance overheads as compared to executing the program on the native guest platform. In this work, we analyze the extent and causes for a DBT system's startup performance latency. We then focus on understanding and alleviating the program translation cost that is a significant contributor to and disproportionately impacts the startup overhead. We propose and assess the potential of a new technique that parallelizes program translations on multi-core machines to reduce its evident run-time costs. We explain the challenges in achieving such parallelization and discuss and evaluate solutions.

Loop structures in programs have been regarded as a primary source of finding parallelism from sequential codes. In this paper, we present a new technique that dynamically detects precise loop structures with their inter-procedural nests on a dynamic binary translation system. Using precompiled application binary code as an input, our mechanism generates the simple but precise markers when they are loaded from their binary code image, and at runtime monitors loop structures with inter-procedural nesting on the fly using Loop-Call Context Graph. We implement our mechanism and evaluate it using SPEC CPU benchmark suite. The results show that our mechanism reveals precise loop structures with interprocedural loop nesting successfully. The results also show that ours can reduce overheads for loop analysis compared with the existing ones. These indicate that our mechanism can be applied to runtime optimization and parallelization as well as hints for performance tuning.

Indirect branches have historically been a challenge for microarchitectures and code generators alike. The recent steady increase in indirect branch predictability has translated into continual performance improvements especially for Out-of-Order processors which benefit more readily from improvements in branch prediction. In contrast, in-order processors which rely on code generators for performance are still challenged by indirect branches; they are a frequent source of issue stalls and the large number of indirect branch targets and unbiased nature of indirect branches complicate the use of traditional branch handling techniques like assert conversion and predication.

Indirect-branch prediction is becoming more important for modern processors as more programs are written in object-oriented languages. Previous hardware-based indirect-branch predictors generally require significant hardware storage or use aggressive algorithms which make the processor front-end more complex. In this paper, we propose a fast and cost-efficient indirect-branch prediction strategy, called Target Address Pointer (TAP) Prediction. TAP Prediction reuses the history-based branch direction predictor to detect occurrences of indirect branches, and then stores indirect-branch targets in the Branch Target Buffer (BTB). The key idea of TAP Prediction is to predict the Target Address Pointers, which generate virtual addresses to index the targets stored in the BTB, rather than to predict the indirect-branch targets directly. TAP Prediction also reuses the branch direction predictor to construct several small predictors. When fetching an indirect branch, these small predictors work in parallel to generate the target address pointer. Then TAP prediction accesses the BTB to fetch the predicted indirect-branch target using the generated virtual address. This mechanism could achieve time cost comparable to that of dedicated-storage-predictors, without requiring additional large amounts of storage. Our evaluation shows that for three representative direction predictors-Hybrid, Perceptrons, and O-GEHL-TAP schemes improve performance by 18.19%, 21.52%, and 20.59%, respectively, over the baseline processor with the most commonly-used BTB prediction. Compared with previous hardware-based indirect-branch predictors, the TAP-Perceptrons scheme achieves performance improvement equivalent to that provided by a 48KB TTC predictor, and it also outperforms the VPC predictor by 14.02%.

With increasing use of runtime polymorphism and reliance on runtime type interpretation, the presence and importance of indirect branches has seen a considerable rise in recent workloads. Evidently, accurate target prediction for indirect branches has emerged as an important problem. While direction prediction of direct branches has received considerable research attention leading to efficient prediction policies and hardware structures implemented inside modern processors, proposals for target prediction for indirect branches has been relatively few. The problem of accurate target prediction for indirect branches is significantly tough since these transfer control to an address stored in a register that is known only at runtime. Unlike conditional direct branches, indirect branches can have more than two targets to be resolved at runtime, for which prediction requires a full 32-bit / 64-bit address to be predicted, in contrast to just a taken or not-taken decision as needed for direction prediction of direct branches. Recent research shows indirect branches, being mispredicted more frequently, can start to dominate the overall branch misprediction cost. In modern processors, the only hardware structure available to facilitate target address prediction for indirect branches is a fixed-size Branch Target Buffer (BTB). BTB is often designed as a set associative cache for storing recent target addresses for branch instructions encountered during execution, with a motivation of being able to reuse the same addresses for future instances, thereby saving latency cycles. Evidently, designing efficient indexing schemes and replacement mechanisms for BTB structures is crucial, more so, for indirect branches, since these serve as the only prediction handle. In this paper, our objective is to examine a hierarchical BTB design for multicores with total size comparable to what exists today, with a motivation towards possible improvement of prediction accuracy by facilitating collaborative constructive learning between programs executing in different cores that encounter similar histories. Specifically, we wish to propose the concept of a small on-chip L1 BTB inside each core, supported by a larger off-chip L2 BTB shared among the cores. Our motivation towards a hierarchical BTB design is twofold. On one hand, in a multiprogramming environment, where the same program is executed in multiple cores, on different test cases, there is a significant potential of target information reuse for indirect branches, as is evident from our experiments on SPEC 2006 workloads. This is typically useful for machine learning based workloads where the training phase is often executed in different cores with the same neural network being trained on different training sets. On the other hand, with different programs in different cores, there is some chance of reuse as well, due to sharing of system libraries. The essence of our idea rests on the fact that programs executed in different cores have similar history patterns that can similarly influence the target addresses. Our design aims to decrease on the on-chip L1 BTB size while investing more storage for the off-chip shared L2 BTB. Suitable allocation and replacement policies support our hierarchical design. Initial experiments show expected accuracy benefits.

Dynamic binary translation which offers a solution for making an executable compiled on a platform to run on another platform automatically resolves the problem of code migration. The current single-threaded dynamic binary translation systems don't have enough scope to improve the performance. Therefore, the multi-core processors and the multi-threaded program are fully used for getting the high performance. Therefore, based on the performance analysis and experiments results of a single-threaded dynamic binary translation system, this paper shows the framework of MDT (Multi-Threaded Dynamic Binary Translation) system and the optimizations implemented on it. For achieving the speculatively translation scheme, the T-Tree (Translation Tree) is built by the servant thread which gives the direction of getting the next pc. Besides that, integrated with the merit of multi-level Cache scheme, the LRU scheme and the full flush scheme, a new management of T-Cache is presented which proves out managing the translated blocks efficiently. The framework and optimizations of MDT are evaluated wholly and partly across SPEC 2006 under the Alpha multi-core environment. The results compared with QEMU demonstrate that the speculative translation and the reducing of T-Cache missing rate are effectively.

One important aspect of making Control-Flow Integrity (CFI) practical is to support modularity. In the context of CFI, modularity refers to the ability to perform instrumentation of modules of an application separately, without considering other modules, and to link instrumented modules into an executable on demand. Modularity support is of crucial importance for accommodating dynamic linking, which loads libraries at runtime and is frequently used in modern software systems since it allows different software vendors to work on or update modules independently. Just-In-Time (JIT) compilation also requires modularity support because a piece of newly generated code by a JIT compiler can be considered a module, which is to be with the JIT compiler. In both dynamic linking and JIT compilation, not all code is available statically. The code of a dynamically linked library is available only after the library has been loaded at runtime, which may happen during the middle of a program execution; in JIT compilation, the binary code of a function is available only after the function has been compiled on the fly. We call this kind of code dynamic code since it is available only after a runtime event. Unfortunately, many CFI systems require all modules of an application, including libraries, to be available at the static instrumentation time. They perform a global analysis on all code to construct a global Control-Flow Graph (CFG). Instrumentation schemes in many CFI systems also assume global code properties. For instance, instrumentation in the classic CFI [Abadi et al. 2005a] inserts identifiers (representing a class of indirect branches and targets) before branch targets and inserts checks before indirect branches to make sure the right identifiers according to the CFG are at the targets. The identifiers are embedded as instructions in code and cannot appear in the rest of the code. However, this property cannot be guaranteed without inspecting the whole program. Many other CFI instrumentation techniques also do not support modularity; the reasons for this are discussed in Section 2.5. In this chapter, we present a modular CFI system that extends CFI to support dynamic code.We start with an overview of the main challenges and solutions when dealing with dynamic code in Section 2.1. In Section 2.2, we present a compilerassisted, type-based scheme that allows efficient CFG construction in the presence of dynamic code. In Section 2.3, we present a modular CFI system that supports dynamic libraries by adopting the type-based CFG construction process and a technique for supporting multi-threading. In Section 2.4, we show that, with some adjustments, the modular CFI system can support JIT compilation. We discuss related work in Section 2.5 and conclude in Section 2.6. The main results of this chapter were published in previous conference papers [Niu and Tan 2014a, Niu and Tan 2014b] and in a Ph.D. dissertation [Niu 2015]. In this chapter, we streamline the presentation by integrating the previous papers and by highlighting the most important ideas and messages that are of interest to future CFI research. Certain details are omitted and interested readers are referred to the previous papers. Furthermore, experimental results included in this chapter may be different from those in the conference publications because we have made improvements since their publication.

Dynamic binary translation (DBT) is a core technologyto many important applications such as system virtualization, dynamic binary instrumentation, and security. However, there are several factors that often impede its performance: 1) emulation overhead before translation; 2) translation and optimization overhead; and 3) translated code quality. The issues also include its retargetabilitythat supports guest applications from different instruction-set architectures (ISAs) to host machines also with different ISAs-an important feature to system virtualization. In this work, we take advantage of the ubiquitous multicore platforms, and use a multithreaded approach to implement DBT. By running the translator and the dynamic binary optimizer on different cores with different threads, it could off-load the overhead incurred by DBT on the target applications; thus, afford DBT of more sophisticated optimization techniques as well as its retargetability. Using QEMU (a popular retargetable DBT for system virtualization) and Low-Level Virtual Machine (LLVM) as our building blocks, we demonstrated in a multithreaded DBT prototype, called Hybrid-QEMU (HQEMU), that it could improve QEMU performance by a factor of 2.6x and 4.1x on the SPEC CPU2006 integer and floating point benchmarks, respectively, for dynamic translation of x86 code to run on x86-64 platforms. For ARM codes to x86-64 platforms, HQEMU can gain a factor of 2.5x speedup over QEMU for the SPEC CPU2006 integer benchmarks. We also address the performance scalability issue of multithreaded applications across ISAs. We identify two major impediments to performance scalability in QEMU: 1) coarse-grained locks used to protect shared data structures, and 2) inefficient emulation of atomic instructions across ISAs. We proposed two techniques to mitigate those problems: 1) using indirect branch translation caching (IBTC) to avoid frequent accesses to locks, and 2) using lightweight memory transactions to emulate atomic instructions across ISAs. Our experimental results show that for multithread applications, HQEMU achieves 25X speedups over QEMU for the PARSEC benchmarks.

This paper presents a novel approach for dynamic binary translation (DBT) to automatically learn translation rules from guest and host binaries compiled from the same source code. The learned translation rules are then verified via binary symbolic execution and used in an existing DBT system, QEMU, to generate more efficient host binary code. Experimental results on SPEC CINT2006 show that the average time of learning a translation rule is less than two seconds. With the rules learned from a collection of benchmark programs excluding the targeted program itself, an average 1.25X performance speedup over QEMU can be achieved for SPEC CINT2006. Moreover, the translation overhead introduced by this rule-based approach is very small even for short-running workloads.

Both dynamic binary translation systems and optimization systems store the translated or optimized code in the software maintained code cache for reuse. The performance of the code cache is crucial. Translated code is usually organized as code blocks in the code cache and each code block transfer control to the next one through a control transfer instruction. As the target address of a control transfer instruction is in the form of its source program counter, the conventional code cache system has to check the address mapping table for the translated target address to find the required target code block, which will cause considerable performance degradation. Control transfer instructions can be divided into two categories as direct control transfer instructions and indirect control transfer instructions. For indirect control transfer instructions, the target address is hold in the register or memory element whose content can be changed during the execution of the program. It is difficult to chain the indirect control transfer instructions with a fixed translated target address through pure software approaches. A novel indirect control transfer chaining approach is proposed in this paper. The principle of the technique is to insert custom chaining instructions into the translated code block while translating the indirect control transfer instructions and execute those chaining instructions to implement dynamical chaining. Some special hardware and software assists are proposed in this paper. Evaluation of the proposed approach is conducted on a code cache simulator. Experiment results show that our hardware assisted indirect control transfer instruction chaining approach can improve the performance of the code cache system dramatically.Keywordscode cacheindirect control transfer instructiondynamic chainingsimulator

Transmeta's Efficeon microprocessor is a full, system-level implementation of the x86 architecture, comprising a native VLIW microprocessor with a software layer, the Code Morphing Software (CMS), that combines an interpreter, dynamic binary translator, optimizer, and run-time system. This software layer provides unique opportunities for studying the tradeoffs between interpretation and dynamic translation with various levels of optimization, while running arbitrary PC workloads. This talk presents some results from such a study, and provides insight into what mix of these emulation modes is most desirable over the range of tested workloads.

The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware. Tape-outs also offer huge pedagogical value garnered from real hands-on exposure to the whole system stack. However, successful tape-outs demand hard-earned experience, and the design process is time consuming and fraught with challenges. Therefore, custom chips have remained the preserve of a small number of research groups, typically focused on circuit design research. This paper describes the CHIPKIT framework. We describe a reusable SoC subsystem which provides basic IO, an on-chip programmable host, memory and peripherals. This subsystem can be readily extended with new IP blocks to generate custom test chips. We also present an agile RTL development flow, including a code generation tool calledVGEN. Finally, we outline best practices for full-chip validation across the entire design cycle.

A study of the feasibility of realizing a compact, multiple function speech processor is presented in this paper. The processor is required to accommodate either a 2400 bps Linear Predictive Coder, a 9600 bps Adaptive Predictive Coder, or a wireline modem at one of the corresponding data rates. This capability is obtained by designing a speech processing board whose function can be customized to any single task by inserting the appropriate set of PROMs. The concurrent goals of compactness and processor sophistication are achieved by augmenting a digital signal processing integrated circuit chip with external support hardware. An example of such an implementation, which uses the Fujitsu MB8764 as the computation engine, is presented. Because the Fujitsu chip lacks a hardware interrupt and has a limited memory address space, special purpose hardware was designed to implement the interrupt capability externally and to expand both program and data memory using a paging scheme. A detailed study of the design indicates that a compact realization of the multiple function processor can be realized using commercially available hardware and one readily designed custom integrated circuit.