Memory access optimization of dynamic binary translation for reconfigurable architectures

Static binary translation (SBT) systems and dynamic binary translation (DBT) systems have their own merits and disadvantages. SBT can perform whole-program optimizations and do not incur run-time overheads. However, the code discovery and the code location problems caused by indirect branches make SBT systems hard to develop. On the other hand, DBT can perform optimizations based on program's runtime behaviors and can handle indirect branches easily. However, because the translation time accounts for a part of the execution time, DBT systems cannot perform aggressive optimizations. Therefore, quality of the code generated by DBT is not as good as that by SBT. In this paper, we present a hybrid binary translation (HBT) system which combines the merits of both SBT and DBT. It leverages the LLVM infrastructure to translate source binary code, optimize, and generate target binary code. It first translates binary statically. If a run-time exception happens, the HBT system switches to dynamic translation. On the EEMBC benchmark suite, our experimental result shows that the HBT system can run about 4 to 20 times faster than a LLVM-based DBT system.

Exploiting runtime memory access traces can be a complementary approach to compiler optimizations for the energy reduction in memory hierarchy. This is particularly important for emerging multimedia applications since they usually have input-sensitive runtime behavior which results in dynamic and/or irregular memory access patterns. These types of applications are normally hard to optimize by static compiler optimizations. The reason is that their behavior stays unknown until runtime and may even change during computation. To tackle this problem, we propose an integrated approach of software [compiler and operating system (OS)] and hardware (data access record table) techniques to exploit data reusability of multimedia applications in Multiprocessor Systems on Chip. Guided by compiler analysis for generating scratch pad data layouts and hardware components for tracking dynamic memory accesses, the scratch pad data layout adapts to an input data pattern with the help of a runtime scratch pad memory manager incorporated in the OS. The runtime data placement strategy presented in this paper provides efficient scratch pad utilization for the dynamic applications. The goal is to minimize the amount of accesses to the main memory over the entire runtime of the system, which leads to a reduction in the energy consumption of the system. Our experimental results show that our approach is able to significantly improve the energy consumption of multimedia applications with dynamic memory access behavior over an existing compiler technique and an alternative hardware technique.

Dynamic binary translation (DBT) is a core technologyto many important applications such as system virtualization, dynamic binary instrumentation, and security. However, there are several factors that often impede its performance: 1) emulation overhead before translation; 2) translation and optimization overhead; and 3) translated code quality. The issues also include its retargetabilitythat supports guest applications from different instruction-set architectures (ISAs) to host machines also with different ISAs-an important feature to system virtualization. In this work, we take advantage of the ubiquitous multicore platforms, and use a multithreaded approach to implement DBT. By running the translator and the dynamic binary optimizer on different cores with different threads, it could off-load the overhead incurred by DBT on the target applications; thus, afford DBT of more sophisticated optimization techniques as well as its retargetability. Using QEMU (a popular retargetable DBT for system virtualization) and Low-Level Virtual Machine (LLVM) as our building blocks, we demonstrated in a multithreaded DBT prototype, called Hybrid-QEMU (HQEMU), that it could improve QEMU performance by a factor of 2.6x and 4.1x on the SPEC CPU2006 integer and floating point benchmarks, respectively, for dynamic translation of x86 code to run on x86-64 platforms. For ARM codes to x86-64 platforms, HQEMU can gain a factor of 2.5x speedup over QEMU for the SPEC CPU2006 integer benchmarks. We also address the performance scalability issue of multithreaded applications across ISAs. We identify two major impediments to performance scalability in QEMU: 1) coarse-grained locks used to protect shared data structures, and 2) inefficient emulation of atomic instructions across ISAs. We proposed two techniques to mitigate those problems: 1) using indirect branch translation caching (IBTC) to avoid frequent accesses to locks, and 2) using lightweight memory transactions to emulate atomic instructions across ISAs. Our experimental results show that for multithread applications, HQEMU achieves 25X speedups over QEMU for the PARSEC benchmarks.

HW/SW Co-designed systems rely on dynamic binary translation and optimizations for efficient execution of binary code. Due to memory ordering properties and other architectural constraints, most binary optimizations are applied to regions of code that are atomically executed. To ensure that the underlying hardware has enough speculative resources to execute the whole atomic region, these systems typically form short atomic regions, with only 20 to 30 instructions. However, the shorter is the atomic region the smaller is the scope for optimizations. We present LAR-CC, a novel technique that enables HW/SW co-designed systems to optimize large atomic regions and dynamically fit them into the available speculative hardware resources by means of conditional commits. The LAR-CC technique consists of two major components: 1) conditional branch instructions to conditionally skip commit operations; 2) code transformations that replace commit operations by conditional commits and enable optimizations to be applied on the large atomic regions. Our experiments show that LAR-CC can effectively achieve dynamic atomic region sizes larger than 1000 instructions, providing sufficiently large scope to apply many advanced optimizations on HW/SW co-designed systems.

Transactional memory (TM) has been shown to be a promising programming model for multi-core systems. We developed a Software-based Transactional Memory (STM) compiler that generates efficient transactional code for transactions to run on a STM runtime without the need of transactional hardware support. Since real-world applications often invoke third party libraries available only in binary form, it is imperative for our STM compiler to support legacy binary functions and provide an efficient solution to convert those invoked inside transactions to the corresponding transactional code. Our STM compiler employs a Lightweight Dynamic Binary Translation and Optimization Module (LDBTOM) to automatically convert legacy binary functions to transactional code. In this paper, we describe our LDBTOM system, which 1) seamlessly integrates the translated code with the STM compiler generated code to run on the STM runtime, and 2) optimizes the translated code taking advantage of dynamic optimization opportunities and STM runtime information. Although the binary code is inherently harder to optimize than high-level source code, our experiment shows that it can be translated and optimized into efficient transactional code by LDBTOM.

Dynamic binary translation (DBT) is a runtime instrumentation technique commonly used to support profiling, optimization, secure execution, and bug detection tools for application binaries. However, DBT frameworks may incorrectly handle multithreaded programs due to races involving updates to the application data and the corresponding metadata maintained by the DBT. Existing DBT frameworks handle this issue by serializing threads, disallowing multithreaded programs, or requiring explicit use of locks. This paper presents a practical solution for correct execution of multithreaded programs within DBT frameworks. To eliminate races involving metadata, we propose the use of transactional memory (TM). The DBT uses memory transactions to encapsulate the data and metadata accesses in a trace, within one atomic block. This approach guarantees correct execution of concurrent threads of the translated program, as TM mechanisms detect and correct races. To demonstrate this approach, we implemented a DBT-based tool for secure execution of x86 binaries using dynamic information flow tracking. This is the first such framework that correctly handles multithreaded binaries without serialization. We show that the use of software transactions in the DBT leads to a runtime overhead of 40%. We also show that software optimizations in the DBT and hardware support for transactions can reduce the runtime overhead to 6%.

A hardware/software co-designed processor transparently supports a ubiquitous ISA (e.g. ×86) with diversified and innovative microarchitectural implementations. It leverages co-designed HW features and dynamic binary translation (DBT) SW to morph existing binary programs to scale performance and save power. On such systems, the portable bytecode of modern dynamic languages (e.g. Java, JavaScript, etc.) is first translated into the code in the architecture ISA by the just-in-time (JIT) compilation in the bytecode virtual machine, and then into the code in the internal implementation ISA by the DBT. This not only incurs the translation overheads twice, but also brings significant emulation inefficiency as the DBT does not have the high level bytecode information. In this paper, we present AccelDroid, which accelerates the Android Dalvik bytecode execution on the HW/SW co-designed processor through direct bytecode translation in the DBT. Our experiments on a HW/SW co-designed Transmeta Efficeon machine show that AccelDroid can improve performance by 78% and save energy by 40% for the CaffeineMark 3.0 benchmark suite.

Transmeta's Efficeon microprocessor is a full, system-level implementation of the x86 architecture, comprising a native VLIW microprocessor with a software layer, the Code Morphing Software (CMS), that combines an interpreter, dynamic binary translator, optimizer, and run-time system. This software layer provides unique opportunities for studying the tradeoffs between interpretation and dynamic translation with various levels of optimization, while running arbitrary PC workloads. This talk presents some results from such a study, and provides insight into what mix of these emulation modes is most desirable over the range of tested workloads.

Condition codes (CCs) are special architected bits that characterize instruction results. Reducing the overhead of emulating condition codes is a critical performance issue in binary translation. In this paper, we propose a two-phase optimization approach for condition codes, which has been implemented in Crossbit-a machine adaptable dynamic binary translator (DBT). First, redundant flag computing code in a basic block is reduced based on the information collected by Crossbit when the block is identified. Then, lazy evaluation technique is used inter basic blocks, which make the condition codes emulation more efficient. Experimental results show that this method works much better than the straightforward emulation method-it eliminate the code size more than 20% and have a performance gain nearly 40% in Crossbit. The data proves that the approach is effective to remove the redundant code and to increase the performance of the translated code.

This paper describes the design and implementation of a novel dynamic binary translation technique specialized for embedded systems. Virtual platforms have been widely used to develop embedded software and dynamic binary translation is essential to boost their speed in simulations. However, unlike application simulation, the code generated for systems simulation is still slow because the simulator must replicate all of the functions of the target hardware. Embedded systems, which focus on providing one or a few functions, utilize only a small portion of the processor's features most of the time. For example, they may use a Memory Management Unit (MMU) in a processor to map physical memory to effective addresses, but they may not need paged memory support as in an OS. We can exploit this to specialize the dynamically translated code for more performance. We built a specialization framework on top of a functional simulator with a dynamic binary translator. Using the framework, we implemented three specializers for an MMU, bi-endianness, and register banks. Experiments with the EEMBC1.1 benchmark showed that the speed of the specialized code was up to 39% faster than the unspecialized code.

Virtualization via dynamic binary translation is essentially an emulator. The main advantage is that the CPU of the guest does not have to be the same as the CPU of the host. Although, dynamic binary translators (DBT) are gaining popularity and offering a promising future in the modern virtual executive environments, the requirements of DBTs’ processing and memory resources have seriously hampered the performance of host platforms. For cloud computing seems to offer incredible lightning-quick processing power and unlimited storage. In this paper, we propose a novel distributed DBT system-DistriBit for resource-limited thin clients computing. Meanwhile, we study the effects of the number of virtual registers and trace length to improve the performance of DistriBit. Our results demonstrate that improving these two factors (the number of virtual registers and trace length) may help to improve program speedup by up to 1.4 to 3.9x and 2 to3x for certain benchmark programs. Key words: Virtualization, dynamic binary translation, DBT, cloud computing, distribit.

In recent years, dynamic reconfigurable processor which can achieve reconfiguration with a few cycles is proposed. The fast reconfiguration makes run-time reconfiguration possible, and the run-time reconfiguration gives a new possibility to the dynamic reconfigurable processor, i.e. the dynamic reconfigurable processor can also execute partitioned independent subtasks with repeated reconfigurations and executions. However, to achieve an execution with the run-time reconfiguration, performance should be evaluated with various overheads: reconfiguration, memory accesses, etc. The overheads depend on reconfigurable architectures, and it is generally difficult to evaluate the overhead. As the overhead may critically affect the performance, designers should carefully explore design space for suitable architectures. In this paper, we propose a dynamic reconfigurable architecture exploration method based on Parameterized Reconfigurable Processor model (PRP-model) and task partitioning optimization algorithm for architecture exploration corresponding to proposed PRP-model. Experimental results showed that the proposed PRP-model and the task partitioning algorithm for PRP-model can fast evaluate various reconfigurable architectures, and designers can easily find suitable reconfigurable architectures by changing the PRPmodel parameters.

A Dynamic Binary Translation (DBT) system dynamically translates program binaries built for a guest platform into code for the host machine that runs the program, one basic block at a time. Even after optimizations, auxiliary tasks performed alongside program emulation by the DBT system introduce performance overheads as compared to executing the program on the native guest platform. In this work, we analyze the extent and causes for a DBT system's startup performance latency. We then focus on understanding and alleviating the program translation cost that is a significant contributor to and disproportionately impacts the startup overhead. We propose and assess the potential of a new technique that parallelizes program translations on multi-core machines to reduce its evident run-time costs. We explain the challenges in achieving such parallelization and discuss and evaluate solutions.

Heterogeneous MPSoCs where different types of cores share a baseline ISA but implement different operational accelerators combine programmability with flexible customization. They hold promise for high performance under power and area limitations. However, transparent binary execution and dynamic scheduling is hard on those platforms. The stateof-the-art approach for transparent accelerated execution is fault-and-migrate (FAM): when a thread executes an accelerating instruction unavailable on the host core, it is forcibly migrated to an accelerating core which implements the instruction natively. Unfortunately, this approach prohibits dynamic scheduling through flexible thread migration, which is essential to any asymmetric platform for efficient utilization of heterogeneous resources. We present two distinct binary-level techniques - Dynamic Binary Rewriting (DBR) and Dynamic Binary Translation (DBT) - which enable selective acceleration, while preserving transparent thread execution and migration, to any core in the system, at any point in time. DBR rewrites binary code to exploit any accelerating instructions available in the host core. DBT implements a-fault-and-rewrite scheme, which sets up trampolines to emulation routines for these accelerating instructions which are not available on the host core. Both methods customize binary code on demand, enabling flexible migration. We evaluate the overhead of DBR and DBT against FAM on a real hardware shared-ISA MPSoC prototype. Experiments with single-thread programs show flexible migration is possible with manageable overhead. We measure the performance of our binary-level techniques by artificially triggering periodic thread migration between a Base and an accelerating (ACC) core. Periodic migration, without aiming for optimized scheduling, results in an average slowdown of about 40% under DBR or about 10% under DBT, compared to FAM driven scheduling. We also show results for a speedup proportional dynamic scheduler, enabled by our techniques, using multi-program workloads. In this case, up to 50% faster execution times can be achieved by leveraging flexible thread migration.

Attacks often exploit vulnerabilities in memory to compromise a system or get classified information. In spite of extensive research, attackers are still able to find security holes. In order to address the attacks exploiting vulnerabilities in memory, we propose fine grain dynamic memory access control with profiling. This technique builds profile data for memory accesses with training, and this data is referenced later to check if an access is an allowed legitimate one. To facilitate the fine grain memory access control at a reasonable overhead, instructions and memory words form access group to represent allowed accesses: within one access group, one instruction is accessing at least two memory words or one memory word is accessed by at least two instructions. One access group is assigned with its unique color, and this color is referenced to verify legitimacy of a memory access. In order to handle memory accesses to run-time generated object, we suggest an efficient addressing methods and identifier for associating group information with the object in profiling procedure. To verify its feasibility in statistical point of view, we have implemented our idea in Bochs simulator and results show that memory access control with profiling data can be reliable.