Hybrid source-level simulation of data caches using abstract cache models

The performance of modern processors is often limited by execution stalls resulting from long memory access latencies. Compile-time optimizations, deep cache hierarchies and prefetching mechanisms already provide significant performance gains, by performing memory accesses in parallel with computation. However, they are reaching a throughput improvement limit. Hence, new solutions that effectively exploit the memory access patterns to improve processing throughput are required. To achieve this objective, a new compiler-assisted data streaming method is proposed. It leverages static analysis and code transformations with an on-chip data streaming support as a viable alternative to prefetching mechanisms for regular code structures. Static analysis is used to identify and encode memory accesses with a dedicated representation. Then, a code transformation algorithm detaches data indexation and address calculation from computation, allowing for a significant code reduction. An on-chip data stream controller, attached to the L1 data cache, is used to autonomously generate memory accesses from the pattern representation and reorganize the data transfers in streams, with the aid of stream buffers. When compared with state-of-the-art prefetchers, the proposed solution provides up to 26 percent of code reduction, an IPC improvement of 2.4x, and an average performance improvement of 40 percent.

When it comes to performance, embedded systems share many problems with their higher-end counterparts. The growing gap between top processor frequency and memory access speed, the memory wall, is one such problem. Driven, in part, by low energy consumption and low cost requirements, embedded systems are often customized to a single application, or a very small set of applications. In addition, time-to-market requirements and the increasing complexity of embedded systems drives the need for fully or partially automated design tools and also to the extensive use of caches and cache hierarchies. The introduction of multi-processor-based embedded platforms has accelerated this trend; as the design space for embedded systems has grown, designers have become unclear as to whether automatic processor customization tools can cope with this increased complexity. The recent introduction of new techniques addressing the automatic customization, such as Architecturally Visible Storage (AVS) memory-enhanced Instruction Set Extension (ISE) identification algorithms, has also created new challenges. AVS memories are distinct from the cache hierarchy and rely on Direct Memory Access (DMA) transfers to communicate with main memory. In an embedded system containing hardware-managed caches, these extra AVS memories, in combination with their corresponding DMA transfers, cause coherence and consistence problems. Although the problems of coherence and consistence are well known in multi-processor systems, conventional solutions may be expensive in terms of area and power consumption, rendering them unacceptable for use in embedded systems. This thesis presents two low cost coherence mechanisms that solve these two problems. The first mechanism addresses embedded systems that already contain a hardware coherence protocol, like many high-end embedded multi-processor systems. Traditionally, the DMA transactions are transparent to the hardware coherence protocol. By ensuring visibility of these DMA transactions to the hardware coherence protocol, coherence can be guaranteed between AVS memories and data cache(s). As a result, minor changes to the DMA engine are required. Moreover, by forcing the processor pipeline to stall if a DMA transfer is active, memory consistence can be guaranteed. This mechanism provides significant speedup when compared to the execution of a non-ISE-enhanced system; however, due to the increase in bus traffic, this speedup comes at the expense of an increase in energy consumption. Coherent and Speculative DMA are both implementations of this mechanism. Single-processor systems do not contain hardware coherence protocols, and would therefore benefit from a lower-cost solution to the coherence and consistence problems than a hardware coherence protocol. By tightly coupling the AVS memories to the hardware cache, coherence and consistence for the complete system can be guaranteed. This coupling requires insignificant changes to the hardware cache's hit detection circuitry and state machine without influencing its critical path, thus it is inherently inexpensive. This mechanism provides significant speedups and reduces the energy consumed if compared to the execution on non-ISE-enhanced systems. Furthermore, the tight coupling enables direct communication between the AVS memories and the data cache, making this mechanism independent from the processor-to-memory distance. Virtual Ways and Way Stealing are both implementations of this mechanism. Besides enforcing coherence and consistence, the ability to integrate the architectural changes into an automated design flow is important. This thesis shows the influence of Coherent DMA, Speculative DMA, Virtual Ways, and Way Stealing on the ISE-identification algorithm. It shows the architectural requirements and the cost for enforcing coherence and consistence that need to be taken into account when applying these mechanisms in an automated flow without formulating new algorithms.

Current high-performance multicore processors provide users with a non-uniform memory access model (NUMA). These systems perform better when threads access data on memory banks next to the core where they run. However, ensuring data locality is difficult. In this paper, we propose compiler analyses and code generation methods to support a lightweight runtime system that dynamically migrates memory pages to improve data locality. Our technique combines static and dynamic analyses and is capable of identifying the most promising pages to migrate. Statically, we infer the size of arrays, plus the amount of reuse of each memory access instruction in a program. These estimates rely on a simple, yet accurate, trip count predictor of our own design. This knowledge lets us build templates of dynamic checks, to be filled with values known only at runtime. These checks determine when it is profitable to migrate data closer to the processors where this data is used. Our static analyses are quadratic on the number of variables in a program, and the dynamic checks are O(1) in practice. Our technique does not require any form of user intervention, neither the support of a third-party middleware, nor modifications in the operating system's kernel. We have applied our technique on several parallel algorithms, which are completely oblivious to the asymmetric memory topology, and have observed speedups of up to 4x, compared to static heuristics. We compare our approach against Minas, a middleware that supports NUMA-aware data allocation, and show that we can outperform it by up to 50% in some cases.

The existing tools of deductive verification allow us to successfully prove the correctness of functions written in high-level languages such as C or Java. However, this may not be enough for critical software because even fully verified code cannot guarantee the correct generation of machine code by the compiler. At the moment, developers of such systems have to accept the compiler correctness assumption, which, however, is extremely undesirable, but inevitable due to the lack of full-fledged systems of formal verification of machine code. It is also worth noting that the verification of machine code by a person directly is an extremely time-consuming task due to the high complexity and large amounts of machine code. One of the approaches to simplify the verification of machine code is automatic deductive verification reusing the formal specification of the high-level language function. The formal specification of the function consists of the specification of the pre- and postcondition, as well as loop invariants, which specify conditions that are satified at each iteration of the loop. When compiling a program into machine code, pre- and postconditions are preserved, which, however, cannot be said about loop invariants. This fact is one of the main problems of automatic verification of machine code with loops. Another important problem is that high-level function variables often have ‘projections' to both registers and memory at the machine code level, and the verification procedure requires that invariants be described for each variable, and therefore the missing invariants must be generated. This paper presents an approach to solving these problems, based on the analysis of the control flow graph, and intelligent search of the locations of variables.

This paper presents the results of a new method for interleaving CPU instructions in x86-64 machine code, such that one can hide executable code within other valid instructions. The aim is to make it more difficult to reverse-engineer software at a machine code level – to obfuscate instructions. A result is a hidden execution path within a visible main instruction path. While previous methods for this instruction obfuscation exist, we present a new method which builds upon past work, and which allows a greater flexibility in the selection of main instruction path instructions. The result of this new approach is to provide a methodology for instruction concealment which is free of restrictions present in prior work.

With the trend toward specialization, an efficient memory-path design is vital to capitalize customization in data-path. A monolithic memory hierarchy is often highly inefficient for irregular applications, traditionally targeted for CPUs. New approaches and tools are required to offer application-specific memory customization combining the benefits of cache and scratchpad memory simultaneously. This paper introduces a novel approach for automated application-specific on-chip memory assignment and tiling. The approach offers two major tools: (1) static memory access analysis and (2) variable-level memory assignment. Static memory analysis performs at the LLVM abstraction. It extracts target-independent pointer behaviors, measures the access strides and analyze the prefetchability of variables. (2) variable-level memory assignment creates a memory allocation graph for memory assignment (cache vs. scratchpad) based on the variables size and their estimated locality. It also explores the opportunity for tiling memory access. For the exploration and results, this paper uses Machsuite benchmarks (with both regular & irregular memory access behaviors), and gem5-Aladdin tool for performance & power evaluation. The proposed approach optimizes the memory hierarchy by automatically combining the benefits of cache, (tiled-) scratchpad at variable level granularity per individual applications. The results demonstrate more than 45% improvement in our power-stall product, on average, over the monolithic cache or scratchpad design.

Graphics processor (GPU) architectures have evolved rapidly in recent years with increasing performance demanded by 3D graphics applications such as games. However, challenges exist in integrating complex GPUs into mobile devices because of power and energy constraints, motivating the need for energy efficiency in GPUs. While a significant amount of power optimisation research effort has concentrated on the CPU system, GPU power efficiency is a relatively new and important area because the power consumed by GPUs is similar in magnitude to CPU power. Power and energy efficiency can be introduced into GPUs at many different levels: 1) Hardware component level - queue structures, caches, filter arithmetic units, interconnection networks, processor cores, etc., can be optimised for power. 2) Algorithm level - the deep and complex graphics processing computation pipeline can be modified to be energy aware. Shader programs written by the user can be transformed to be energy aware. 3) System level - co-ordination at the level of task allocation, voltage and frequency scaling, etc., requires knowledge and control of several different GPU system components We outline two strategies for applying energy optimisations at different levels of granularity in a GPU: (1) Texture Filter Memory (TFM) - an energy-efficient addition to the texture memory hierarchy component; and (2) an overall system level Dynamic Voltage and Frequency Scaling (DVFS) based energy reduction strategy based on an accurate tile-level slack prediction. Texture Filter Memory is an augmentation of the standard GPU texture cache hierarchy. Instead of a regular data cache hierarchy, we employ a small first level register based structure that is optimised for the relatively predictable memory access stream in the texture filtering computation. Predictability of the memory access patterns leads us to reduce power by using a simpler register access structure. Power is saved by avoiding the expensive tag lookup and comparisons present in regular caches. Further, the texture filter memory is a very small structure, whose access energy is much smaller than a data cache of similar performance. Dynamic Voltage and Frequency Scaling, an established energy management technique, can be applied in GPUs by first predicting the workload in a given frame, and, where sufficient slack exists, lowering the voltage and frequency levels so as to save energy while still completing the work within the frame rendering deadline. We apply DVFS in a tiled graphics renderer, where the workload prediction and voltage/frequency adjustment is performed at a tile-level of granularity, which creates opportunities for on-the-fly correction of prediction inaccuracies, ensuring high frame rates while still delivering low power. Tile-level energy management schemes are distinctly superior to schemes that work at the granularity of the entire frame in terms of energy saving and quality of the rendering measured in terms of frames rendered per second. The prediction mechanism relies both on the history - workload observed for the tile in recent frames, as well as rank - a quantification of the tile workload based on the frame structure, consisting of geometry, pixel shading, texture, and raster operations workloads.

Providing adequate data bandwidth is extremely important for a wide-issue superscalar processor to achieve its full performance potential. Adding a large number of ports to a data cache, however, becomes increasingly inefficient and can add to the hardware complexity significantly. This paper takes an alternative or complementary approach for providing more data bandwidth, called the data-decoupled architecture. The approach, with support from the compiler and/or hardware, partitions the memory stream into two independent streams early in the processor pipeline, and feeds each stream to a separate memory access queue and cache. Under this model, the paper studies the potential of decoupling memory accesses to program's local variables that are allocated on the run-time stack. Using a set of integer and floating-point programs from the SPEC95 benchmark suite, it is shown that local variable accesses constitute a large portion of all the memory references, while their reference space is very small, averaging around 7 words per (static) procedure. To service local variable accesses quickly, two optimizations, fast data forwarding and access combining, are proposed and studied. Some of the important design parameters, such as the cache size, the number of cache ports, and the degree of access combining, are studied based on simulations. The potential performance of the proposed scheme is measured using various configurations, and it is concluded that the scheme can become a viable alternative to building a single multi-ported data cache.

The fundamental configuration of a high-performance microprocessor for virtual memory systems is described, which was designed to TRON specifications[1]. This microprocessor has a pipeline configuration that is separated into register-to-register operations and load/store operations; this enables high performance to be achieved, together with the provision of high-speed on-chip resources such as stack cache and data cache. By separating into the register-to-register operation pipeline and the memory access pipeline, it is possible to execute register-to-register operations and memory access in parallel, and to reduce the number of instruction execution clocks. High performance has been additionally achieved by splitting the on-chip operand cache into stack data and other-than-stack data portions, which enables the cache for stack data to be operated as a virtual register.

Today, security is a requirement for smartphone operating systems that are used to store and handle sensitive information. However, smartphone users usually download third-party applications that can leak personal data without user authorization. For this reason, the dynamic taint analysis mechanism is used to control the manipulation of private data by third-party apps [9]. But this technique does not detect control flows. In particular, untrusted applications can circumvent Android system and get privacy sensitive information through control flows. In this paper, we propose a hybrid approach that combines static and dynamic analysis to propagate taint along control dependencies in Android system. To evaluate the effectiveness of our approach, we analyse 27 free Android applications. We found that 14 of these applications use control flows to transfer sensitive data. We successfully detect that 8 of them leaked private information. Our approach creates 19 % performance overhead that is due to the propagation of taint in the control flow. By using our approach, it becomes possible to detect leakage of personal data through control flows.

Debugging virtual machines (VMs) presents unique challenges, especially meta-circular VMs, which are written in the same language they implement. Making sense of runtime state for such VMs requires insight and interaction at multiple levels of abstraction simultaneously. For example, debugging a Java VM written in Java requires understanding execution state at the source code, bytecode and machine code levels. However, the standard debugging interface for Java, which has a platform-independent execution model, is itself platform-independent. By definition, such an interface provides no access to platform-specific details such as machine code state, stack and register values. Debuggers for low-level languages such as C and C++, on the other hand, have direct access only to low-level information from which they must synthesize higher-level views of execution state. An ideal debugger for a meta-circular VM would be a hybrid: one that uses standard platform-independent debugger interfaces but which also interacts with the execution environment in terms of low-level, platform-dependent state. This paper presents such a hybrid architecture for the meta-circular Maxine VM. This architecture adopts unchanged a standard debugging interface, the Java Platform Debugger Architecture (JPDA), in combination with the highly extensible NetBeans Integrated Development Environment. Using an extension point within the interface, additional machine-level information can be exchanged between a specialized server associated with the VM and plug-in extensions within NetBeans.

In High Performance Computing (HPC) the performance of applications is paramount, which is has led to a wide body of work on optimizing compilers. However, compilers alone are naturally limited in their potential, as they cannot exploit run-time information to specialize generated code. For this reason, run-time components to perform such specializations are gaining traction. In particular, application or use-case driven specializations have a high potential for optimizations, as they can benefit from explicit guidance from the application or the user, which allows for targeted changes incorporating specific knowledge. Because full compilers and sources are usually unavailable during computation, specializations have to be done at machine code level. However, existing libraries for binary specialization face structural limitations of separating optimization and code generation, in addition to numerous implementation gaps. In this paper, we describe BinOpt, a novel and robust library for performing application-driven binary optimization and specialization using LLVM. A machine code function is lifted directly to LLVM-IR, optimized in LLVM while making use of application-specified information, used to generate a new specialization for a function, and integrated back to the original code. We apply this technique to existing optimized code and show that significant performance improvements can be observed with only a small optimization time overhead.

Timing analysis is an important prerequisite for the design of embedded real-time systems. In order to get tight and safe bounds for the timing of a program, precise information about its control flow and data flow is needed. While actual timings can only be derived from the machine code, many of the supporting analyses (deriving timing-relevant data such as points-to and loop bound information) operate much more effectively on the source code level. At this level, they can use highlevel information that would otherwise be lost during the compilation to machine code. During the optimization stage, compilers often apply transformations, such as loop unrolling, that modify the program's control flow. Such transformations can invalidate information derived from the source code. In our approach, we therefore apply such optimizations already at the source-code level and transform the analyzed information accordingly. This way, we can marry the goals of precise timing analysis and optimizing compilation. In this article we present our implementation of this concept within the SATIrE source-to-source analysis and transformation framework. SATIrE forms the basis for the TuBound timing analyzer. In the ALL-TIMES EU FP7 project we extended SATIrE to exchange timing-relevant analysis data with other European timing analysis tools. In this context, we explain how timing-relevant information from the source code level can be communicated to a wide variety of tools that apply various forms of static and dynamic analysis on different levels.

Nowadays, the presence of cache hierarchies tends to be a common trend in processor architectures, even in hardware for real-time embedded systems. Caches are used to fill the gap between the processor and the main memory, reducing access times based on spatial and temporal locality properties of tasks. Cache hierarchies are going even further however at the price of increased complexity. In this paper, we present a safe static data cache analysis method for hierarchies of non-inclusive caches. Using this method, we show that considering the cache hierarchy in the context of data caches allows tighter estimates of the worst case execution time than when considering only the first cache level. We also present considerations about the update policy for data caches.

In this paper, we investigate methods for improving the hit rates in the first level of memory hierarchy. Particularly, we propose victim cache structures to reduce the number of accesses to more power consuming structures such as level 2 caches. We compare the proposed victim cache techniques to increasing the associativity or the size of the level 1 data cache and show that the enhanced victim cache technique yield better energy-delay and energy-delay-area products. We also propose techniques that predict the hit/miss behavior of the victim cache accesses and bypass the victim cache when a miss can be determined quickly. We report simulation results obtained from SimpleScalar/ARM modeling a representative Network Processor architecture. The simulations show that the victim cache is able to reduce the energy consumption by as much as 17.6% (8.6% on average) while reducing the execution time by as much as 8.4% (3.7% on average) for a set of representative applications.