An evaluation of multiple branch predictor and trace cache advanced fetch unit designs for dynamically scheduled superscalar processors

The need for multiple branch prediction is inherent to wide instruction fetching. This paper presents a completion time multiple branch predictor called the Tree-based Multiple Branch Predictor (TMP) that builds on previous single branch prediction techniques. It employs a tree structure of branch predictors, or tree-node predictors, and achieves accurate multiple branch prediction by leveraging the high accuracies of the individual branch predictors. A highly-efficient TMP design uses the 2-bit saturating counters for the tree-node predictors. To achieve higher prediction rate, the TMP employs two-level schemes for the tree-node predictors resulting in a three-level TMP design. Placing the TMP at completion time reduces the critical latency in the front-end of the pipeline; the resultant longer update latency does not significantly impact the overall performance. In this paper the TMP is applied to a trace cache design and shown to be very effective in increasing its performance.

As the issue width and the number of function units of superscalar processors continue to increase, the fetch unit must support a large fetch bandwidth in order to fully utilise the datapath resources. This trend makes power issue worse in the fetch unit since the traditional instruction fetch mechanism is not optimised for power consumption. This paper explores the problem of extra power consumption in traditional instruction caches because of dynamic control flows. Capturing the dynamic paths/characteristics of code during the course of execution, trace caches provide a potential framework for power optimisation in the fetch unit. Our study shows that conventional trace caches (CTC) may increase power consumption in the fetch unit because of the simultaneous access to both the trace cache and the instruction cache, and sequential trace caches (STC) have the advantage of lower power consumption at the cost of a significant performance loss. In order to address this problem, we perform a detailed study of trace distribution and access locality. Based on this study, we first propose a new model, the selective trace cache (SLTC). SLTC uses both compiler and hardware support to selectively control trace cache lookup and update. Experimental evaluation shows that our selective trace cache achieves up to 42.2% power reduction over CTC and an additional reduction of up to 21.8% over STC, on the average, while only trading a performance loss of no more than 1.8% compared to CTC. Further, we propose a dynamic direction prediction based trace cache (DPTC), which eliminates the need for compilation and instruction set architecture (ISA) modification involved in SLTC. Powered by a fetch direction predictor, DPTC achieves competitive power efficiency. On the average, DPTC reduces the power consumption by up to 40.5% and 17.6% in the fetch unit compared to CTC and STC, respectively, by trading a performance loss of less than 2.4% to CTC.

Recent superscalar processors issue four instructions per cycle. These processors are also powered by highly-parallel superscalar cores. The potential performance can only be exploited when fed by high instruction bandwidth. This task is the responsibility of the instruction fetch unit. Accurate branch prediction and low I-cache miss ratios are essential for the efficient operation of the fetch unit. Several studies on cache design and branch prediction address this problem. However, these techniques are not sufficient. Even in the presence of efficient cache designs and branch prediction, the fetch unit must continuously extract multiple, non-sequential instructions from the instruction cache, realign these in the proper order, and supply them to the decoder. This paper explores solutions to this problem and presents several schemes with varying degrees of performance and cost. The most-general scheme, the collapsing buffer, achieves near-perfect performance and consistently aligns instructions in excess of 90% of the time, over a wide range of issue rates. The performance boost provided by compiler optimization techniques is also investigated. Results show that compiler optimization can significantly enhance performance across all schemes. The collapsing buffer supplemented by compiler techniques remains the best-performing mechanism. The paper closes with recommendations and suggestions for future.

We propose an enhanced multiple branch predictor using per-primary address branch histories. Using this scheme, the interferences among different branches are reduced, enhancing the average prediction accuracy. Also, since the branches predicted each cycle do not suffer from successive dependencies, predictions are generated in parallel. This scheme results in higher average branch prediction accuracy than the previous global history scheme under the same implementation cost. For two branch predictions, the prediction accuracy of integer benchmarks varies between 92.0 and 96.9 percent. For floating point benchmarks including nasa7, the accuracy is between 94.8 and 95.8 percent.KeywordsPrediction AccuracySecondary BranchBranch PredictionTertiary BranchHistory RegisterThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Most current single-chip processors employ an on-chip instruction cache to improve performance. A miss in this instruction cache will cause an external memory reference which must compete with data references for access to the external memory, thus affecting the overall performance of the processor. One common way to reduce the number of off-chip instruction requests is to increase the size of the on-chip cache. An alternative approach is presented in this paper, in which a combination of an instruction cache, instruction queue and instruction queue buffer is used to achieve the same effect with a much smaller instruction cache size. Such an approach is significant for emerging technologies where high circuit densities are initially difficult to achieve yet a high level of performance is desired, or for more mature technologies where chip area can be used to provide more functionality. The viability of this approach is demonstrated by its implementation in an existing single-chip processor.

Several studies have considered reducing instruction cache misses and branch penalty stall cycles by means of various forms of code placement. Most proposed approaches rearrange procedures or basic blocks in order to speed up execution on sequential architectures with branch prediction. Moreover, most works focus mainly on instruction cache performance and disregard execution cycles. To the best of our knowledge, no work has specifically addressed statically scheduled ILP machines like VLIWs, with control-transfer delay slots. We propose a new code positioning algorithm especially designed for VLIW-style architectures, which allows to trade off tighter schedule for program locality. Our measurements indicate that code positioning, as a result of tighter program schedule and removed unconditional jumps, can significantly reduce the number of execution cycles, by up to 21%, while improving program locality and instruction cache performance.

Computing relies on architecture specifications to decouple hardware and software development. Historically these have been prose documents, with all the problems that entails, but research over the last ten years has developed rigorous and executable-as-test-oracle specifications of mainstream architecture instruction sets and “user-mode” concurrency, clarifying architectures and bringing them into the scope of programming-language semantics and verification. However, the system semantics, of instruction-fetch and cache maintenance, exceptions and interrupts, and address translation, remains obscure, leaving us without a solid foundation for verification of security-critical systems software.In this paper we establish a robust model for one aspect of system semantics: instruction fetch and cache maintenance for ARMv8-A. Systems code relies on executing instructions that were written by data writes, e.g. in program loading, dynamic linking, JIT compilation, debugging, and OS configuration, but hardware implementations are often highly optimised, e.g. with instruction caches, linefill buffers, out-of-order fetching, branch prediction, and instruction prefetching, which can affect programmer-observable behaviour. It is essential, both for programming and verification, to abstract from such microarchitectural details as much as possible, but no more. We explore the key architecture design questions with a series of examples, discussed in detail with senior Arm staff; capture the architectural intent in operational and axiomatic semantic models, extending previous work on “user-mode” concurrency; make these models executable as test oracles for small examples; and experimentally validate them against hardware behaviour (finding a bug in one hardware device). We thereby bring these subtle issues into the mathematical domain, clarifying the architecture and enabling future work on system software verification.

Good API design enables many clients to effectively use the core functionality implemented by the APIs. For real-world applications however, correctly using the APIs and identifying what methods to use and how to invoke them appropriately can be challenging. Researchers have developed a number of API synthesis approaches that enable a semantically rich form of API completion where the client provides a description of desired functionality, e.g., in the form of test suites, and the automatic tools create method sequences using the desired APIs based on the given correctness criteria (e.g., all given tests pass). However, existing API synthesis approaches are largely limited to creating single basic blocks of code and do not readily handle multiple blocks in the presence of loops (or recursion) and complex test executions. A key issue with handling multiple blocks is the very large space of possible method sequences and their combinations. This paper introduces EdSynth, an API synthesis approach that explores the sequence spaces on-demand during the test execution; that is, the given tests not only provide a validation mechanism - as is common in test-driven API synthesis - but also play a vital role in guiding the space exploration by helping prune much of it. EdSynth follows the spirit of recent work on test-execution-driven synthesis and lazily initializes candidates during the execution of given tests where the part of the candidate completion that is actually executed directly determines the generation of future candidates. To further optimize the space exploration, EdSynth ranks API candidates based on a set of pre-defined heuristics. We evaluate EdSynth's ability to synthesize complex APIs in the presence of conditional statements, loops and multiple basic blocks. The experimental results show that EdSynth is effective at handling synthesis tasks with multiple API sequences in both the conditions and bodies of loops/branches; moreover, when applied to synthesis of straight-line code, EdSynth compares well with a state-of-the-art API synthesis tool that only handles straight-line code. The experiments show that EdSynth's ranking strategies help reduce synthesis time by 43%.

This paper presents a novel approach to make the on-chip instruction cache of a SoC to function simultaneously as a regular instruction cache and a real time program trace compressor. This goal is accomplished by exploiting the dictionary feature of the instruction cache with a small support circuit attached to the side of the cache. The trace compression works in both the bypass mode and the online mode. Compared with related work, this work has the advantage of utilizing the existing instruction cache, which is indispensable in modern SoCs, and thus saves significant amount of hardware resource. The RTL implementation of a 4KB trace-capable instruction cache, a 4KB data cache and an academic ARM7 processor core has been accomplished. The experiments show that the cache achieves average compression ratio of 90% with a very small hardware overhead of 3652 gates. In addition, the trace support circuit does not impact the global critical path. Therefore, the proposed approach is highly feasible on-chip debugging/monitoring solution for SoCs, even for cost sensitive ones such as consumer electronics.

Frequent control dependencies caused by IF- and loop-statements limit the parallelism usable in High-Level Synthesis. Loop pipelining is a powerful way to increase parallelism, but is often limited by these control dependencies. Multiple branch prediction (MBP-SC) applies loop pipelining and speculative computation to the most probable path and serves other paths during the restore phase (prediction error correction). In this paper we combine MBP-SC and loop pipelining and give a scheduling algorithm. Further MBP-SC improvement comes from parallel branch execution. The results show a considerable speedup compared to previous approaches. >

The paper focuses on reducing power in instruction cache by eliminating the fetching of instructions that are not needed from a cache line. We propose a mechanism that predicts which instructions are going to be used out of a cache line before that line is fetched into the instruction buffer. The average instruction cache power savings obtained by using our fetch predictor is 22% for SPEC95 benchmark suite.

Increasing the execution power requires a high instruction issue bandwidth, and decreasing instruction encoding and applying some code improving techniques cause code expansion. Therefore, the instruction memory hierarchy performance has become an important factor of the system performance. An instruction placement algorithm has been implemented in the IMPACT-I (Illinois Microarchitecture Project using Advanced Compiler Technology - Stage I) C compiler to maximize the sequential and spatial localities, and to minimize mapping conflicts. This approach achieves low cache miss ratios and low memory traffic ratios for small, fast instruction caches with little hardware overhead. For ten realistic UNIX * programs, we report low miss ratios (average 0.5%) and low memory traffic ratios (average 8%) for a 2048-byte, direct-mapped instruction cache using 64-byte blocks. This result compares favorably with the fully associative cache results reported by other researchers. We also present the effect of cache size, block size, block sectoring, and partial loading on the cache performance. The code performance with instruction placement optimization is shown to be stable across architectures with different instruction encoding density.

Semiconductor technology evolution suggests that permanent failure rates will increase dramatically with scaling, in particular for SRAM cells. While well known approaches such as error correcting codes exist to recover from failures and provide fault-free chips, they will not be affordable anymore in the future due to their growing cost. Consequently, other approaches like fine-grained disabling and reconfiguration of hardware elements (e.g. individual functional units or cache blocks) will become economically necessary. This fine-grained disabling will degrade performance compared to a fault-free execution. To the best of our knowledge, all static worst-case execution time (WCET) estimation methods assume fault-free processors. Their result is not safe anymore when fine-grained disabling of hardware components is used. In this paper we provide the first method that statically calculates a probabilistic WCET bound in the presence of permanent faults in instruction caches. The proposed method derives a probabilistic WCET bound for a program, cache configuration, and probability of cell failure. As our method relies on static analysis to bound the longest path, its probabilistic nature only stems from the probability that faults actually occur. Our method is computationally tractable because it does not require an exhaustive enumeration of all the possible combinations of faulty cache blocks. Experimental results show that it provides WCET estimates very close to, but never below, the method that derives probabilistic WCETs by enumerating all possible locations of faulty cache blocks. The proposed method not only allows to quantify the impact of permanent faults on WCET estimates, but, most importantly, can be used in architectural exploration frameworks to select the most appropriate fault management mechanisms and design parameters for current and future chip designs.

Code placement techniques have traditionally improved instruction fetch bandwidth by increasing instruction locality and decreasing the number of taken branches. However, traditional code placement techniques have less benefit in the presence of a trace cache that alters the placement of instructions in the instruction cache. Moreover, as pipelines have become deeper to accommodate increasing clock rates, branch misprediction penalties have become a significant impediment to performance. We evaluate pattern history table partitioning, a feedback directed code placement technique that explicitly places conditional branches so that they are less likely to interfere destructively with one another in branch prediction tables. On SPEC CPU benchmarks running on an Intel Pentium 4, branch mispredictions are reduced by up to 22% and 3.5% on average. This reduction yields a speedup of up to 16.0% and 4.5% on average. By contrast, branch alignment, a previous code placement technique, yields only up to a 4.7% speedup and less than 1% on average.

The trace cache is a recently proposed solution to achieving high instruction fetch bandwidth by buffering and reusing dynamic instruction traces. This work presents a new block-based trace cache implementation that can achieve higher IPC performance with more efficient storage of traces. Instead of explicitly storing instructions of a trace, pointers to blocks constituting a trace are stored in a much smaller trace table. The block-based trace cache renames fetch addresses at the basic block level and stores aligned blocks in a block cache. Traces are constructed by accessing the replicated block cache using block pointers from the trace table. Performance potential of the block-based trace cache is quantified and compared with perfect branch prediction and perfect fetch schemes. Comparing to the conventional trace cache, the block-based design can achieve higher IPC, with less impact on cycle time.Results: Using the SPECint95 benchmarks, a 16-wide realistic design of a block-based trace cache can improve performance 75% over a baseline design and to within 7% of a baseline design with perfect branch prediction. With idealized trace prediction, it is shown the block-based trace cache with an 1K-entry block cache achieves the same performance of the conventional trace cache with 32K entries.