Control-Flow Independence Reuse via Dynamic Vectorization

Code placement techniques have traditionally improved instruction fetch bandwidth by increasing instruction locality and decreasing the number of taken branches. However, traditional code placement techniques have less benefit in the presence of a trace cache that alters the placement of instructions in the instruction cache. Moreover, as pipelines have become deeper to accommodate increasing clock rates, branch misprediction penalties have become a significant impediment to performance. We evaluate pattern history table partitioning, a feedback directed code placement technique that explicitly places conditional branches so that they are less likely to interfere destructively with one another in branch prediction tables. On SPEC CPU benchmarks running on an Intel Pentium 4, branch mispredictions are reduced by up to 22% and 3.5% on average. This reduction yields a speedup of up to 16.0% and 4.5% on average. By contrast, branch alignment, a previous code placement technique, yields only up to a 4.7% speedup and less than 1% on average.

As Out-of-Order (OOO) cores scale to very large instruction windows to extract higher instruction level parallelism (ILP), the cost of mis-speculation increases tremendously. Branch predictors sitting at the head of the pipeline are a significant contributor to the speculatively executed code. It is critical to limit the execution time down the wrong path for the sake of performance and energy efficiency. Architects continue to increase the branch predictor sizes and improve the prediction algorithms to mitigate the increasing cost of the mis-speculations. Unfortunately, there still exists branches whose outcomes are predominantly data dependent, and that significantly contribute to the overall mispredictions. This work is the first to characterize data dependent branches at a scale spanning 100+ workloads drawn from several application categories and establish a clear motivation to address them. We find that branches which have only one load instruction feeding them and only simple operations to compute the branch direction from the load value are responsible for a significant fraction of the overall mispredictions. We call such branches Direct Data Dependent (3D) branches. We develop a family of synergistic techniques to avoid mispredictions from such 3D-branches. We describe 3D-Branch Overrider, our novel branch overriding technique that operates in the front-end of the pipeline to minimize the branch misprediction penalties. On a modern Icelake-like OOO core equipped with the state-of-the-art branch predictor, we find that our technique provides a 12.7% reduction in branch mispredictions resulting in 3.1% Instructions Per Cycle (IPC) gain while needing just 5.7KB in additional storage. As future cores become wider and deeper, we show that the gains from 3D-Branch Overrider nicely increases, even if the size of the underlying branch predictor is aggressively scaled.

There is wide agreement that one of the most important impediments to the performance of current and future pipelined superscalar processors is the presence of conditional branches in the instruction stream. Speculative execution seems to be one solution of choice to the branch problem, but speculative work is discarded if a branch is mispredicted. Therefore, we need a very accurate branch predictor; 95% accuracy is not good enough. This paper proposes branch classification to help improve the accuracy of branch predictors. Branch classification allows an individual branch instruction to be associated with the branch predictor best suited to predict its direction. Using this approach, a hybrid branch predictor can be constructed such that each component branch predictor predicts those branches for which it is best suited. This paper suggests one classification scheme, analyzes several branch predictors, and proposes a hybrid branch predictor that achieves higher prediction accuracy than any branch predictor previously reported in the literature.

The reasons for performance losses due to conditional branch mispredictions are first studied. Branch misprediction penalties are broken into three categories: pipeline-fill penalty, window-fill penalty, and serialization penalty. The first and third of these produce most of the performance loss, but the second is also significant. Previously proposed dual (or multi) path execution methods attempt to reduce all three penalties, but these methods are also quite complex. Most of the complexity is caused by simultaneously executing instructions from multiple paths.A good engineering compromise is to avoid the complexity of multiple path execution by focusing on methods that reduce only the pipeline and window re-fill penalties. Dual Path Instruction Processing (DPIP) is proposed as a simple mechanism that fetches, decodes, and renames, but does not execute, instructions from the alternative path for low confidence predicted branches at the same time as the predicted path is being executed. All the stages of the pipeline front-end are hidden once the misprediction is detected. This method thus targets the pipeline-fill penalty and is shown to achieve a good trade-off between performance and complexity. To reduce the window-fill penalty, we further propose the addition of a pre-scheduling engine that schedules instructions from the alternative path in an estimated execution order. Thus, after a misprediction, a high number of instructions from the alternate path can be immediately issued to execution, achieving an effect similar to very fast re-filling of the window. Performance evaluation of DPIP in a 14-stage superscalar processor (like IBM Power 4) shows an average IPC improvement of up to 10% for the bzip2 benchmark, and an average of 8% for ten benchmarks from the SPECint95 and SPECint2000 suites.

Modern wide-issue superscalar processors tend to adopt deeper pipelines in order to attain high clock rates. This trend increases the number of on-the-fly instructions in processors and a mispredicted branch can result in substantial amounts of wasted work. In order to mitigate these wasted works, an accurate branch prediction is required for the high performance processors. In order to improve the prediction accuracy, we propose the bimode++ branch predictor. It is an enhanced version of the bimode branch predictor. Throughout execution from the start to the end of a program, some branch instructions have the same result at all times. These branches are defined as extremely biased branches. The bimode++ branch predictor is unique in predicting the output of an extremely biased branch with a simple hardware structure. In addition, the bimode++ branch predictor improves the accuracy using the refined indexing and a fusion function. Our experimental results with benchmarks from SpecFP, SpecINT, multi-media and server area show that the bimode++ branch predictor can reduce the misprediction rate by 13.2% to the bimode and by 32.5% to the gshare.

The effective performance of wide-issue superscalar processors depends on many parameters, such as branch prediction accuracy, available instruction-level parallelism, and instruction-fetch bandwidth. This paper explores the relations between some of these parameters, and more particularly, the requirement in instruction-fetch bandwidth. We introduce new enhancements to boost effectively the instruction-fetch bandwidth of conventional fetch engines. However, experiments strongly show that performance improves less for a given instruction-fetch bandwidth gain as the base fetch bandwidth increases. At the level of bandwidth exhibited by the proposed schemes, the performance improvement is small. This clearly brings to light potential relations between the fetch bandwidth and the other parameters. We provide a model to explain this behaviour and quantify some relations. Based on the experimental observation that the available parallelism in an instruction window of size N grows as the square root /spl radic/N, we derive from the model that the instruction fetch bandwidth requirement increases as the square root of the distance between mispredicted branches. We also show that the instruction fetch bandwidth requirement increases linearly with the parallelism available in a fixed-size instruction window.

Branch prediction is one of the main hurdles in the roadmap towards deeper pipelines and higher clock frequencies. This work presents a new approach to enhancing current branch predictors: Selective Branch Prediction Reversal. The rationale behind this proposal is the fact that many branch mispredictions can be avoided if branch prediction is selectively reversed. We present a Branch Prediction Reversal Unit (BPRU) that selectively reverses branch predictions by correlating with the predicted values of the branch inputs, in addition to recent control flow. As a case study, we have included the BPRU in an already proposed branch predictor, the Branch Predictor through Value Prediction (BPVP). The effect is a reduction by half in its original misprediction rate. We have also measured the improvement when the BPRU engine is used in a hybrid scheme composed of a BPVP and a gshare predictor. Results using immediate updates show average reductions in misprediction rate ranging from 7% to 14%. Performance evaluation of the proposed BPRU in a 20-stage superscalar processor shows an IPC improvement of up to 9%.

Instruction level parallelism has become more and more important in today's microprocessor design. These microprocessors have multiple function units, which can execute more than one instruction at the same machine cycle to enhance the uniprocessor performance. Since the function units are usually pipelined in such microprocessors, branch misprediction penalty tremendously degrades the CPU performance. In order to reduce the branch misprediction penalty, predicated operation has been introduced in such microprocessor design as one of the new architectural features, which allows compilers to remove branches from programs. The compiler converts programs with conditional branch to predicated code on these microprocessors with predicate operation support by using comparison instructions to set up Boolean predicates corresponding to branch conditions. Instructions previously guarded by a branch are guarded by a predicate. Predicates guard execution by either executing or nullifying the instruction according to the predicate's value. In this report we present an algorithm to convert a control flow graph into predicated code. The algorithm shown in this report not only minimizes the number of predicates used for basic blocks in the control flow, but also moves the predicate assignments as early as possible to relax dependence constrains introduced by the if-conversion for the later phases of compiler such as the instruction scheduler and register allocator. The algorithm doesn't simply assign the predicate associated with a basic block to the instructions in the basic block. It assigns the predicate associated with the “farthest” dominator basic block to an instruction, to which the instruction can be speculatively moved without introducing compensation code in the instruction scheduling. An optimization algorithm for if-converted code is discussed in the report as well. The algorithm is specially developed for predicated code optimization, which is different from the traditional optimizations. Common subexpression hoisting and sinking employs flow analysis, that extends the traditional common subexpression elimination technique. This algorithm requires traditional flow analysis information such as dominate and post-dominate relationship. Actually, the algorithm can be generalized on predicated code after if-conversion, which is particularly beneficial for single assignment DAG representation of the predicated code.

Branch instructions are recognized as a major impediment to exploiting instruction level parallelism. Even with sophisticated branch prediction techniques, many frequently executed branches remain difficult to predict. An architecture supporting predicated execution may allow the compiler to remove many of these hard-to-predict branches, reducing the number of branch mispredictions and thereby improving performance. We present an in-depth analysis of the characteristics of those branches which are frequently mispredicted and examine the effectiveness of an advanced compiler to eliminate these branches. Over the benchmarks studied, an average of 27% of the dynamic branches and 56% of the dynamic branch mispredictions are eliminated with predicated execution support.

We evaluate the effects of guarded (or conditional, or predicated) execution on the performance of an instruction level parallel processor employing dynamic branch prediction. First, we assess the utility of guarded execution, both qualitatively and quantitatively, using a variety of application programs. Our assessment shows that guarded execution significantly increases the opportunities, for both compiler and dynamic hardware, to extract and exploit parallelism. However, existing methods of specifying guarded execution have several drawbacks that limit its use. Second, we study the interaction of guarded execution and dynamic branch prediction and show that the use of guarded execution significantly increases the number of instructions between mispredicted branches. Third, we propose a new method of specifying guarded execution. The proposed method uses special GUARD instructions, which can be used to incorporate guarded execution into existing instruction sets. GUARD instructions realize the full power of guarded execution, without the drawbacks of existing methods of specifying guarded execution.

High-performance multiprocessor systems built around out-of-order processors with aggressive branch predictors execute many memory references that turn out to be on a mispredicted branch path. Previous work that focused on uniprocessors showed that these wrong-path memory references may pollute the caches by bringing in data that are not needed on the correct execution path and by evicting useful data or instructions. Additionally, they may also increase the amount of cache and memory traffic. On the positive side, however, they may have a prefetching effect for memory references on the correct path. While computer architects have thoroughly studied the impact of wrong-path effects in uniprocessor systems, there is no previous work on its effects in multiprocessor systems. In this paper, we explore the effects of wrong-path memory references on the memory system behavior of shared-memory multiprocessor (SMP) systems for both broadcast and directory-based cache coherence. Our results show that these wrong-path memory references can increase the amount of cache-to-cache transfers by 32%, invalidations by 8% and 20% for broadcast and directory-based SMPs, respectively, and the number of writebacks by up to 67% for both systems. In addition to the extra coherence traffic, wrong-path memory references also increase the number of cache line state transitions by 21% and 32 % for broadcast and directory-based SMPs, respectively. In order to reduce the performance impact of these wrong-path memory references, we introduce two simple mechanisms - filtering wrong-path blocks that are not likely-to-be-used and wrong-path aware cache replacement - that yield speedups of up to 37%.

Branch misprediction is one of the important causes of performance degradation in superpipelined and superscalar processors. Most of the existing branch predictors, based on the exploiting of branch history, suffer from prediction accuracy decrease caused by frequent context switches. The goal of this research is to reduce misprediction rate (MPR) when the context switches are frequent, and not to increase the MPR when the context switches are relatively rare. We propose two independent, but closely related modifications of global adaptive prediction mechanisms: first, to flush only the branch history register (BHR) at context switch, instead of reinitialization of the whole predictor, and second, to use two separated BHRs, one for user and one for kernel branches, instead of one global history register. We have evaluated the ideas by measurements on real traces from IBS (Instruction Benchmark Set), and have shown that both modifications reduce MPR at negligible hardware cost.

Modern embedded processors use small and simple branch predictors to improve performance. Using complex and accurate branch predictors, while desirable, is not possible as such predictors impose high power and area overhead which is not affordable in an embedded processor. As a result, for some applications, misprediction rate can be high. Such mispredictions result in energy wasted down the mispredicted path. We introduce area-aware and low-complexity pipeline gating mechanisms to reduce energy lost to possible branch mispredictions in embedded processors. We show that by using a simple gating mechanism which comes with 33-bit area overhead, on average, we can reduce the number of executed instructions by 17% (max: 30%) while paying a negligible performance cost (average 1.1%).KeywordsArea OverheadEmbed ProcessorConfidence EstimationWaste ActivityBranch InstructionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

When CPU enters the era of instruction level parallelism, the execution cycle of instructions has become shorter and shorter. Meanwhile, the processing speed of CPU has become faster and faster. However, only from the execution process of CPU instructions, there is a key factor that limits the speed of CPU processing instructions, namely, the branch instruction processing. The branch predictor is used to predict the execution result of the current branch instruction (taken/non-taken), which can make the program counter get the target instruction to continue to execute according to the prediction result before calculating the target jump address. If the prediction is correct, it will greatly improve the execution efficiency of CPU. This paper starts from the two basic categories of static branch prediction and dynamic branch prediction. Then analyse their prediction principle, prediction accuracy and component consumption used for prediction. For static branch predictor, its hardware consumption is less than that of dynamic branch predictor, and its accuracy is also lower than that of dynamic branch predictor. Moreover, their shortcomings are pointed out in this paper. At last, based on the original branch predictor, better branch predictors developed in recent years are introduced. In the future, considering the hardware consumption, I think the development of branch predictor will continue to optimize the branch predictor combining global history and local history, from the number of stored entries, the correlation between branch instructions and so on. On the premise of executing a large number of instructions, every little improvement in the efficiency of the predictor will bring huge benefits.

Transistor budgets have been increasing at a very fast pace in the last years. This increasing transistor density will lead next generation processors to have a billion transistors available. It is the task of the computer architect to find the best way to use them.Out of order superscalar processors exploit parallelism at the finer grain, exploiting Instruction Level Parallelism (ILP). They issue multiple instructions per cycle, often in an order other than the specified by the programmer, using branch prediction and other speculative execution techniques in order to increase the available parallelism.Very Long Instruction Word (VLIW) processors also exploit parallelism at the instruction level, but they mostly rely on the compiler to detect the available parallelism. This increased compiler role allows a simpler design, and can be run at a faster clock rate, compensating for the loss of ILP.Chip Multiprocessors (CMP) join several narrow superscalar/VLIW components into a single processor, and mostly rely on the Thread Level Parallelism (TLP) for performance. The small and simple components can also run at a faster clock rate compensating for the loss of ILP, and significantly increasing throughput.Simultaneous Multithreaded (SMT) processors are based on wide superscalars, and exploit both ILP and TLP by issuing instructions from several different threads to the same pipeline, obtaining the benefits of TLP without sacrificing the ILP on single-threaded applications.For each of these options we will need to find the best balance between performance, design complexity, and power consumption among other factors. Also, the frontiers between them are not clear, and many intermediate design points can be found which leads to better/simpler/cheaper processors for the next generation of high performance computers.