Filter Data Cache: An Energy-Efficient Small L0 Data Cache Architecture Driven byMiss Cost Reduction

Energy efficiency has become the first priority when designing high-performance computing systems, and caches play a key role in improving the performance. However, as caches represent a dominant proportion of the total area in a processor, they dissipate a significant amount of the total energy consumption. A non-unified data cache architecture is suggested to tackle this issue without sacrificing performance. This architecture is based on the principle of providing a smaller cache for a group of frequently used references, which does not require a large cache to provide a high hit rate. Thus, the non-unified design reduces the number of accesses to a relatively large L1 data cache, thereby saving energy and enabling efficient performance. This paper examines the energy consumption of a non-unified data cache design as compared to that of a conventional unified data cache. As a powerful system requires allocation of a significant number of cores on a single die, minimizing the L1 caches in each core helps to incorporate more cores in one chip. Therefore, we conduct experiments for different sizes of L1 data caches, ranging from quite large to small. The experimental results show that compared to a conventional L1 data cache, the non-unified design reduces data cache dynamic energy consumption, on average, by up to 22% for a small data cache (such as a 4KB cache), and by up to 82% for a relatively large (such as a 32KB) cache.

The pattern prediction algorithm has been shown in past research to provide substantial improvements in energy consumption, without sacrificing performance when used to predict access to the filter cache in the instruction memory hierarchy. Since, the pattern predictor and the filter cache are an overhead to the existing cache architecture, it is imperative that they should together not lower the energy efficiency of the existing architecture. Thus, the extra energy consumed by the predictor should be minimal as compared to the overall energy reduction achieved by the introduction of the filter cache in the instruction memory hierarchy. In this paper, we present the hardware architecture of a low-power, single-cycle pattern predictor, which can be used in conjunction with the filter cache. The predictor has been functionally validated against results from simple scalar simulations on the MediaBench benchmark suite. The predictor is capable of performing single cycle predictions for processor clocks of over 0.5 GHz (in 0.35/spl mu/ process technology) and adds an area overhead of a mere 1500 to 2700 gates to the silicon area depending on the configuration chosen. The predictor is area and power efficient, and does not add any significant overheads to the processor's power and area budget. Thus providing an elegant and attractive solution for inclusion in modern embedded processors.

Energy consumption plays a crucial role in the design of embedded processors especially for portable devices. Since memory access consumes a significant portion of the energy of a processor, the design of fast low-energy caches has become a very important aspect of modern processor design. In this paper, we present a novel cache architecture for reduced energy instruction caches. Our proposed cache architecture consists of the L1 cache, multiple line buffers, and a prediction mechanism to predict which line buffer, or L1 cache to access next. We used simulation to evaluate our proposed architecture and compare it with the HotSpot cache, Filter cache, Predictive line buffer cache and Way-Halting cache. Simulation results show that our approach can reduce instruction cache energy consumption, on average, by 75% without sacrificing performance.

Energy consumption is one of the most important issues in resource-constrained embedded systems. Many such systems run Java-based applications due to Java's architecture-independent format (bytecode). Standard techniques for executing bytecode programs, e.g. interpretation or just-in-time compilation, have performance or memory issues that make them unsuitable for resource-constrained embedded systems.A superoperator-extended, lightweight Java Virtual Machine (JVM) can be used in resource-constrained embedded systems to improve performance and reduce memory consumption. This paper shows that such a JVM also significantly reduces energy consumption. This is due primarily to a considerable reduction in the number of memory accesses and thus in energy consumption in the instruction and data TLBs and caches and, in most cases, in DRAM energy consumption. Since the fraction of processor energy dissipated in these units is approximately 60%, the energy savings achieved are significant.The paper evaluates the number of load, store, and computational instructions eliminated by the use of proposed superoperators as compared to a simple interpreter on a set of embedded benchmarks. Using cache and DRAM per access energy we estimate the total processor/DRAM energy saved by using our JVM. Our results show that with 32KB caches the reduction in energy consumption ranges from 40% to 60% of the overall processor, plus DRAM energy. Even higher savings may be achieved with smaller caches and increased access to DRAM as DRAM access energy is fairly high.

Silicon technology advances have made it possible to pack millions of transistors --- switching at high clock speeds --- on a single chip. While these advances bring unprecedented performance to electronic products, they pose difficult power/energy consumption problems. For example, large number of transistors in dense on-chip cache memories consume significant static (leakage) power even if the cache is not used by the current computation. While previous compiler research studied code and data restructuring for improving data cache performance, to our knowledge, there is no compiler-based study that targets data cache leakage power consumption. In this paper, we present code restructuring techniques for array-based and pointer-intensive applications for reducing data cache energy consumption. The idea is to let the compiler to analyze the code and insert instructions that turn off cache lines that keep variables not used by the current computation. This turning off does not destroy contents of a cache line, and waking up the cache line incurs very little overhead. Due to data locality, we find that at a given time only a small portion of the data cache needs to be active; the remaining part can be placed into a leakage-saving mode (state); i.e., they can be turned off. Our preliminary results indicate that the proposed strategy reduces the cache energy consumption significantly. We also show that several compiler optimizations increase the effectiveness of our strategy.

With advances in semiconductor technology, power management has increasingly become a very important design constraint in processor design. In embedded processors, instruction fetch and decode consume more than 40% of processor power. This calls for development of power minimization techniques for the fetch and decode stages of the processor pipeline. For this, filter cache has been proposed as an architectural extension for reducing the power consumption. A filter cache is placed between the CPU and the instruction cache (I-cache) to provide the instruction stream. A filter cache has the advantages of shorter access time and lower power consumption. However, the downside of a filter cache is a possible performance loss in case of cache misses. In this article, we present a novel technique---decode filter cache (DFC)---for minimizing power consumption with minimal performance impact. The DFC stores decoded instructions. Thus, a hit in the DFC eliminates instruction fetch and its subsequent decoding. The bypassing of both instruction fetch and decode reduces processor power. We present a runtime approach for predicting whether the next fetch source is present in the DFC. In case a miss is predicted, we reduce the miss penalty by accessing the I-cache directly. We propose to classify instructions as cacheable or noncacheable, depending on the decode width. For efficient use of the cache space, a sectored cache design is used for the DFC so that both cacheable and noncacheable instructions can coexist in the DFC sector. Experimental results show that the DFC reduces processor power by 34% on an average and our next fetch prediction mechanism reduces miss penalty by more than 91%.

Very small instruction caches have been shown to greatly reduce fetch energy. However, for many applications the use of a small filter cache can lead to an unacceptable increase in execution time. In this paper, we propose the tagless hit instruction cache (TH-IC), a technique for completely eliminating the performance penalty associated with filter caches, as well as a further reduction in energy consumption due to not having to access the tag array on cache hits. Using a few metadata bits per line, we are able to more efficiently track the cache contents and guarantee when hits will occur in our small TH-IC. When a hit is not guaranteed, we can instead fetch directly from the L1 instruction cache, eliminating any additional cycles due to a TH-IC miss. Experimental results show that the overall processor energy consumption can be significantly reduced due to the faster application running time and the elimination of tag comparisons for most of the accesses.

Leakage power in data cache memories represents a sizable fraction of total power consumption, and many techniques have been proposed to reduce it. As a matter of fact, during a fixed period of time, only a small subset of cache lines is used. Previous techniques put unused lines, for example, to drowsy state or switch them off completely (cache decay) in order to save power.

As transistors keep shrinking and on-chip data caches keep growing, static power dissipation due to leakage of caches takes an increasing fraction of total power in processors. Several techniques have already been proposed to reduce leakage power by turning off unused cache lines. However, they all have to pay the price of performance degradation. This paper presents a cache architecture, the snug set-associative (SSA) cache, that does not only cut most of static power dissipation but also reduces execution times. The SSA cache reduces leakage power by implementing the minimum set-associative scheme, which only activates the minimal numbers of ways in each cache set, while the performance losses incurred by this scheme are compensated by the base-offset load/store queues. These two techniques are both developed based on the principle of locality and they work together nicely - experimental results show that the minimum set-associative scheme can cut static power consumption of the L1 data cache by 90% on average for SPECint2000 benchmarks, while the execution times are reduced by 3% when the default 8-entry load/store queue is modified to the base-offset design. Furthermore, the SSA cache can trim the leakage power of L2 data cache by 96% on average while still accomplishing a 3% reduction in execution times.

A high bandwidth L-1 data cache is essential for achieving high performance in wide-issue processors. Previous studies have shown that using multiple small single-ported caches instead of a monolithic large multi-ported one for L-1 data cache can be a scalable and inexpensive way to provide higher bandwidth. Many schemes have been proposed on how to direct the memory references to these multiple caches in order to achieve a close match to the performance of an ideal multi-ported cache. However, most previous designs seldom take dynamic data access patterns into consideration and thus suffer from access conflicts within one cache and unbalanced loads between the caches. We observe that if one can group data references defined in a program into several regions (access regions) to allow parallel accesses, then providing separate small caches (access region cache) for these regions may prove to have better performance than previous multi-cache schemes. The register-guided memory reference partition approach proposed in this paper effectively identifies these semantic regions and organizes them in multiple caches in an adaptive way to maximize concurrent accesses without incurring too much overhead. In our design, the base register number, not its content, in the memory reference instruction is used as a basic guide for instruction steering. A reassignment mechanism is applied to capture the pattern when program is moving across its access regions. In addition, a distribution mechanism is introduced to further reduce residual conflicts, which adaptively enables access regions to extend or shrink among the physical caches. Our simulations of SPEC CPU2000 benchmarks have shown that the register-guided approach can reduce the conflicts effectively, distribute memory reference instructions properly, and yield considerable performance improvement in terms of IPC.

Joint task offloading and data caching in mobile edge computing networks

When it comes to performance, embedded systems share many problems with their higher-end counterparts. The growing gap between top processor frequency and memory access speed, the memory wall, is one such problem. Driven, in part, by low energy consumption and low cost requirements, embedded systems are often customized to a single application, or a very small set of applications. In addition, time-to-market requirements and the increasing complexity of embedded systems drives the need for fully or partially automated design tools and also to the extensive use of caches and cache hierarchies. The introduction of multi-processor-based embedded platforms has accelerated this trend; as the design space for embedded systems has grown, designers have become unclear as to whether automatic processor customization tools can cope with this increased complexity. The recent introduction of new techniques addressing the automatic customization, such as Architecturally Visible Storage (AVS) memory-enhanced Instruction Set Extension (ISE) identification algorithms, has also created new challenges. AVS memories are distinct from the cache hierarchy and rely on Direct Memory Access (DMA) transfers to communicate with main memory. In an embedded system containing hardware-managed caches, these extra AVS memories, in combination with their corresponding DMA transfers, cause coherence and consistence problems. Although the problems of coherence and consistence are well known in multi-processor systems, conventional solutions may be expensive in terms of area and power consumption, rendering them unacceptable for use in embedded systems. This thesis presents two low cost coherence mechanisms that solve these two problems. The first mechanism addresses embedded systems that already contain a hardware coherence protocol, like many high-end embedded multi-processor systems. Traditionally, the DMA transactions are transparent to the hardware coherence protocol. By ensuring visibility of these DMA transactions to the hardware coherence protocol, coherence can be guaranteed between AVS memories and data cache(s). As a result, minor changes to the DMA engine are required. Moreover, by forcing the processor pipeline to stall if a DMA transfer is active, memory consistence can be guaranteed. This mechanism provides significant speedup when compared to the execution of a non-ISE-enhanced system; however, due to the increase in bus traffic, this speedup comes at the expense of an increase in energy consumption. Coherent and Speculative DMA are both implementations of this mechanism. Single-processor systems do not contain hardware coherence protocols, and would therefore benefit from a lower-cost solution to the coherence and consistence problems than a hardware coherence protocol. By tightly coupling the AVS memories to the hardware cache, coherence and consistence for the complete system can be guaranteed. This coupling requires insignificant changes to the hardware cache's hit detection circuitry and state machine without influencing its critical path, thus it is inherently inexpensive. This mechanism provides significant speedups and reduces the energy consumed if compared to the execution on non-ISE-enhanced systems. Furthermore, the tight coupling enables direct communication between the AVS memories and the data cache, making this mechanism independent from the processor-to-memory distance. Virtual Ways and Way Stealing are both implementations of this mechanism. Besides enforcing coherence and consistence, the ability to integrate the architectural changes into an automated design flow is important. This thesis shows the influence of Coherent DMA, Speculative DMA, Virtual Ways, and Way Stealing on the ISE-identification algorithm. It shows the architectural requirements and the cost for enforcing coherence and consistence that need to be taken into account when applying these mechanisms in an automated flow without formulating new algorithms.

Tuning cache hierarchies in platforms for embedded systems can significantly reduce energy consumption. In this paper we combined two optimization methods for tuning both instruction and data cache configurations in a two-level memory hierarchy, where both levels have separate instruction and data caches. This kind of hierarchy allows us to evaluate instruction and data caches branches separately, although previous approaches have applied the same method for both branches of the hierarchy. This work evaluates several methods intended for two-level hierarchies, and the results showed that when we combine different methods for each branch of the hierarchy, results can be improved. Experiments based on simulations were performed for 12 applications from the Mibench suite benchmark and the combined method achieved better efficiency in 60% of the evaluated cases compared with existing heuristics. The proposed solution is only 11% less economic in terms of energy consumption than optimal values and required, on average, 42 simulations to conclude optimization mechanism, representing only 9% of the design space.

In this paper, we present the characterization and design of energy-efficient, on chip cache memories. The characterization of power dissipation in on-chip cache memories reveals that the memory peripheral interface circuits and bit array dissipate comparable power. To optimize performance and power in a processor's cache, a multidivided module (MDM) cache architecture is proposed to conserve energy in the bit array as well as the memory peripheral circuits. Compared to a conventional, nondivided, 16-kB cache, the latency and power of the MDM cache are reduced by a factor of 1.9 and 4.6, respectively. Based on the MDM cache architecture, the energy efficiency of the complete memory hierarchy is analyzed with respect to cache parameters in a multilevel processor cache design. This analysis was conducted by executing the SPECint92 benchmark programs with the miss ratios for reduced instruction set computer (RISC) and complex instruction set computer (CISC) machines.

The L1 data cache is a time-critical module and, at the same time a major consumer of energy. To reduce its energy-delay product, we apply two principles of low-power design: specialize part of the cache structure and break the cache down into smaller caches. To this end, we propose a new L1 data cache structure that combines a Specialized Stack Cache (SSC) and a Pseudo Set-Associative Cache (PSAC). Individually, our SSC and PSAC designs have a lower energy-delay product than previously-proposed related designs. In addition, their combined operation is very effective. Relative to a conventional 2-way 32 KB data cache, a design containing a 4-way 32 KB PSAC and a 512 B SSC reduces the energy-delay product of several applications by an average of 44%.