Impact of Cache Architecture and Interface on Performance and Area of FPGA-Based Processor/Parallel-Accelerator Systems

The authors earlier proposed a new space partitioning for mapping computations of the radiosity method onto a highly pipelined parallel architecture (L.S. Shen et al., 1990; 1991). This shelling technique can alleviate the communication load between the host and the processors in a shared-memory architecture, but the system performance might deteriorate when increasing the number of processors. A memory structure which is a hierarchy of resident set, cache, and main memory is presented. It can reduce the average access time of a patch and thus provide better balancing between processing throughput and memory bandwidth, to enhance the pipelinability of computations. A method of selecting the resident set is described, and different policies in the cache design are investigated. >

As the number of instructions executed in parallel increases, superscalar processors will require higher bandwidth from data caches. Because of the high cost of true multi-ported caches, alternative cache designs must be evaluated. The purpose of this study is to examine the data cache bandwidth requirements of high-degree superscalar processors, and investigate alternative solutions. The designs studied range from classic solutions like multi-banked caches to more complex solutions recently proposed in the literature. The performance tradeoffs of these different cache designs are examined in details. Then, using a chip area cost model, all solutions are compared with respect to both cost and performance. While many cache designs seem capable of achieving high cache bandwidth, the best cost/performance t,radeoff varies significantly depending on the dedicated area cost, ranging from multi-banked cache designs to hybrid multi-banked/multi-ported caches or even true multi-ported caches. For instance, we find that an 8-bank cache with minor optimizations perform 10% better than a true a-port cache at half the cost, or that a 4-bank 2 ports per bank cache performs better than a true 4-port cache and uses 45% less chip area.

Mobile devices are quickly becoming the most widely used processors in consumer devices. Since their major power supply is battery, energy-efficient computing is highly desired. In this article, we focus on energy-efficient cache design in emerging mobile platforms. We observe that more than 40% of L2 cache accesses are OS kernel accesses in interactive smartphone applications. Such frequent kernel accesses cause serious interferences between the user and kernel blocks in the L2 cache, leading to unnecessary block replacements and high L2 cache miss rate. We first propose to statically partition the L2 cache into two separate segments, which can be accessed only by the user code and kernel code, respectively. Meanwhile, the overall size of the two segments is shrunk, which reduces the energy consumption while still maintaining the similar cache miss rate. We then find completely different access behaviors between the two separated kernel and user segments and explore the multi-retention STT-RAM-based user and kernel segments to obtain higher energy savings in this static partition-based cache design. Finally, we propose to dynamically partition the L2 cache into the user and kernel segments to minimize overall cache size. We also integrate the short-retention STT-RAM into this dynamic partition-based cache design for maximal energy savings. The experimental results show that our static technique reduces cache energy consumption by 75% with 2% performance loss, and our dynamic technique further shows strong capability to reduce cache energy consumption by 85% with only 3% performance loss.

Deep convolutional neural networks (CNNs) have recently shown very high accuracy in a wide range of cognitive tasks, and due to this, they have received significant interest from the researchers. Given the high computational demands of CNNs, custom hardware accelerators are vital for boosting their performance. The high energy efficiency, computing capabilities and reconfigurability of FPGA make it a promising platform for hardware acceleration of CNNs. In this paper, we present a survey of techniques for implementing and optimizing CNN algorithms on FPGA. We organize the works in several categories to bring out their similarities and differences. This paper is expected to be useful for researchers in the area of artificial intelligence, hardware architecture and system design.

Domain-specific systems-on-chip, a class of heterogeneous many-core systems, are recognized as a key approach to narrow down the performance and energy-efficiency gap between custom hardware accelerators and programmable processors. Reaching the full potential of these architectures depends critically on optimally scheduling the applications to available resources at runtime. Existing optimization-based techniques cannot achieve this objective at runtime due to the combinatorial nature of the task scheduling problem. As the main theoretical contribution, this paper poses scheduling as a classification problem and proposes a hierarchical imitation learning (IL)-based scheduler that learns from an Oracle to maximize the performance of multiple domain-specific applications. Extensive evaluations with six streaming applications from wireless communications and radar domains show that the proposed IL-based scheduler approximates an offline Oracle policy with more than 99% accuracy for performance- and energy-based optimization objectives. Furthermore, it achieves almost identical performance to the Oracle with a low runtime overhead and successfully adapts to new applications, many-core system configurations, and runtime variations in application characteristics.

3 Chapter - Efficient packet classification with digest caches

Energy efficiency for high-performance computing has become the main concern in many scientific applications on modern core architecture. Although the performance of most applications might be increased by optimizing the thread-level parallelism and multi-processors, efficient data-level parallelism becomes an important factor for large matrix-matrix multiplication. Most modern computers can calculate vectorization using single-instruction multiple-data execution (SIMD) by applying Advanced Vector Extensions (AVX) to enhance parallel data processing. In this paper, we focus on the effect of the loop unrolling methods for Strassen's algorithm. When the recursive levels are increased energy consumption becomes an issue related to longer execution time. For this reason, we contributed an efficient loop-unrolling method to save energy consumption by reducing the cache misses and increasing the data locality. Moreover, two different loop-unrolling methods in different recursive levels for the square matrix sizes range from 1024 to 16384 are compared. According to experimental results, the proposed loop unrolling methods reduce the number of cache misses and data transferring from the main memory which leads to saving energy/power consumption for Green computing. The proposed energy-efficient Strassen's algorithm gains not only the improved performance by nearly (98%) but also save energy consumption by nearly (95%) than without unrolling method, respectively.

Domain-specific systems improve the performance of specific applications compared to general-purpose processing systems by deploying custom hardware accelerators. These hardware accelerators are generated using high-level synthesis (HLS) tools. The HLS tools enable a comprehensive design space exploration, optimizing the accelerators' compute performance. However, they often ignore the challenges of implementing the accelerators in a system-on-chip, particularly how they access memory. Our work introduces a buffering system design that improves accelerators' memory accesses by intelligently employing burst transactions to prefetch useful data from external memory to on-chip local buffers. Our design is dynamic, parametric, and transparent to the accelerators generated by HLS tools. We derive the buffering system parameters using appropriate compiler-based analysis passes and memory channel latency constraints. The proposed buffering system design results in, on average, 8.8× performance improvements while lowering memory channel utilization by 53.2% for a set of PolyBench kernels.

We propose an asymmetric multi-processor reconflgurable SoC architecture comprised of a master CPU running embedded Linux and loosely-coupled slave CPUs executing dedicated software processes. The slave processes are mapped into the host OS as ghost processes, and are able to communicate with each other and the master via standard operating system communication abstractions. Custom hardware accelerators can be also added to the slave or master CPUs. We describe an architectural case study of an MP3 decoding application of 12 different single and multi-CPU configurations, with and without custom hardware. Analysis of system performance under master CPU load (computation and IO), and a time-area cost model reveals the counter-intuitive result that multiple CPUs and appropriate software partitioning can lead to more efficient and load-resilient architecture than a single CPU with custom hardware offload capabilities, at a lower design cost.

With the rise of machine learning and data analytics, the ability to process large and diverse sets of data efficiently has become crucial. Research has shown that data transformation is a key performance bottleneck for applications across a variety of domains, from data analytics to scientific computing. Custom hardware accelerators and GPU implementations targeting specific data transformation tasks can alleviate the problem, but suffer from narrow applicability and lack of generality.To tackle this problem, we propose a GPU-accelerated data transformation engine grounded on pushdown transducers. We define an extended pushdown transducer abstraction (effPDT) that allows expressing a wide range of data transformations in a memory-efficient fashion, and is thus amenable for GPU deployment. The effPDT execution engine utilizes a data streaming model that reduces the application’s memory requirements significantly, facilitating deployment on high- and low-end systems. We showcase our GPU-accelerated engine on a diverse set of transformation tasks covering data encoding/decoding, parsing and querying of structured data, and matrix transformation, and we evaluate it against publicly available CPU and GPU library implementations of the considered data transformation tasks. To understand the benefits of the effPDT abstraction, we extend our data transformation engine to also support finite state transducers (FSTs), we map the considered data transformation tasks on FSTs, and we compare the performance and resource requirements of the FST-based and the effPDT-based implementations.

A typical machine learning development cycle maximizes performance during model training and then minimizes the memory and area footprint of the trained model for deployment on processing cores, graphics processing units, microcontrollers or custom hardware accelerators. However, this becomes increasingly difficult as machine learning models grow larger and more complex. Here we report a methodology for automatically generating predictor circuits for the classification of tabular data. The approach offers comparable prediction performance to conventional machine learning techniques as substantially fewer hardware resources and power are used. We use an evolutionary algorithm to search over the space of logic gates and automatically generate a classifier circuit with maximized training prediction accuracy, which consists of no more than 300 logic gates. When simulated as a silicon chip, our tiny classifiers use 8–18 times less area and 4–8 times less power than the best-performing machine learning baseline. When implemented as a low-cost chip on a flexible substrate, they occupy 10–75 times less area, consume 13–75 times less power and have 6 times better yield than the most hardware-efficient ML baseline.

It is generally accepted that a custom hardware implementation of a set of computations will provide superior speed and energy efficiency relative to a software implementation. However, the cost and difficulty of hardware design is often prohibitive, and consequently, a software approach is used for most applications. In this article, we introduce a new high-level synthesis tool calledLegUpthat allows software techniques to be used for hardware design. LegUp accepts a standard C program as input and automatically compiles the program to a hybrid architecture containing an FPGA-based MIPS soft processor and custom hardware accelerators that communicate through a standard bus interface. In the hybrid processor/accelerator architecture, program segments that are unsuitable for hardware implementation can execute in software on the processor. LegUp can synthesize most of the C language to hardware, including fixed-sized multidimensional arrays, structs, global variables, and pointer arithmetic. Results show that the tool produces hardware solutions of comparable quality to a commercial high-level synthesis tool. We also give results demonstrating the ability of the tool to explore the hardware/software codesign space by varying the amount of a program that runs in software versus hardware. LegUp, along with a set of benchmark C programs, is open source and freely downloadable, providing a powerful platform that can be leveraged for new research on a wide range of high-level synthesis topics.

Providing adequate data bandwidth is extremely important for a wide-issue superscalar processor to achieve its full performance potential. Adding a large number of ports to a data cache, however, becomes increasingly inefficient and can add to the hardware complexity significantly. This paper takes an alternative or complementary approach for providing more data bandwidth, called the data-decoupled architecture. The approach, with support from the compiler and/or hardware, partitions the memory stream into two independent streams early in the processor pipeline, and feeds each stream to a separate memory access queue and cache. Under this model, the paper studies the potential of decoupling memory accesses to program's local variables that are allocated on the run-time stack. Using a set of integer and floating-point programs from the SPEC95 benchmark suite, it is shown that local variable accesses constitute a large portion of all the memory references, while their reference space is very small, averaging around 7 words per (static) procedure. To service local variable accesses quickly, two optimizations, fast data forwarding and access combining, are proposed and studied. Some of the important design parameters, such as the cache size, the number of cache ports, and the degree of access combining, are studied based on simulations. The potential performance of the proposed scheme is measured using various configurations, and it is concluded that the scheme can become a viable alternative to building a single multi-ported data cache.

The continued growth of Internet traffic associated with the popularity of online social networks (OSNs) raises the possibility that knowledge of the underlying social graph connecting users may be exploited to enhance content distribution and delivery. In our previous work, we found that partitioning OSN users and their data based on their social graph, and using the partitioning as basis for content placement decisions across cache servers yield significant improvements over randomly-partitioned implementations. In this paper we go beyond content placement and use the same partitioning scheme to push updates across distributed caches to maintain consistency in OSN state. To avoid making singular assumptions on the degree to which user interaction patterns are influenced either by social considerations or content popularity, we parameterized our experiments to determine the impact of various interaction mixes on different cache designs. We found that in the new hybrid scheme that combines socially-aware cooperative content placement, on one hand, with social graph-driven proactive content delivery on the other, significant performance benefit continues to be driven by the former rather than the latter. Consequently, the hybrid scheme is likewise sensitive to the granularity of graph partitioning, which in turn dictates cache sizes and the number of cache servers.

Validation and debug challenges of system-on-chips (SoCs) are getting increasingly difficult. As we reach the limits of Dennard scaling, efforts to improve system performance and energy efficiency have resulted in the integration of a wide variety of complex hardware accelerators in SoCs. Hence, it is essential to address the validation and debug of hardware accelerators. High-level synthesis (HLS) is a promising technique to rapidly create customized hardware accelerators. In this paper, we present the hybrid quick error detection ( H -QED) approach that overcomes validation and debug challenges for hardware accelerators by leveraging HLS techniques in both the presilicon and post-silicon stages. H-QED improves error detection latencies (time elapsed from when a bug is activated to when it is detected) by 2–5 orders of magnitude with one cycle latencies in presilicon scenarios and bug coverage threefold higher compared to traditional validation techniques. H-QED also uncovered previously unknown bugs in the CHStone benchmark suite, which is widely used by the HLS community. H-QED incurs an 8% accelerator area overhead with negligible silicon performance impact for post-silicon stage, and we also introduce techniques to minimize any possible intrusiveness introduced by H-QED.