Improving performance of codes with large/irregular stride memory access patterns via high performance reconfigurable computers

High performance reconfigurable computers (HPRCs), which combine general-purpose processors (GPPs) and field programmable gate arrays (FPGAs), are now commercially available. These interesting architectures allow for the creation of reconfigurable processors. HPRCs have already been used to accelerate integer and fixed-point applications. However, extensive parallelism and deeply pipelined floating-point cores are necessary to make MHz-scale FPGAs competitive with GHz-scale GPPs, thus making it difficult to accelerate certain kinds of floating-point kernels. Kernels with variable length nested loops, e.g., sparse matrix-vector multiply, have been problematic because of the loop-carried dependence associated with the pipelined floating-point units. While hardware description language (HDL)-based kernels have shown moderate success in addressing this problem, the use of a high-level language (HLL)-based approach to accelerate such applications has been rather elusive. If HPRCs are to become a part of mainstream military and scientific computing, we should emphasize the use of HLL-based programming, whenever possible, rather than HDL-based hardware design. The primary reason is the increased programmer productivity associated with HLLs when compared with HDLs. For example, the floating-point addition statement z = x+y, a single line in an HLL, corresponds to hundreds of lines of HDL. In this paper, we describe the design and implementation of a sparse matrix Jacobi processor to solve systems of linear equations, Ax=b. The parallelized, deeply pipelined, IEEE-754-compliant 32-bit floating-point sparse matrix Jacobi iterative solver runs on a contemporary HPRC. The FPGA-based components are implemented using only an HLL (the C programming language) and the Carte HLL-to-HDL compiler. An HLL-based streaming accumulator allows for the implementation of fully pipelined loops and results in a 2.5-fold wall clock runtime speedup when compared with an equivalent software-only implementation.

The increasing computational requirements of today's software systems have led researchers to investigate ways of accelerating military and scientific computing applications. Contemporary field programmable gate arrays (FPGAs) are now equipped with multimillion gate logic fabrics, faster clock rates, reasonably large on-chip memory, and fast I/O resources for off-chip communication. The use of FPGAs as reconfigurable computational units complementing a fixed computational device such as a general-purpose processor (GPP) is the basic idea behind what are known as high performance reconfigurable computers (HPRCs). These exciting architectures allow development of reconfigurable processors that target the computationally intensive parts of a given application. Ideally, one should use a high-level language (HLL) rather than a hardware description language (HDL) to implement HPRC-based applications. However, in order to accelerate some applications, an HDL must be used to design computational kernels. The HPRC used in the joint research project between the U.S. Army Engineer Research and Development Center {DoD Supercomputing} Resource Center (ERDC DSRC) and Jackson State University (JSU) employs the SRC Computers' Carte development environment. Carte allows application development using a conventional HLL, an HLL-to-HDL compiler, and custom-built VHDL-based kernels ("user macros" in SRC parlance). Currently, the off-the-shelf Carte mechanism for incorporating user macros does not directly support the common case of a multiple file VHDL hierarchy. This research explores a novel approach that allows multiple file VHDL kernels to be mapped onto the SRC-7 HPRC. The approach facilitates the development of FPGA-based elements via a hybrid technique that uses the Carte HLL-to-HDL compiler in conjunction with multiple file VHDL-based user macros. This paper describes the use of this novel approach to map a parameterized, parallelized, and pipelined FPGA-based sparse matrix vector multiply kernel onto an SRC-7 HPRC. The HPRC-based version runs nearly four times faster than the software-only version.

In this paper we describe an FPGA implementation for solving Internet ranking algorithms. Sparse Matrix by Vector multiplication forms a major part of these algorithms. Due to memory bandwidth problems, general purpose processors only achieve a fraction of peak processing power when dealing with sparse matrices. Field-Programmable Gate Arrays (FPGAs) have the potential to significantly improve this performance. In order to support real-life Internet ranking problems a large memory is needed to store the sparse matrix. This memory requirement cannot be fulfilled with the FPGA's on-board block-RAM and SDRAM is expensive and limited in size. Commodity memory such as Double Data Rate (DDR) SDRAM is cheap and fully scalable and would appear to be the best solution. This paper discusses the possibility of a custom architecture on FPGA, for Internet ranking algorithms, using DDR SDRAM. An overview of work to date is also presented as well as a plan for future work.

This thesis work presents the implementation and validation of image processing problems in hardware to estimate the performance and precision gain. It compares the implementation for the addressed problem on a Field Programmable Gate Array (FPGA) with a software implementation for a General Purpose Processor (GPP) architecture. For both solutions the implementation costs for their development is an important aspect in the validation. The analysis of the flexibility and extendability that can be achieved by a modular implementation for the FPGA design was another major aspect. This work is based upon approaches from previous work, which included the detection of Binary Large OBjects (BLOBs) in static images and continuous video streams [13, 15]. One addressed problem of this work is the tracking of the detected BLOBs in continuous image material. This has been implemented for the FPGA platform and the GPP architecture. Both approaches have been compared with respect to performance and precision. This research project is motivated by the MI6 project of the Computer Vision research group, which is located at the Bonn-Rhein-Sieg University of Applied Sciences. The intent of the MI6 project is the tracking of a user in an immersive environment. The proposed solution is to attach a light emitting device to the user for tracking the created light dots on the projection surface of the immersive environment. Having the center points of those light dots would allow the estimation of the user’s position and orientation. One major issue that makes Computer Vision problems computationally expensive is the high amount of data that has to be processed in real-time. Therefore, one major target for the implementation was to get a processing speed of more than 30 frames per second. This would allow the system to realize feedback to the user in a response time which is faster than the human visual perception. One problem that comes with the idea of using a light emitting device to represent the user, is the precision error. Dependent on the resolution of the tracked projection surface of the immersive environment, a pixel might have a size in cm2. Having a precision error of only a few pixels, might lead to an offset in the estimated user’s position of several cm. In this research work the development and validation of a detection and tracking system for BLOBs on a Cyclone II FPGA from Altera has been realized. The system supports different input devices for the image acquisition and can perform detection and tracking for five to eight BLOBs. A further extension of the design has been evaluated and is possible with some constraints. Additional modules for compressing the image data based on run-length encoding and sub-pixel precision for the computed BLOB center-points have been designed. For the comparison of the FPGA approach for BLOB tracking a similar implementation in software using a multi-threaded approach has been realized. The system can transmit the detection or tracking results on two available communication interfaces, USB and RS232. The analysis of the hardware solution showed a similar precision for the BLOB detection and tracking as the software approach. One problem is the strong increase of the allocated resources when extending the system to process more BLOBs. With one of the applied target platforms, the DE2-70 board from Altera, the BLOB detection could be extended to process up to thirty BLOBs. The implementation of the tracking approach in hardware required much more effort than the software solution. The design of high level problems in hardware for this case are more expensive than the software implementation. The search and match steps in the tracking approach could be realized more efficiently and reliably in software. The additional pre-processing modules for sub-pixel precision and run-length-encoding helped to increase the system’s performance and precision.

Motion estimation in videos is a computationally intensive process. A popular strategy for dealing with such a high processing load is to accelerate algorithms with dedicated hardware such as graphic processor units (GPU), field programmable gate arrays (FPGA), and digital signal processors (DSP). Previous approaches addressed the problem using accelerators together with a general purpose processor, such as acorn RISC machines (ARM). In this work, we present a co-processing architecture using FPGA and DSP. A portable platform for motion estimation based on sparse feature point detection and tracking is developed for real-time embedded systems and smart video sensors applications. A Harris corner detection IP core is designed with a customized fine grain pipeline on a Virtex-4 FPGA. The detected feature points are then tracked using the Lucas---Kanade algorithm in a DSP that acts as a co-processor for the FPGA. The hybrid system offers a throughput of 160 frames per second (fps) for VGA image resolution. We have also tested the benefits of our proposed solution (FPGA + DSP) in comparison with two other traditional architectures and co-processing strategies: hybrid ARM + DSP and DSP only. The proposed FPGA + DSP system offers a speedup of about 20 times and 3 times over ARM + DSP and DSP only configurations, respectively. A comparison of the Harris feature detection algorithm performance between different embedded processors (DSP, ARM, and FPGA) reveals that the DSP offers the best performance when scaling up from QVGA to VGA resolutions.

Numerous studies have shown that field-programmable gate arrays (FPGAs) often achieve large speedups compared to microprocessors. However, one significant limitation of FPGAs that has prevented their use on important applications is the requirement for regular memory access patterns. Traversal caches were previously introduced to improve the performance of FPGA implementations of algorithms with irregular memory access patterns, especially those traversing pointer-based data structures. However, a significant limitation of previous traversal caches is that speedup was limited to traversals repeated frequently over time, thus preventing speedup for algorithms without repetition, even if the similarity between traversals was large. This paper presents a new framework that extends traversal caches to enable performance improvements in such cases and provides additional improvements through reduced memory accesses and parallel processing of multiple traversals. Most importantly, we show that, for algorithms with highly similar traversals, the traversal cache framework achieves approximately linear kernel speedup with additional area, thus eliminating the memory bandwidth bottleneck commonly associated with FPGAs. We evaluate the framework using a Barnes-Hut n-body simulation case study, showing application speedups ranging from 12x to 13.5x on a Virtex4 LX100 with projected speedups as high as 40x on today’s largest FPGAs.

Software-based radios implemented on general-purpose processors are cheaper, easier, and faster to develop, maintain, and upgrade than hardware-based equivalents. Unfortunately, today's general-purpose processors are not fast enough to process certain waveforms. The current alternative is to use application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs) for all the signal processing. These operate at very high speeds but are dramatically more expensive and require lengthy development cycles. A single, often small, block is responsible for the bulk of signal processing in a large class of radios. It is therefore very wasteful to implement all the signal processing in a high-speed device such as an ASIC or FPGA. In this paper, we present a software-centric architecture that offloads only the most computationally expensive tasks to an FPGA. The resultant system combines the advantages of both platforms, while minimizing the disadvantages of each. An open-source software radio platform is combined with a consumer off-the-shelf (COTS) FPGA development board to create a hardware-accelerated multichannel receiver. The FPGA is efficiently utilized by partitioning the device into multiple accelerator regions and taking advantage of runtime partial reconfiguration to reconfigure each region as needed during operation. Comparisons between a software-only receiver and a hardware-accelerated implementation are performed.

The literature review explores the evolution and advancements in the implementation of latency-based solvers for real-time simulation of power electronic systems using field-programmable gate arrays (FPGAs).The journey begins with Dr. Johnson, who highlights the exceptional capability of FPGAs to outperform traditional general-purpose processors in floating-point throughput for power flow computations.[1] This foundational work sets the stage for subsequent research that delves deeper into the acceleration of various computational tasks using FPGA technology.As the field continues to mature, new methodologies and algorithms are developed to harness the full potential of FPGAs.These innovations aim to not only improve the performance of power electronic simulations, but also to enhance the flexibility and reconfigurability of the systems.Researchers are exploring novel approaches such as adaptive solvers, where the latency of the system dynamically adjusts based on the complexity of the simulation.[2] This adaptive nature allows for efficient resource allocation and utilization in real-time applications, leading to improved overall system performance.Furthermore, advancements in FPGA architecture and design techniques enable the integration of multiple solvers on a single chip, catering to the increasing demand for multi-domain simulations.This integration paves the way for seamless co-simulation of diverse power electronic systems, including converters, inverters, and motor drives.Additionally, the emergence of high-level synthesis tools simplifies the development process, making it more accessible to researchers and practitioners alike.[3] These tools enable the conversion of high-level hardware description languages into FPGA-specific designs, alleviating the need for manual low-level coding.Moreover, the availability of FPGA development boards and development kits provides a testing ground for prototype implementations, allowing researchers to evaluate their designs in a real-world scenario.[1] In conclusion, the literature review highlights the evolutionary journey of latency-based solvers for real-time power electronic simulations using FPGAs.From the initial groundbreaking work to the current state-of-the-art advancements, researchers have continuously pushed the boundaries to achieve higher performance and accuracy.With ongoing research and innovation, the field is poised for further growth and development, promising even more efficient and versatile solutions for real-time power electronic simulations.[4] [5] build on this foundation by addressing the parallelization of the sparse Matrix-Solve phase in SPICE applications.Their findings indicate that reduced-precision floating-point datapaths can significantly enhance performance, achieving speedups of 2-18 times over single-core microprocessor implementations.This article emphasizes the importance of exploiting parallelism and optimizing resource usage in FPGA designs.The authors present an FPGA-based framework that effectively tackles and addresses the inherent challenges associated with simulating intricate physical phenomena utilizing unstructured spatial discretization.This innovative and pioneering approach significantly enhances data locality and memory access patterns, thereby showcasing a remarkable and astonishing 90 times speedup as compared to high-performance microprocessors.[1] The demonstrated outcomes and results of this work serve to underscore and emphasize the critical significance of efficient and effective memory management when harnessing and leveraging the immense computational power offered by FPGAs.This research proves to be a groundbreaking and pivotal advancement in the field, with far-reaching implications and potential for further exploration and application in diverse areas of study.[6] Introduce FPGA-SPICE, which is a state-of-the-art simulation-based power estimation framework that perfectly integrates seamlessly with cutting-edge architecture exploration tools.

There is still no partial reconfiguration tool support on low-cost Field Programmable Gate Arrays (FPGAs) such as old-fashioned Spartan-3 and state-of-the-art Spartan-6 FPGA families by Xilinx. This forces the designers and engineers, who are using the partial reconfiguration capability of FPGAs, to use expensive families such as Virtex-4, Virtex-5 and Virtex-6 which are officially supported by partial reconfiguration (PR) software. Moreover, Xilinx still does not offer a portable, dedicated self-reconfiguration engine for all of the FPGAs. Self-reconfiguration is achieved with general-purpose processors such as MicroBlaze and PowerPC which are too overqualified for this purpose. In this study, we propose a new self-reconfiguration mechanism for Spartan-6 FPGAs. This mechanism can be used to implement large and complex designs on small FPGAs as chip area can be dramatically reduced by exploiting the dynamic partial reconfiguration feature for on-demand functionality loading and maximal utilization of the hardware. This approach is highly attractive for designing low-cost compute-intensive applications such as high performance image processing systems. For Spartan-6 FPGAs, we have developed hard-macros and exploited the self-reconfiguration engine, compressed Parallel Configuration Access Port (cPCAP) [1], that we designed for Spartan-3. The modified cPCAP core with block RAM controller, bitstream decompressor unit and Internal Configuration Access Port (ICAP) Finite State Machine (FSM) occupies only about 82 of 6,822 slices (1.2% of whole device) on a Spartan-XC6SLX45 FPGA and it achieves the maximum theoretical reconfiguration speed of 200MB/s (ICAP, 16-bit at 100MHz) proposed by Xilinx. We have also implemented a Reconfigurable Processing Element (RPE) whose arithmetic unit can be reconfigured on-the-fly. Multiple RPEs can be utilized to design a General Purpose Image Processing System (GPIPS) that can implement a number of different algorithms during runtime. As an illustrative example, we programmed the GPIPS on Spartan-6 for switching between two applications on-demand such as two-dimensional filtering and block-matching.

Many problems involving network design can be found in the real world, such as electric power circuit planning, telecommunications and phylogenetic trees. In general, solutions for these problems are modeled as forests represented by a graph manipulating thousands or millions of input variables, making it hard to obtain the solutions in a reasonable time. To overcome this restriction, Evolutionary Algorithms (EAs) with dynamic data structures (encodings) have been widely investigated to increase the performance of EAs for Network Design Problems (NDPs). In this context, this paper proposes a parallelization of the node-depth encoding (NDE), a data structure especially designed for NDPs. Based on the NDE the authors have developed a parallel algorithm and a hardware architecture implemented on FPGA (Field-Programmable Gate Array), denominated Hardware Parallelized NDE (HP-NDE). The running times obtained in a general purpose processor (GPP) and the HP-NDE are compared. The results show a significant speedup in relation to the GPP solution, solving NDP in a time limited by a constant. Such time upper bound can be satisfied for any size of network until the hardware resources available on the FPGA are depleted. The authors evaluated the HP-NDE on a Stratix IV FPGA with networks containing up to 2048 nodes.

K-nearest neighbour (KNN) is a supervised classification technique that is widely used in many fields of study to classify unknown queries based on some known information about the dataset. KNN is known to be robust and simple to implement when dealing with data of small size. However it performs slowly when data are large and have high dimensions. Therefore, KNN classifiers can benefit from the parallelism offered by Field Programmable Gate Arrays (FPGAs) to accelerate the algorithm. In addition, the KNN classifier is sensitive to the user defined parameter (K) which is the number of nearest neighbours. This parameter is known to affect the performance of the classifier; thus users would want the classifier to be easily adaptable to different values of K. In this work, we propose two adaptive FPGA architectures of the KNN classifier, and compare the implementations of each one of them with an equivalent implementation running on a general purpose processor (GPP). The proposed hardware implementations outperformed GPP by 76× for the first architecture and 68× for the second. In addition, we propose a novel dynamic partial reconfiguration (DPR) architecture of the KNN classifier, which allows for an efficient dynamic partial reconfiguration of a classifier implementation on FPGA for different K's. This DPR implementation offers 5× speed-up in the reconfiguration time of a KNN classifier on FPGA.

Due to the rapid growth of Internet, there is an increasing need for efficiently classifying packets with many header fields in large rule sets. For example, in Software Defined Networking (SDN), the OpenFlow table lookup can require 15 packet header fields to be examined. In this paper, we present several decomposition-based packet classification implementations with efficient optimization techniques. In the searching phase, packet headers are split or combined. In the merging phase, the partial searching results from all the fields are merged to generate the final result. We prototype our implementations on state-of-the-art Field Programmable Gate Array (FPGA), multi-core General Purpose Processor (GPP), and Graphics Processing Unit (GPU). On FPGA, we propose two optimization techniques to divide generic ranges; modular processing elements are constructed and concatenated into a systolic array. On multi-core GPP, we parallelize both the searching and merging phases using parallel program threads. On the GPU-accelerated platform, we minimize branch divergence and reduce the data communication overhead. Experimental results show that 500 Million Packets Per Second (MPPS) throughput and 3 us latency can be achieved for 1.5 K rule sets on FPGA. We achieve 14.7 MPPS throughput and 30.5 MPPS throughput for 32 K rule sets on multi-core GPP and GPU-accelerated platforms, respectively. As a heterogeneous solution, our GPU-accelerated packet classifier shows 2X speedup compared to the implementation using multi-core GPP only. Compared with prior works, our designs can match long packet headers against very complex rule sets.

Modern processors rely on advanced memory management techniques such as predicting and prefetching to bridge the processor–memory gap. However, current mechanisms are primarily designed for a specific memory hierarchy level or access pattern, resulting in inefficiencies for diverse workloads with mixed memory access patterns. To address this challenge, this paper proposes DHCM, a Dynamic Hierarchy Coordination Mechanism that intelligently schedules prediction hierarchies and dynamically optimizes memory access processes to enhance system performance. DHCM integrates a hierarchy coordination mechanism driven by a State Trigger. This mechanism dynamically leverages system feedback to prioritize and coordinate memory operations, enabling the simultaneous management of both off-chip load requests and on-chip cache accesses. Through extensive evaluations on the ChampSim simulator, DHCM demonstrates its adaptability and efficiency with an average IPC improvement of 34.08% and 24.09% on single-core and multi-core systems, respectively. Additionally, DHCM contributes a 64.17% miss coverage and 89.33% DRAM-loads reduction.

Model-Based Iterative Reconstruction (MBIR) algorithms iteratively use expensive computational operators of forward and backward projections. The irregular memory access pattern of these operators makes them a memory-bound application. Their computation time must be reduced to meet clinical routine constraints. This paper proposes a hardware accelerator architecture based on Field Programmable Gate Arrays (FPGAs) through high-level language, as an alternative to GPU architecture. This acceleration is based on an offline memory access analysis to address the main bottleneck of the algorithm and maximize the data reuse rate. The offline analysis allows for the tuning of the architecture parameters so that they converge to an optimal solution. Then, the Berkeley Roofline model guides our optimization steps by iteratively analyzing the design performance. Our design flow significantly improved the algorithm's computational intensity and overcame the memory bottleneck. Thus, our architecture takes advantage of the FPGA local memory to achieve significant memory bandwidth and efficiently harness the pipeline without stalling the computation. Furthermore, we present the scaling-up strategy from mid-range FPGA to high-end FPGA and any concerns of portability. We used two Intel FPGA devices to implement the algorithm, and then we compared the results with our GPU implementation in terms of speedup and energy efficiency. Our experimental results show that our design has achieved better computational throughput than the works on FPGA architectures reported in the literature.

In this paper, we propose a software shared virtual memory (SVM) system for heterogeneous multicore accelerator clusters with explicitly managed memory hierarchies. The target cluster consists of a single manager node and many compute nodes. The manager node contains a generalpurpose processor and larger main memory, and each compute node contains a heterogeneous multicore processor and smaller main memory. These nodes are connected with an interconnection network, such as Gigabit Ethernet. The heterogeneous multicore processor in each compute node consists of a general-purpose processor element (GPE) and multiple accelerator processor elements (APEs). The GPE runs an OS and the multiple APEs are dedicated to compute-intensive workloads. The GPE is typically backed by a deep on-chip cache hierarchy and hardware cache coherence. On the other hand, the APEs have small explicitly-addressed local memory instead of caches. This APE local memory is not coherent with the main memory. Different main and local memory units in the accelerator cluster can be viewed as an explicitly managed memory hierarchy: global memory, node local memory, and APE local memory. Since coherence protocols of previous software SVM proposals cannot effectively handle such a memory hierarchy, we propose a new coherence and consistency protocol, called hierarchical centralized release consistency (HCRC). Our software SVM system is built on top of HCRC and software-managed caches. We evaluate the effectiveness and analyze the performance of our software SVM system on a 32-node heterogeneous multicore cluster (a total of 192 APEs).