Design of OpenCL-compatible multithreaded hardware accelerators with dynamic support for embedded FPGAs

In this modern technology era, in System on Chip (SoC) design, performance of processing units in computer is always on demand. By having hardware accelerators in computer system, core processor can offload task to it and this creates parallel execution to improve the processing speed. One of the functional blocks that are crucial in Hardware Accelerator's design is Storage Unit, which is used to keep data that is needed for processing or either the processed data. Under conventional hardware accelerator's design, the storage architecture is shaped to suit a certain processing algorithm and this introduced less flexibility in SoC design. In this paper, a novel design of storage architecture that is able to handle multiple accelerator engines and also modular to typical specification of hardware accelerators has been implemented.

Design of exact reduct rough set hardware accelerator for early-stage diabetes risk prediction

With the expansion of Convolutional Neural Networks (CNNs) applications in various domains such as autonomous driving and real-time data processing, the demand for efficient computational resources has increased dramatically. Traditional computing platforms such as CPUs struggle to manage the complex and data-intensive tasks required by modern CNNs. This paper delves into the development and system optimization of dedicated hardware accelerators - GPUs, FPGAs, and ASICs to meet these demands. Through innovative architectural design and optimization techniques, these gas pedals improve computational speed and energy efficiency. Our study demonstrates that through these optimizations, the processing efficiency of hardware accelerators is significantly improved while energy consumption is effectively controlled, setting a new standard for the design of future hardware accelerators for convolutional neural networks. In addition, I discuss practical applications and future challenges, providing a comprehensive overview of current technologies and their potential development. This research highlights the critical role of advanced hardware accelerators in enabling the next generation of AI applications, ensuring technological advancement and sustainability in high-demand computing environments.

Hardware accelerators are playing an increasingly important role in the field of computer science and engineering, especially in accelerating deep learning and large-scale data processing. However, the traditional hardware accelerator design often relies on manual adjustment and experience accumulation, which is inefficient. Therefore, the use of machine learning technology to optimize hardware accelerator design has become a new trend. In this paper, a hardware accelerator optimization design method based on machine learning is proposed, including hardware design, algorithm optimization, system-level optimization, and deep learning model compression. Through the research and verification of the actual case, this research proves the effectiveness and feasibility of the hardware accelerator optimization design method based on machine learning. Compared with the traditional design method, this method can find better design scheme in shorter time and has higher performance prediction accuracy. Therefore, this method can significantly improve the design efficiency and performance of hardware accelerators. The results of this study are of great significance for promoting the development of hardware accelerator technology. It can promote scientific research and engineering applications in related fields to make greater progress, and promote the development of artificial intelligence, big data processing and other fields.

Scaling the memory hierarchy is a major challenge when we scale the number of cores in a multicore processor. Software Managed Multicore (SMM) architectures come up as one of the promising solutions. In an SMM architecture, there are no caches, and each core has only a local scratchpad memory [Banakar et al. 2002]. As the local memory usually is small, large applications cannot be directly executed on it. Code and data of the task mapped to each core need to be managed between global memory and local memory. This article solves the problem of efficiently managing code on an SMM architecture. The primary requirement of generating efficient code assignments is a correct management cost model. In this article, we address this problem by proposing a cost calculation graph. In addition, we develop two heuristics CMSM (Code Mapping for Software Managed multicores) and CMSM_advanced that result in efficient code management execution on the local scratchpad memory. Experimental results collected after executing applications from the MiBench suite [Guthaus et al. 2001] demonstrate that merely by adopting the correct management cost calculation, even using previous code assignment schemes, we can improve performance by an average of 12%. Combining the correct management cost model and a more optimized code mapping algorithm together, our heuristics can reduce runtime in more than 80% of the cases, and by up to 20% on our set of benchmarks, compared to the state-of-the-art code assignment approach [Jung et al. 2010]. When compared with Instruction-level Parallelism (ILP) results, CMSM_advanced performs an average of 5% worse. We also simulate the benchmarks on a cache-based system, and find that the code management overhead on SMM core with our code management is much less than memory latency of a cache-based system.

Low power & mobile hardware accelerators for deep convolutional neural networks

Homomorphic Encryption (HE) allows performing specific algebraic computations on encrypted data without the need for decryption. For this reason, HE is emerging as a strong privacy-preserving solution in cloud computing environments since it allows to keep data secure even in the case the cloud server is not trusted. HE libraries such as Microsoft SEAL have been recently released; however, such libraries are not specifically designed for resource-constrained platforms and they are often expensive in terms of computational resources and memory consumption, which limits their usage in edge devices. This limitation is contained by the SEAL-Embedded library, the first C-based HE library specifically designed for embedded platforms. In this article, we propose a hardware accelerator specifically designed for the SEAL-Embedded library and its implementation of the CKKS scheme: the proposed hardware presents a configurable Number Theoretic Transform (NTT) unit for all the polynomial degrees available on the SEAL-Embedded, a memory architecture able to reduce the I/O latency and a dedicated module for the generation of roots of unity. A complete system that includes a 32-bit RISC-V processor has been implemented on a Xilinx ZCU106 FPGA board to test the functionality of the hardware accelerator and to measure performance improvements. The results showed a speed-up of around x1000 with the hardware acceleration respect to the pure software implementation of the SEAL-Embedded library for the symmetric encryption function.

We present an automatic, static program transformation that schedules and generates efficient memory transfers between a computer host and its hardware accelerator, addressing a well-known performance bottleneck. Our automatic approach uses two simple heuristics: to perform transfers to the accelerator as early as possible and to delay transfers back from the accelerator as late as possible. We implemented this transformation as a middle-end compilation pass in the pips /Par4All compiler. In the generated code, redundant communications due to data reuse between kernel executions are avoided. Instructions that initiate transfers are scheduled effectively at compile-time. We present experimental results obtained with the Polybench 2.0, some Rodinia benchmarks, and with a real numerical simulation. We obtain an average speedup of 4 to 5 when compared to a naïve parallelization using a modern gpu with Par4All , hmpp , and pgi , and 3.5 when compared to an OpenMP version using a 12-core multiprocessor.KeywordsAutomatic parallelizationcommunication optimizationsource-to-source compilationheterogeneous parallel architecture gpu

In theoretical computer science, parallel computation has been traditionally studied by investigating time, processor, and space complexities of various problems in a model of parallel random access machine called the PRAM model. The PRAM model assumes that there are some number of processors and a global memory into which the data is stored. In one step, a processor can read or write one word from the global memory into its registers or it can perform some simple operations such as adding two numbers or comparing them. Unfortunately, this model is too simplistic and does not capture some important aspects of parallel computation that are observed in practice. In particular, real computers are connected to each other by an interconnection network.; This imposes the following constraints that are not taken into account by the PRAM model: communication latency and local versus global communication, block transfers, memory and network conflicts, and the bandwidth of the interconnection network. We give below a brief description of these constraints and indicate some recent work that has been done in order to obtain more realistic models of parallel computation.

This dissertation describes research activities broadly concerning the area of High-level synthesis (HLS), but more specifically, regarding the HLS-based design of energy-efficient hardware (HW) accelerators. HW accelerators, mostly implemented on FPGAs, are integral to the heterogeneous architectures employed in modern high performance computing (HPC) systems due to their ability to speed up the execution while dramatically reducing the energy consumption of computationally challenging portions of complex applications. Hence, the first activity was regarding an HLS-based approach to directly execute an OpenCL code on an FPGA instead of its traditional GPU-based counterpart. Modern FPGAs offer considerable computational capabilities while consuming significantly smaller as compared to high-end GPUs. Several different implementations of the K-Nearest Neighbor algorithm were considered on both FPGA- and GPU-based platforms and their performance was compared. FPGAs were generally more energy-efficient than the GPUs in all the test cases. Eventually, we were also able to get a faster (in terms of execution time) FPGA implementation by using an FPGA-specific OpenCL coding style and utilizing suitable HLS directives. The second activity was targeted towards the development of a methodology complementing HLS to automatically derive optimization directives (also known as power intent) from a system-level design description and use it to drive the design steps after HLS, by producing a directive file written using the common format (CPF) to achieve shut-off (PSO) in case of an ASIC design. The proposed LP-HLS methodology reduces the design effort by enabling designers to infer low information from the system-level description of a design rather than at the RTL. This methodology required a SystemC description of a generic management module to describe the design context of a HW module also modeled in SystemC, along with the development of a tool to automatically produce the CPF file to accomplish PSO. Several test cases were considered to validate the proposed methodology and the results demonstrated its ability to correctly extract the low information and apply it to achieve optimization in the backend flow.

Neural networks have been widely deployed in a number of applications due to their strong learning and feature extraction ability. To meet the ever increasing accuracy requirements from various applications, neural network models have become more complicated and diversified by exploiting more layers, larger model size, and more diverse functions. As a result, the design of application specified hardware accelerators for neural networks is also becoming more difficult, especially for real-time embedded applications. To efficiently bridge the gap between algorithm and hardware design phases, a pre-RTL neural network simulator (SimuNN) is proposed in this paper to enable early phase verification and fast prototyping. SimuNN can be used for inference, simulation and also evaluation, results of which can guide the design of both neural network models and hardware accelerators. SimuNN supports inference in various data precision, and is compatible with TensorFlow, which makes it easy for platform migration. It provides multi-level trace results that can be taken as the gold model of RTL designs. Moreover, the hardware performance under various quantizations, dataflow organizations and hardware configurations can be evaluated by SimuNN based on a generalized hardware model. Based on that, two dataflow organization schemes are concluded for determining the optimal configurations of hardware architecture under various hardware and performance constraints.

This chapter demonstrates distinctive features of FPGA-based hardware accelerators. In order to compete with the existing alternative solutions (both in hardware and in software) a wide level parallelism must be implemented in circuits with small propagation delays. For providing such characteristics several useful techniques are discussed and analyzed such as the ratio between combinational and sequential computations at different levels. Individual data items are represented in the form of long size input vectors that are processed concurrently in accelerators producing long size output vectors. Thus, pre- (conversion of individual items to long size input vectors) and post- (conversion of long size output vectors to individual items) processing operations have to be carried out. The technique of communication-time data processing is introduced and its importance is underlined. Core network-based architectures of hardware accelerators are discussed and the best architectures are chosen for future consideration. Finally, different aspects of design and implementation of FPGA-based hardware accelerators are analyzed and three sources for the designs are chosen that are synthesizable hardware description language specifications, the proposed reusable components, and intellectual property cores available on the market. At the end of the chapter a few useful examples are presented in detail and discussed.

This article designs a Yolov4 Tiny convolutional neural network hardware accelerator based on FPGA. A four-stage pipeline convolutional array structure has been proposed. In the design, the NC4HW4 parameter rearrangement and Im2col dimensionality reduction algorithm are used as the core to maximize the parallelism of matrix operations under limited resources. Secondly, a PE convolutional computing unit structure was designed, and a resource-efficient and highly reliable convolutional computing module was implemented by combining INT8 DSP resource reuse technology. Finally, the accelerator will be deployed on Xilinx’s Zynq7030 development board. The experimental results show that at a clock frequency of 130 MHz, the power consumption of the hardware accelerator is only 2.723 W, and the performance is 59.54 Gbps. Compared with related research, it has improved more than 2.1 times. This accelerator can complete hardware-accelerated computing tasks in object detection with high energy efficiency.

We describe a massively parallel Nested Factorization (NF) linear solver for large systems of equations. NF is a powerful classic preconditioner receiving renewed attention due to its potential on emerging parallel architectures, especially Graphics Processing Units (GPUs). We build on the Massively Parallel NF (MPNF) framework described by Appleyard et al. (2011). MPNF divides the threedimensional grid into ‘kernels’, assigns each kernel a color, such that no neighboring kernels share the same color. Parallelism is exploited by operating on all the kernels of a given color simultaneously and cycling through the NF operations color by color. Our MPNF algorithm is designed with special attention to asynchronous CPU-to-GPU memory transfer during the setup phase. Moreover, a CUDA-based BiCGStab Krylov solver and a customized ‘reduction kernel’ with greater bandwidth are used. The key features of the algorithm are: 1) a special ordering of the matrix elements that maximizes coalesced access to GPU global memory and speeds up kernel execution by several folds, 2) application of twisted factorization, which increases the number of concurrent threads at no additional cost, and (3) extension to multiple GPUs by first solving the so-called halo region in each GPU and overlapping peer-to-peer memory transfer between GPUs with solution of the interior regions. The GPU-based NF solver is demonstrated using several large problems, and we breakdown the performance details of all the algorithmic components. For the SPE10 model (highly heterogeneous with over one million cells) on a 512-core Tesla M2090 GPU, our implementation achieves a speed up of 26 for single-precision and 19 for double-precision computations compared with a single core of the Xeon X5660 CPU. Moreover, the (3072-core) 6-GPU solution of a highly refined SPE10 model (26.9 million cells) is more than five times faster than the single-GPU solution.

Programming of heterogeneous platforms requires deep understanding of system architecture on all levels, which help applications design to leveraging the best data and work decomposition between CPU and an accelerating hardware like GPUs. However, in many cases the applications are being converted form a conventional CPU programming language like C++, or from accelerator friendly but still low level languages like OpenCL, and the main problem is to determine which part of the application is leveraging from being offloaded to GPU. Another problem is to estimate, how much performance increase one might gain due to the accelerating in the particular GP GPU device. Each platform has its unique limitations that are affecting performance of offloaded computing tasks, e.g. data transfer tax, task initialization overhead, memory latency and bandwidth limitations. In order to take into account those constraints, software developers need tooling for collecting right information and producing recommendations to make the best design and optimization decisions. In this presentation we will introduce two new GPU performance analysis types in Intel® VTune™ Profiler, and a methodology of heterogeneous applications performance profiling supported by the analyses. VTune Profiler is a well-known tool for performance characterization on CPUs, now it includes GPU Offload Analysis and GPU Hotspots Analysis of applications written on most offloading models with OpenCL, SYCL/Data Parallel C++, and OpenMP Offload. The GPU Offload analysis helps to identify how CPU is interacting with GPU(s) by creating and submitting tasks to offload queues. It provides metrics and performance data such as GPU Utilization, Hottest GPU Computing Tasks, Tasks instance count and timing, kernel Data Transfer Size, SIMD Width measurements, GPU Execution Units (EU) threads occupancy, and Memory Utilization. All together the metrics are providing a systematic picture on how effectively tasks were offloaded and executed on GPUs. The GPU Hotspots analysis is intended to examine computing tasks or kernels efficiency running on GPU EUs and interacting with GPU memory subsystem. Inefficiencies that are conditioned by compute kernels implementation or compiler issues are resulting in idling of EUs or increased latencies in data fetching from memory sources to EU registers, which is eventually leading to performance degradation. Due to complexity of GPU memory subsystem (L1, L2 Caches, Shared Local Memory, L3 Cache, GPU DRAM, CPU LLC and DRAM), analyzing data access inefficiencies is even more problematic. The GPU Hotspots analysis is addressing those problems by presenting a visualization of a current GPU Memory Hierarchy Diagram, detailed data transfer tracing between different memory agents, memory bandwidth measurements, barriers and atomics analysis. In addition, VTune is analyzing each compute kernel on a source level, providing performance metrics against source lines or assembly instructions. Memory Latency metrics are helping to determine most inefficient data accesses on a source line level. Supplementary GPU Instruction Count analysis clarifies with instruction set in a kernel generated by a Compiler. The GPU analyses in VTune are well developed for OpenCL language and run-time, however the most recent SYCL language and its extension Data Parallel C++ along with Level Zero run-time are supported as well, running on all Intel GPUs from Gen9 HD Graphics to Intel Iris Xe Graphics (a discrete GPU card). Results of performance profiling on different GPU architectures will be presented in the session. VTune Profiler for GPUs is a newly extended toolset which is being actively developed along with development of new acceleration architectures at Intel. New features and analysis concepts are constantly appearing in the tool fulfilling the needs of software architects and developers.