Fixed Point Implementation of Tiny-Yolo-v2 using OpenCL on FPGA

This paper describes a hardware-oriented two-stage algorithm that can be deployed in a resource-limited field-programmable gate array (FPGA) for fast-object detection and recognition with out external memory. The first stage is the bounding boxes proposal with a conventional object detection method, and the second is convolutional neural network (CNN)-based classification for accuracy improvement. Frequently accessing external memories significantly affects the execution efficiency of object classification. Unfortunately, the existing CNN models with a large number of parameters are difficult to deploy in FPGAs with limited on-chip memory resources. In this study, we designed a compact CNN model and performed the hardware-oriented quantization for parameters and intermediate results. As a result, CNN-based ultra-fast-object classification was realized with all parameters and intermediate results stored on chip. Several evaluations were performed to demonstrate the performance of the proposed algorithm. The object classification module consumes only 163.67 Kbits of on-chip memories for ten regions of interest (ROIs), this is suitable for low-end FPGA devices. In the aspect of accuracy, our method provides a correctness rate of 98.01% in open-source data set MNIST and over 96.5% in other three self-built data sets, which is distinctly better than conventional ultra-high-speed object detection algorithms.

Convolutional Neural Network (CNN) has achieved great success in various fields of machine vision, such as image classification and recognition, image segmentation, and video analysis. In specific applications, it is often necessary to customize the network structure by altering image size, convolution kernel size, pooling size, network architecture, and the number of network layers. These customizations pose significant challenges to the architecture of CNN accelerator especially in real-time systems, and as the number of these network variables increases, the internal convolution calculations become extremely demanding. The inference speed and energy consumption of CNN are becoming more and more important, which requires the design of the accelerator to adapt to different network architecture and run efficiently. Field-programmable gate arrays (FPGAs) is an ideal choice for a CNN accelerator due to its high programmability and low power consumption. To address the aforementioned challenges, we present NRUCA, a novel network reconfigurable universal CNN accelerator based on FPGA. On one hand, we have designed a flexible architecture that can dynamically configure CNN’s network structure parameters. These parameters can be sent to the FPGA via a configuration file, enabling the same design to run different CNN networks and allowing to modify on the fly. This forms the basis of the architecture’s adaptability presented in this paper. On the other hand, we employ a multi-channel parallel and efficient pipeline matrix multiplication architecture to implement the convolution and fully-connected layers, which constitute the majority of the computational load in CNNs. By utilizing an innovative inverted matrix multiplication algorithm and an interleaving cache data method, we reduce the internal cache required for multi-channel data. Furthermore, most of the intermediate calculation data does not need to be output to DDR, significantly improving the operation efficiency of the accelerator. We also fully leverage the FPGA chip architecture to compile multiple calculation kernels, which can be flexibly scheduled and combined to cater to various application scenarios. Based on the flexible and efficient overall design architecture, the layout of various resources of FPGA is balanced, which enables the designed FPGA project to compile four acceleration kernels with the running clock up to 300MHz in Xilinx Vitis and test them on Xilinx Alveo U250. We verified the adaptability and acceleration capabilities of our FPGA project using LeNet, AlexNet, VGG11, VGG13, VGG16, and VGG19 networks. Experiments demonstrated that our FPGA architecture achieves 33X and 35X energy savings compared to the Intel Xeon 5220R CPU, and 1.08X and 1.05X energy savings compared to the Nvidia Tesla P100 GPU, when accelerating AlexNet and VGG16, respectively.

Object detection algorithms using convolutional neural networks (CNNs) achieve high detection accuracy, but it is challenging to realize real-time object detection due to their high computational complexity, especially on resource-constrained mobile platforms. In this article, we propose an algorithm-hardware co-optimization approach to designing a real-time object detection system. We first develop a compact object detection model based on a binarized neural network (BNN), which employs a new layer structure, the DenseToRes layer, to mitigate information loss due to deep quantization. We also propose an efficient object detection processor that runs object detection with high throughput using limited hardware rescources. We develop a resource-efficient processing unit supporting variable precision with minimal hardware overheads. Implemented in field-programmable gate array (FPGA), the object detection processor achieves 64.51 frames/s throughput with 64.92 mean average precision (mAP) accuracy. Compared to prior FPGA-based designs for object detection, our design achieves high throughput with competitive accuracy and lower hardware implementation costs.

Flora in Indonesia has about 25% of the species of flowering plant species present in the world. Roses are one type of flowering plants and are usually used as an ornamental plant that has a thorny stem. Roses have more than 150 species. In Indonesia there are several flower gardens that is larger than the others. One of the famous flower garden in Indonesia is located on Malang city, East Java. The flower garden in Malang has several varieties of many roses and has a large production of roses. To help the sales system of roses there, the researchers want to create a program that can classify the type of roses in order to help simplify the system of automatic sales of roses without through manual sorting. So that will accelerate the sale of roses with an automated system. Ordinary people with limited botanical knowledge usually don’t know how to classify the flowers just by looking at them. To classify the flowers properly, it is important to provide enough information, and one of them is the name of it. Convolutional Neural Network (CNN) is one method of deep learning that can be used for image classification process. The CNN design is motivated by the discovery of the visual mechanism, the visual cortex present in the brain. CNN has been widely used in many real-world applications, such as Face Recognition, Image Classification and Recognition, and Object Detection, because this is one of the most efficient method for extracting important features. In this research, the classification accuracy value obtained from the test data is 96.33% using 2-dimension Red Green Blue (RGB) input image, and the size of each image is 32 × 32 pixels that are trained with CNN algorithm and the network structure of four convolution layers and four layers pooling supported by dropout technique.

The objective of this thesis was to evaluate the viability of implementation of an object recognition algorithm driven by deep learning for aerospace manufacturing, maintenance and assembly tasks. Comparison research has found that current computer vision methods such as, spatial mapping was limited to macro-object recognition because of its nodal wireframe analysis. An optical object recognition algorithm was trained to learn complex geometric and chromatic characteristics, therefore allowing for micro-object recognition, such as cables and other critical components. This thesis investigated the use of a convolutional neural network with object recognition algorithms. The viability of two categories of object recognition algorithms were analyzed: image prediction and object detection. Due to a viral epidemic, this thesis was limited in analytical consistency as resources were not readily available. The prediction-class algorithm was analyzed using a custom dataset comprised of 15 552 images of the MaxFlight V2002 Full Motion Simulator’s inverter system, and a model was created by transfer-learning that dataset onto the InceptionV3 convolutional neural network (CNN). The detection-class algorithm was analyzed using a custom dataset comprised of 100 images of two SUVs of different brand and style, and a model was created by transfer-learning that dataset onto the YOLOv3 deep learning architecture. The tests showed that the object recognition algorithms successfully identified the components with good accuracy, 99.97% mAP for prediction-class and 89.54% mAP. For detection-class. The accuracies and data collected with literature review found that object detection algorithms are accuracy, created for live -feed analysis and were suitable for the significant applications of AVI and aircraft assembly. In the future, a larger dataset needs to be complied to increase reliability and a custom convolutional neural network and deep learning algorithm needs to be developed specifically for aerospace assembly, maintenance and manufacturing applications.

Based on the vigorous development of deep learning, image classification nowadays is mainly accomplished by using the Convolutional Neural Networks whose main computation tasks are usually calculated by Graphics Processing Units. Although the computing speed of Graphics Processing Units is faster than Central Processing Units when considering the neural networks, the power consumption of Graphics Processing Units is significantly huge. In this paper, we investigate the hardware acceleration of neural networks by using the Field Programmable Gate Array via High-Level Synthesis. We study the binarized neural networks with binarized weights and activation functions, which is suitable to be implemented by the Field Programmable Gate Array. The proposed design has high computing speed and low power consumption. Our method is applied to a robot vision application to classify robots from human and others. Experiment results show that our design provides good performance for image classification.

With the ever-increasing compute demands of artificial intelligence (AI) workloads, there is extensive interest in leveraging field-programmable gate-arrays (FPGAs) to quickly deploy hardware accelerators for the latest convolutional neural networks (CNNs). Recent FPGA architectures are also evolving to better serve the needs of AI, but accelerators need extensive re-design to leverage these new features. The Stratix 10 NX chip by Intel is a new FPGA that replaces traditional DSP blocks with in-fabric AI tensor blocks that provide 15x more multipliers and up to 143 TOPS of performance, at the cost of lower precision (INT8) and significant restrictions on how many operands can be fed to the multipliers from the programmable routing. In this paper, we explore different CNN accelerator structures to leverage the tensor blocks, considering the various tensor block modes, operand bandwidth restrictions, and on-chip memory restrictions. We incorporate the most performant techniques into HPIPE, a layer-pipelined and sparse-aware CNN accelerator for FPGAs. We enhance HPIPE's software compiler to restructure the CNN computations and on-chip memory layout to take advantage of the additional multipliers offered by the new tensor block architecture, while also avoiding stalls due to data loading restrictions. We achieve cycle-by-cycle speedups in tensor mode of up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8}.\mathbf{3}\mathbf{x}$</tex> for Mobilenet-v1 versus the original HPIPE design using conventional DSPs. On the FPGA, we achieve a throughput of 28,541 and 29,429 images/s on Mobilenet-v1 and Mobilenet-v2 respectively, outperforming all previous FPGA accelerators by at least 4.0x, including one on an AI-optimized Xilinx chip. We also outperform NVIDIA's V100 GPU, a machine learning targeted GPU on a similar process node with a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{1}.\mathbf{7}\mathbf{x}$</tex> larger die size, by up to 17x with a batch size of one and 1.3x with NVIDIA's largest reported batch size of 128.

Nowadays Convolutional Neural Networks (CNNs) are being employed in a wide range of industrial technologies for a variety of sectors, such as medical, automotive, aviation, agriculture, space, etc. This paper reviews the state-of-the-art in both the field of CNNs for image classification and object detection and Autonomous Driving Systems (ADSs) in a synergetic way. Layer-based details of CNNs along with parameter and floating-point operation number calculations are outlined. Using an evolutionary approach, the majority of the outstanding image classification CNNs, published in the open literature, is introduced with a focus on their accuracy performance, parameter number, model size, and inference speed, highlighting the progressive developments in convolutional operations. Results of a novel investigation of the convolution types and operations commonly used in CNNs are presented, including a timing analysis aimed at assessing their impact on CNN performance. This extensive experimental study provides new insight into the behaviour of each convolution type in terms of training time, inference time, and layer level decomposition. Building blocks for CNN-based object detection are also discussed, such as backbone networks and baseline types, and then representative state-of-the-art designs are outlined. Experimental results from the literature are summarised for each of the reviewed models. This is followed by an overview of recent ADSs related works and current industry activities, aiming to bridge academic research and industry practice on CNNs and ADSs. Design approaches targeted at solving problems of automakers in achieving real-time implementations are also proposed based on a discussion of design constraints, human vs. machine evaluations and trade-off analysis of accuracy vs. size. Current technologies, promising directions, and expectations from the literature on ADSs are introduced including a comprehensive trade-off analysis from a human-machine perspective.

Field-Programmable Gate Array (FPGA) based hardware accelerators offer reconfigurability, performance, adaptability, and good energy efficiency. The majority of Convolutional Neural Network (CNN) based inference systems are initially developed using standardized frameworks like PyTorch, Tensor Flow, and more. These Python or Python-like models can be mapped on FPGAs to build accelerators. Mapping frameworks to port designs on an FPGA convert the CNN models to high-level languages like C/C++ or OpenCL so that standard tools like high-level synthesis can facilitate the mapping of models on an FPGA. The logic utilization and performance of FPGA-based accelerators are dependent on the CNN network parameters, architectural selection (data-flow, pipelined, etc.), and synthesis-based control of design generation. A scalable multi-layer CNN hardware accelerator is modeled in Vitis 2020 HLS tool. Early estimation of performance and hardware resources helps choose the best CNN network before those are executed for time-consuming high-level synthesis and physical design mapping for FPGAs. We present various Machine Learning (ML) models to estimate the Logic Utilization and Computation Time from the Python design descriptions of CNNs. Our results show a very successful and accurate estimation for performance and resource utilization over various multi-layer CNN networks in negligible time before running HLS synthesis.

Convolutional neural networks (CNNs) are emerging as powerful tools for image processing in important commercial applications. We focus on the important problem of improving the latency of image recognition. While CNNs are highly amenable to prefetching and multithreading to avoid memory latency issues, CNNs’ large data – each layer’s input, filters, and output – poses a memory bandwidth problem. While previous work captures only some of the enormous data reuse, full reuse implies that the initial input image and filters are read once from off-chip and the final output is written once off-chip without spilling the intermediate layers’ data to off-chip. We propose Occam to capture full reuse via four contributions. First, we identify the necessary conditions for full reuse. Second, we identify the dependence closure as the sufficient condition to capture full reuse using the least on-chip memory. Third, because the dependence closure is often too large to fit in on-chip memory, we propose a dynamic programming algorithm that optimally partitions a given CNN to guarantee the least off-chip traffic at the partition boundaries for a given on-chip capacity. While tiling is well-known, our contribution determines the optimal cross-layer tiles. Occam’s partitions reside on different chips, forming a pipeline so that a partition’s filters and dependence closure remain on-chip as different images pass through (i.e., each partition incurs off-chip traffic only for its inputs and outputs). Finally, because the optimal partitions may result in an unbalanced pipeline, we propose staggered asynchronous pipelines (STAPs) that replicate bottleneck stages to improve throughput by staggering mini-batches across replicas. Importantly, STAPs achieve balanced pipelines without changing Occam’s optimal partitioning. Our simulations show that, on average, Occam cuts off-chip transfers by 21× and achieves 2.04× and 1.21× better performance, and 33% better energy than the base case, respectively. Using a field-programmable gate array (FPGA) implementation, Occam performs 6.1× and 1.5× better, on average, than the base case and Layer Fusion, respectively.

Field programmable gate array (FPGA) is widely considered as a promising platform for convolutional neural network (CNN) acceleration. However, the large numbers of parameters of CNNs cause heavy computing and memory burdens for FPGA-based CNN implementation. To solve this problem, this paper proposes an optimized compression strategy, and realizes an accelerator based on FPGA for CNNs. Firstly, a reversed-pruning strategy is proposed which reduces the number of parameters of AlexNet by a factor of 13× without accuracy loss on the ImageNet dataset. Peak-pruning is further introduced to achieve better compressibility. Moreover, quantization gives another 4× with negligible loss of accuracy. Secondly, an efficient storage technique, which aims for the reduction of the whole overhead cache of the convolutional layer and the fully connected layer, is presented respectively. Finally, the effectiveness of the proposed strategy is verified by an accelerator implemented on a Xilinx ZCU104 evaluation board. By improving existing pruning techniques and the storage format of sparse data, we significantly reduce the size of AlexNet by 28×, from 243 MB to 8.7 MB. In addition, the overall performance of our accelerator achieves 9.73 fps for the compressed AlexNet. Compared with the central processing unit (CPU) and graphics processing unit (GPU) platforms, our implementation achieves 182.3× and 1.1× improvements in latency and throughput, respectively, on the convolutional (CONV) layers of AlexNet, with an 822.0× and 15.8× improvement for energy efficiency, separately. This novel compression strategy provides a reference for other neural network applications, including CNNs, long short-term memory (LSTM), and recurrent neural networks (RNNs).

The application of deep learning technology has increased rapidly in recent years. Technologies in deep learning increasingly emulate natural human abilities, such as knowledge learning, problem-solving, and decision-making. In general, deep learning can carry out self-training without repetitive programming by humans. Convolutional neural networks (CNNs) are deep learning algorithms commonly used in wide applications. CNN is often used for image classification, segmentation, object detection, video processing, natural language processing, and speech recognition. CNN has four layers: convolution layer, pooling layer, fully connected layer, and non-linear layer. The convolutional layer uses kernel filters to calculate the convolution of the input image by extracting the fundamental features. The pooling layer combines two successive convolutional layers. The third layer is the fully connected layer, commonly called the convolutional output layer. The activation function defines the output of a neural network, such as 'yes' or 'no'. The most common and popular CNN activation functions are Sigmoid, Tanh, ReLU, Leaky ReLU, Noisy ReLU, and Parametric Linear Units. The organization and function of the visual cortex greatly influence CNN architecture because it is designed to resemble the neuronal connections in the human brain. Some of the popular CNN architectures are LeNet, AlexNet and VGGNet.

Network router virtualization has recently gained much interest in the research community, as it allows multiple virtual router instances to run on a common physical router platform. The key metrics in designing network virtual routers are (1) number of supported virtual router instances, (2) total number of prefixes, and (3) ability to quickly update the virtual table. Existing merging algorithms use leaf pushing and a shared next hop data structure to eliminate the large memory bandwidth requirement. However, the size of the shared next hop table grows linearly with the number of virtual routers. Due to the limited amount of on-chip memory and the number of I/O pins of Field Programmable Gate Arrays (FPGAs), existing designs cannot support large number of tables and/or large number of prefixes. This paper exploits the abundant parallelism and on-chip memory bandwidth available in the state-of-the-art FPGAs, and proposes a compact trie representation and a hybrid data structure to reduce the memory requirement of virtual routers. The approach does not require leaf-pushing; therefore, reduces the size of each entry of the data structure. Our algorithm demonstrates substantial reduction in the memory footprint compared with the state-of-the-art. Also, it eliminates the shared next hop information data structure and simplifies the table updates in virtual routers. Using a state-of-the-art FPGA, the proposed architecture can support up to 3.1M IPv4 prefixes. Employing the dual-ported memory available in the current FPGAs, we map the proposed data structure on a novel SRAMbased linear pipeline architecture to achieve high throughput. The post place-and-route result shows that our architecture can sustain a throughput of 394 million lookups per second, or 126 Gbps (for the minimum packet size of 40 Bytes).

Convolutional Neural Networks (CNNs) require massive parallelism due to the high-precision floating-point arithmetic operations they perform. So, demand of processing power in them is significantly higher than what a standard CPU can offer. This has traditionally made CNNs more suited for running on a Graphics Processing Unit (GPU). However, GPUs consume much more power than CPUs, rendering the former impractical for implementing CNNs in Edge AI (Artificial Intelligence), where restraining power consumption is paramount. On the other hand, FPGAs (Field Programmable Gate Arrays) are more suited for AI computing at the edge as they consume much lesser power than GPUs and even CPUs. Additionally, GPUs and CPUs are not suited for real-time AI applications, which require both high throughput and low latency at the same time and FPGAs excel at all these requirements. The purpose of this paper is to provide a study of the performance of FPGA as the most suitable platform for AI-based computing at the edge. To achieve this, we chose AlexNet, a popular CNN image classifier, for which we present a case study on four different platforms: CPU, GPU, embedded RISC core, and FPGA fabric. Then, we quantitatively measure and compare the performance in terms of inference time (time needed to classify an image) on all these platforms. Inference time using FPGA is reduced by almost 64, 1.6, and 1.1 times compared to a dual-core ARM, an i5-6400 CPU, and an Nvidia GPU, respectively.

Field Programmable Gate Arrays (FPGAs) have been targeted as a new accelerator of the HPC field. This is because the barrier to using FPGAs has been gradually lowered due to the widespread use of high-level synthesis (HLS) technology. In addition, the bandwidth of external memory in FPGAs is much lower than that of other accelerators widely used in HPC, such as NVIDIA V100 GPUs. However, the latest FPGAs can use High Bandwidth Memory 2 (HBM2), which has a memory bandwidth of up to 512GB/s. Therefore, we believe FPGAs will be a viable option for speeding up applications. However, unlike CPUs and GPUs, FPGAs do not have caches and memory networks to exploit the full potential of HBM2, which may limit the efficiency of the application. In this paper, we propose a memory system for HBM2 and HPC applications. We show the prototype implementation of the system and evaluate its performance. We also demonstrate the use of the proposed system from an application developed in High-Level Synthesis (HLS) written in C++.