Convolution Neural Network Development Support System using Approximation Methods to Evaluate Inference Accuracy and Memory Usage in an Embedded System

This paper proposes a hardware oriented dropout algorithm for an efficient field-programmable gate array (FPGA) implementation. Dropout is a regularization technique, which is commonly used in neural networks such as multilayer perceptrons (MLPs), convolutional neural networks (CNNs), among others. To generate a dropout mask to randomly drop neurons during training phase, random number generators (RNGs) are usually used in software implementations. However, RNGs consume considerable FPGA resources in hardware implementations. The proposed method is able to minimize the resources required for FPGA implementation of dropout by performing a simple rotation operation to a predefined dropout mask. We apply the proposed method to MLPs and CNNs and evaluate them on MNIST and CIFAR-10 classification. In addition, we employ the proposed method in GoogLeNet training using own dataset to develop a vision system for home service robots. The experimental results demonstrate that the proposed method achieves the same regularized effect as the ordinary dropout algorithm. Logic synthesis results show that the proposed method significantly reduces the consumption of FPGA resources in comparison to the ordinary RNG-based approaches.

In this article, the method of text classification using a convolutional neural network is presented. The problem of text classification is formulated, the architecture and the parameters of a convolutional neural network for solving the problem are described, the steps of the solution and the results of classification are given. The convolutional network which was used was trained to classify the texts of the news messages of Internet information portals. The semantic preprocessing of the text and the translation of words into attribute vectors are generated using the open word2vec model. The analysis of the dependence of the classification quality on the parameters of the neural network is presented. The using of the network allowed obtaining a classification accuracy of about 84%. In the estimation of the accuracy of the classification, the texts were checked to belong to the group of semantically similar classes. This approach allowed analyzing news messages in cases where the text themes and the number of classification classes in the training and control samples do not equal.

In recent years, convolutional neural networks have been widely used in image processing and have shown good results. Particularly useful was their ability to automatically extract image features (textures and shapes of objects). The article proposes a method that improves the accuracy and speed of recognition of an ultra-precise neural network based on image recognition of people’s faces. At first, a recurrent neural network is introduced into the convolutional neural network, thereby studying the characteristics of the image more deeply. Deep image characteristics are studied in parallel using a convolutional and recurrent neural network. In line with the idea of skipping the ResNet convolution layer, a new ShortCut3- ResNet residual module is built. A double optimization model is created to fully optimize the convolution process. A study of the influence of various parameters of a convolutional neural network on network performance is demonstrated, also analyzed using simulation experiments. As a result, the optimal parameters of the convolutional neural network are established. Ex- periments show that the method presented in this paper can study various images of people’s faces regardless of age, gender, and also improves the accuracy of feature extraction and image recognition ability.

Convolutional neural networks (CNNs) are emerging as powerful tools for image processing in important commercial applications. We focus on the important problem of improving the latency of image recognition. While CNNs are highly amenable to prefetching and multithreading to avoid memory latency issues, CNNs’ large data – each layer’s input, filters, and output – poses a memory bandwidth problem. While previous work captures only some of the enormous data reuse, full reuse implies that the initial input image and filters are read once from off-chip and the final output is written once off-chip without spilling the intermediate layers’ data to off-chip. We propose Occam to capture full reuse via four contributions. First, we identify the necessary conditions for full reuse. Second, we identify the dependence closure as the sufficient condition to capture full reuse using the least on-chip memory. Third, because the dependence closure is often too large to fit in on-chip memory, we propose a dynamic programming algorithm that optimally partitions a given CNN to guarantee the least off-chip traffic at the partition boundaries for a given on-chip capacity. While tiling is well-known, our contribution determines the optimal cross-layer tiles. Occam’s partitions reside on different chips, forming a pipeline so that a partition’s filters and dependence closure remain on-chip as different images pass through (i.e., each partition incurs off-chip traffic only for its inputs and outputs). Finally, because the optimal partitions may result in an unbalanced pipeline, we propose staggered asynchronous pipelines (STAPs) that replicate bottleneck stages to improve throughput by staggering mini-batches across replicas. Importantly, STAPs achieve balanced pipelines without changing Occam’s optimal partitioning. Our simulations show that, on average, Occam cuts off-chip transfers by 21× and achieves 2.04× and 1.21× better performance, and 33% better energy than the base case, respectively. Using a field-programmable gate array (FPGA) implementation, Occam performs 6.1× and 1.5× better, on average, than the base case and Layer Fusion, respectively.

Deep convolutional neural network (DCNN) is an advanced technology in image recognition. Because of extreme computing resource requirements, DCNN implementation with software alone cannot achieve real-time requirement. Therefore, the need to implement DCNN accelerator hardware is increasing. In this paper, we present a field programmable gate array (FPGA)-based hardware accelerator design of DCNN targeting handwritten Hangul character recognition application. Also, we present design optimization techniques in SDAccel environments for searching the optimal FPGA design space. The techniques we used include memory access optimization and computing unit parallelism, and data conversion. We achieved about 11.19 ms recognition time per character with Xilinx FPGA accelerator. Our design optimization was performed with Xilinx HLS and SDAccel environment targeting Kintex XCKU115 FPGA from Xilinx. Our design outperforms CPU in terms of energy efficiency (the number of samples per unit energy) by 5.88 times, and GPGPU in terms of energy efficiency by 5 times. We expect the research results will be an alternative to GPGPU solution for real-time applications, especially in data centers or server farms where energy consumption is a critical problem.

A genetic algorithm (GA), is used to optimize the many parameters of a convolutional neural network (CNN) that control the structure of the network. CNNs are used in image classification problems where it is necessary to generate feature descriptors to discern between image classes. Because of the deep representation of image data that CNNs are capable of generating, they are increasingly popular in research and industry applications. With the increasing number of use cases for CNNs, more and more time is being spent to come up with optimal CNN structures for different applications. Where one CNN might succeed at classification, another can fail. As a result, it is desirable to more easily find an optimal CNN structure to increase classification accuracy. In the proposed method, a GA is used to evolve the parameters that influence the structure of a CNN. The GA compares CNNs by training them on images of concrete containing cracks. The best CNN after several generations of the GA is then compared to the state-of-the-art CNN for crack detection. This work shows that it is possible to generalize the process of optimizing a CNN for image classification through the use of a GA.

In this study, Field Programmable Gate Array (FPGA) implementation of Conic Section Function Neural Network (CSFNN) for a classification problem focused on iris plant is presented. This work demonstrates for the first time to our knowledge, the feed-forward computation of CSFNN implementation on FPGA. Using l6-bit floating point arithmetic and the look-up tables (LUTs) for the sigmoid function and the square root function, 83% and 72% of slices and LUTs on Spartan 3-E XC3S1600E are used for the realization of CSFNN with five neurons. The classification results obtained from the FPGA implementation and software simulation show that the accuracy error between two platforms is only 0.1%.

Convolutional Neural Network (CNN) has achieved great success in various fields of machine vision, such as image classification and recognition, image segmentation, and video analysis. In specific applications, it is often necessary to customize the network structure by altering image size, convolution kernel size, pooling size, network architecture, and the number of network layers. These customizations pose significant challenges to the architecture of CNN accelerator especially in real-time systems, and as the number of these network variables increases, the internal convolution calculations become extremely demanding. The inference speed and energy consumption of CNN are becoming more and more important, which requires the design of the accelerator to adapt to different network architecture and run efficiently. Field-programmable gate arrays (FPGAs) is an ideal choice for a CNN accelerator due to its high programmability and low power consumption. To address the aforementioned challenges, we present NRUCA, a novel network reconfigurable universal CNN accelerator based on FPGA. On one hand, we have designed a flexible architecture that can dynamically configure CNN’s network structure parameters. These parameters can be sent to the FPGA via a configuration file, enabling the same design to run different CNN networks and allowing to modify on the fly. This forms the basis of the architecture’s adaptability presented in this paper. On the other hand, we employ a multi-channel parallel and efficient pipeline matrix multiplication architecture to implement the convolution and fully-connected layers, which constitute the majority of the computational load in CNNs. By utilizing an innovative inverted matrix multiplication algorithm and an interleaving cache data method, we reduce the internal cache required for multi-channel data. Furthermore, most of the intermediate calculation data does not need to be output to DDR, significantly improving the operation efficiency of the accelerator. We also fully leverage the FPGA chip architecture to compile multiple calculation kernels, which can be flexibly scheduled and combined to cater to various application scenarios. Based on the flexible and efficient overall design architecture, the layout of various resources of FPGA is balanced, which enables the designed FPGA project to compile four acceleration kernels with the running clock up to 300MHz in Xilinx Vitis and test them on Xilinx Alveo U250. We verified the adaptability and acceleration capabilities of our FPGA project using LeNet, AlexNet, VGG11, VGG13, VGG16, and VGG19 networks. Experiments demonstrated that our FPGA architecture achieves 33X and 35X energy savings compared to the Intel Xeon 5220R CPU, and 1.08X and 1.05X energy savings compared to the Nvidia Tesla P100 GPU, when accelerating AlexNet and VGG16, respectively.

Convolutional neural network (CNN) has become a successful algorithm in the region of artificial intelligence and a strong candidate for many computer vision algorithms. But the computation complexity of CNN is much higher than traditional algorithms. With the help of GPU acceleration, CNN-based applications are widely deployed in servers. However, for embedded platforms, CNN-based solutions are still too complex to be applied. Various dedicated hardware designs on field-programmable gate arrays (FPGAs) have been carried out to accelerate CNNs, while few of them explore the whole design flow for both fast deployment and high power efficiency. In this paper, we investigate state-of-the-art CNN models and CNN-based applications. Requirements on memory, computation and the flexibility of the system are summarized for mapping CNN on embedded FPGAs. Based on these requirements, we propose Angel-Eye, a programmable and flexible CNN accelerator architecture, together with data quantization strategy and compilation tool. Data quantization strategy helps reduce the bit-width down to 8-bit with negligible accuracy loss. The compilation tool maps a certain CNN model efficiently onto hardware. Evaluated on Zynq XC7Z045 platform, Angel-Eye is $6 {\times }$ faster and $5{\times }$ better in power efficiency than peer FPGA implementation on the same platform. Applications of VGG network, pedestrian detection and face alignment are used to evaluate our design on Zynq XC7Z020. NIVIDA TK1 and TX1 platforms are used for comparison. Angel-Eye achieves similar performance and delivers up to $16 {\times }$ better energy efficiency.

In this paper we describe a single-node, double precision Field Programmable Gate Array (FPGA) implementation of the Conjugate Gradient algorithm in the context of Lattice Quantum Chromodynamics. As a benchmark of our proposal we invert numerically the Dirac–Wilson operator on a 4-dimensional grid on three Xilinx hardware solutions: Zynq Ultrascale+ evaluation board, the Alveo U250 accelerator and the largest device available on the market, the VU13P device. In our implementation we separate software/hardware parts in such a way that the entire multiplication by the Dirac operator is performed in hardware, and the rest of the algorithm runs on the host. We find out that the FPGA implementation can offer a performance comparable with that obtained using current CPU or Intel’s many core Xeon Phi accelerators. A possible multiple node FPGA-based system is discussed and we argue that power-efficient High Performance Computing (HPC) systems can be implemented using FPGA devices only.

The convolution neural networks have revolutionized the computer vision domain. It has proven to be a dominant technology to carry out tasks such as image classification, semantic segmentation, and object detection. The convolution neural networks surpass the performance of the existing algorithms such as SIFT, HOG, etcetera. Where, instead of manually engineering the features, supervised learning help to learn the essential low-level and high-level features necessary for classifications. The convolution neural networks have become a popular tool to counter computer vision problems. However, it is computationally, and memory intensive to train and deploy the network because of the model size of a deep convolution neural networks. However, the research in the field of design space exploration (DSE) of neural networks and compression techniques to develop compact architectures, have made convolution neural networks memory and computationally efficient. These techniques have also improved the feasibility of convolution neural network for deployment on embedded targets. The paper explores the concept of compact convolution filters to reduce the number of parameters in a convolution neural network. The intuition behind the approach is that replacing convolution filters with a stack of compact convolution filters helps in developing a compact architecture with competitive accuracy. This paper explores the fire module a compact convolution filter and proposes a method of recreating a state-of-the-art architecture VGG-16 using the fire modules to develop a compact architecture, which is further trained on the CIFAR-10 dataset and deployed on a real-time embedded platform known as Bluebox 2.0 by NXP using RTMaps software framework.

Defect inspection is a crucial part of industrial manufacturing. However, it relies heavily on human effort on manual visual inspection. Various machine vision techniques have been introduced to replace human labour and to improve inspection quality and efficiency. The limitation of these techniques is that the algorithms need to be engineered again with each different use case. In this work, a Convolutional Neural Network (CNN) is used to classify the defects of the Chemical Mechanical Planarization (CMP) ring. The trained CNN model achieved an accuracy of 91% and the time taken for each inference process is around 1800 msec. To achieve computational efficiency, the CNN model is performed on the embedded device. The previous implementation of embedded CNN deploys OpenCL-based high-level synthesis accelerator on a high-end Field Programmable Gate Array (FPGA) board. In this work, the model inference is accelerated by PipeCNN FPGA implementation on Cyclone-VSE DE1-SoC, a low-end embedded FPGA board. Several configurations of hardware parameters are tested to search for the best setup of the FPGA resources. The hardware implementation has improved approximately seven times faster, as the inference time for each classification has improved from 1800 msec to 250 msec. However, the model implemented using the hardware is observed to produce lower inference accuracy as the accuracy drops from 91% to 81%.

Field-Programmable Gate Array (FPGA) based hardware accelerators offer reconfigurability, performance, adaptability, and good energy efficiency. The majority of Convolutional Neural Network (CNN) based inference systems are initially developed using standardized frameworks like PyTorch, Tensor Flow, and more. These Python or Python-like models can be mapped on FPGAs to build accelerators. Mapping frameworks to port designs on an FPGA convert the CNN models to high-level languages like C/C++ or OpenCL so that standard tools like high-level synthesis can facilitate the mapping of models on an FPGA. The logic utilization and performance of FPGA-based accelerators are dependent on the CNN network parameters, architectural selection (data-flow, pipelined, etc.), and synthesis-based control of design generation. A scalable multi-layer CNN hardware accelerator is modeled in Vitis 2020 HLS tool. Early estimation of performance and hardware resources helps choose the best CNN network before those are executed for time-consuming high-level synthesis and physical design mapping for FPGAs. We present various Machine Learning (ML) models to estimate the Logic Utilization and Computation Time from the Python design descriptions of CNNs. Our results show a very successful and accurate estimation for performance and resource utilization over various multi-layer CNN networks in negligible time before running HLS synthesis.

This paper presents a thorough discussion of the proposed field-programmable gate array (FPGA) implementation for fringe pattern demodulation using the one-dimensional continuous wavelet transform (1D-CWT) algorithm. This algorithm is also known as wavelet transform profilometry. Initially, the 1D-CWT is programmed using the C programming language and compiled into VHDL using the ImpulseC tool. This VHDL code is implemented on the Altera Cyclone IV GX EP4CGX150DF31C7 FPGA. A fringe pattern image with a size of 512×512 pixels is presented to the FPGA, which processes the image using the 1D-CWT algorithm. The FPGA requires approximately 100 ms to process the image and produce a wrapped phase map. For performance comparison purposes, the 1D-CWT algorithm is programmed using the C language. The C code is then compiled using the Intel compiler version 13.0. The compiled code is run on a Dell Precision state-of-the-art workstation. The time required to process the fringe pattern image is approximately 1 s. In order to further reduce the execution time, the 1D-CWT is reprogramed using Intel Integrated Primitive Performance (IPP) Library Version 7.1. The execution time was reduced to approximately 650 ms. This confirms that at least sixfold speedup was gained using FPGA implementation over a state-of-the-art workstation that executes heavily optimized implementation of the 1D-CWT algorithm.

Convolution Neural Networks (CNNs) are responsible for the major discoveries in image classification and they are considered as the core of most current computer vision systems. In the implementation of deep CNN, Field-Programmable Gate Arrays (FPGAs) offer a promising paradigm towards major leaps in computational performance while achieving high-energy efficiency. Although current CNN accelerations on FPGA have demonstrated good performance, one major issue is that previously proposed implementation do not achieve a good balance between latency, precision, and hardware complexity. In order to overcome this problem, this paper proposes a highly optimized FPGA implementation of a CNN, named NASH-CNN (Neuro-inspired ArchitectureS in Hardware for CNN). An application for handwritten digit recognition, based on MNIST dataset, is evaluated. The experiment shows that our implementation achieves better performance/accuracy/complexity balance when compared to previously proposed schemes.