Research on Dynamic Reconfigurable Convolutional Neural Network Accelerator

Convolutional neural networks (CNNs) have demonstrated remarkable performance in many areas but require significant computation and storage resources. Quantization is an effective method to reduce CNN complexity and implementation. The main research objective is to develop a scalable quantization algorithm for CNN hardware design and model the performance metrics for the purpose of CNN implementation in resource-constrained devices (RCDs) and optimizing layers in deep neural networks (DNNs). The algorithm novelty is based on blending two quantization techniques to perform full model quantization with optimum accuracy, and without additional neurons. The algorithm is applied to a selected CNN model and implemented on an FPGA. Implementing CNN using broad data is not possible due to capacity issues. With the proposed quantization algorithm, we succeeded in implementing the model on the FPGA using 16-, 12-, and 8-bit quantization. Compared to the 16-bit design, the 8-bit design offers a 44% decrease in resource utilization, and achieves power and energy reductions of 41% and 42%, respectively. Models show that trading off one quantization bit yields savings of approximately 5.4K LUTs, 4% logic utilization, 46.9 mW power, and 147 μJ energy. The models were also used to estimate performance metrics for a sample DNN design.

Convolutional Neural Network (CNN) has achieved great success in various fields of machine vision, such as image classification and recognition, image segmentation, and video analysis. In specific applications, it is often necessary to customize the network structure by altering image size, convolution kernel size, pooling size, network architecture, and the number of network layers. These customizations pose significant challenges to the architecture of CNN accelerator especially in real-time systems, and as the number of these network variables increases, the internal convolution calculations become extremely demanding. The inference speed and energy consumption of CNN are becoming more and more important, which requires the design of the accelerator to adapt to different network architecture and run efficiently. Field-programmable gate arrays (FPGAs) is an ideal choice for a CNN accelerator due to its high programmability and low power consumption. To address the aforementioned challenges, we present NRUCA, a novel network reconfigurable universal CNN accelerator based on FPGA. On one hand, we have designed a flexible architecture that can dynamically configure CNN’s network structure parameters. These parameters can be sent to the FPGA via a configuration file, enabling the same design to run different CNN networks and allowing to modify on the fly. This forms the basis of the architecture’s adaptability presented in this paper. On the other hand, we employ a multi-channel parallel and efficient pipeline matrix multiplication architecture to implement the convolution and fully-connected layers, which constitute the majority of the computational load in CNNs. By utilizing an innovative inverted matrix multiplication algorithm and an interleaving cache data method, we reduce the internal cache required for multi-channel data. Furthermore, most of the intermediate calculation data does not need to be output to DDR, significantly improving the operation efficiency of the accelerator. We also fully leverage the FPGA chip architecture to compile multiple calculation kernels, which can be flexibly scheduled and combined to cater to various application scenarios. Based on the flexible and efficient overall design architecture, the layout of various resources of FPGA is balanced, which enables the designed FPGA project to compile four acceleration kernels with the running clock up to 300MHz in Xilinx Vitis and test them on Xilinx Alveo U250. We verified the adaptability and acceleration capabilities of our FPGA project using LeNet, AlexNet, VGG11, VGG13, VGG16, and VGG19 networks. Experiments demonstrated that our FPGA architecture achieves 33X and 35X energy savings compared to the Intel Xeon 5220R CPU, and 1.08X and 1.05X energy savings compared to the Nvidia Tesla P100 GPU, when accelerating AlexNet and VGG16, respectively.

Over the last years, Convolutional Neural Networks (CNNs) have been widely used in remote sensing applications, such as marine surveillance, traffic management or road networks detection. However, since CNNs have extremely high computation, bandwith and memory requirements, the hardware implementation of a CNN on space-grade devices like FPGAs for the on-board processing of the acquired images has brought many challenges. In fact, FPGAs show great advantages in terms of reconfigurability and performance-power ratio, whichmakes them a suitable election to deploy CNNs, although the computational capabilities of this type of hardware devices available on-board is limited, so implementations have to be carefully planned. In this paper, the authors present their work towards the implementation of an efficient CNN onto a spacegrade FPGA in order to achieve the on-board processing of very-high resolution remotely sensed images as soon as the data are provided by the sensor. All this work has been conducted within the EU-funded VIDEO project. As it will be presented in this paper, the work includes the introduction of a methodology based on the project constraints, the evaluation of different state-of-the-art CNN architectures by means of a new efficiency measurement also proposed in this work, the introduction of a new efficient CNN architecture, and finally, its optimized hardware implementation by means of high-level synthesis tools.

Convolutional Neural Network (CNN) accelerators are popular specialized platforms for efficient CNN processing. As semiconductor manufacturing technology scales down to nano scale, process variation dramatically affects the chip's quality. Process variation causes delay variation within the chip due to transistor parameter differences. CNN accelerators adopt a large number of Processing Elements (PEs) for parallel computing, which are highly susceptible to process variation effects. Fast CNN processing desires consistent performance among PEs, otherwise the processing speed is limited by the slowest PE within the chip. In this work, we first quantitatively model and analyze the impact of process variation on CNN accelerator's operating frequency. We further analyze the utilization of CNN accelerator and the characteristics of CNN models. We then leverage the PE underutilization to propose a sub-matrix reformation mechanism and leverage the pixel similarity of images to propose a weight transfer technique. Both techniques are able to tolerate the low-frequency PEs, and achieve performance improvement at chip level. Evaluation results show our techniques are able to achieve significant processing speed improvement with negligible accuracy loss.

Convolutional neural network (CNN) accelerators have achieved great success from cloud to edge scenarios. However, given the trend towards even larger and deeper neural network models, it remains a challenging problem to efficiently process these CNNs especially on edge devices with limited energy budget. Accordingly, reducing the energy consumption is of paramount importance for sustainable CNN accelerators. In this paper, we propose AdaPrune, a novel pruning technique that reduces model size and computation to achieve performance improvement and energy savings for CNN accelerators. Unlike previous pruning techniques that sacrifice either computational regularity or accuracy, AdaPrune maintains both by customizing CNN pruning for the underlying accelerators to maximally leverage the sparsity benefits. AdaPrune consists of two techniques: input channel group pruning and output channel group pruning. By analyzing the weight fetching patterns of sparse CNN accelerators, AdaPrune adaptively switches between the two techniques to guarantee that the zeros are evenly distributed in each fetching group. In doing so, the pruned network structure preserves customized computational regularity for the underlying accelerators, thereby boosting the performance and energy efficiency. We evaluate AdaPrune on three sparse CNN accelerators with different spatial tiling strategies. The experimental results show that AdaPrune achieves up to 1.6× performance speedup, and 1.5× energy savings compared to unstructured pruning.

This article proposes an uninterrupted processing technique for the convolutional neural network (CNN) accelerator. It primarily allows the CNN accelerator to simultaneously perform both processing element (PE) operation and data fetching that reduces its latency and enhances the achievable throughput. Corresponding to the suggested technique, this work also presents a low latency VLSI-architecture of the CNN accelerator using the new random access line-buffer (RALB)-based design of PE array. Subsequently, the proposed CNN-accelerator architecture has been further optimized by reusing the local data in PE array, incurring better energy conservation. Our CNN accelerator has been hardware implemented on Zynq-UltraScale + MPSoC-ZCU102 FPGA board, and it operates at a maximum clock frequency of 340 MHz, consuming 4.11 W of total power. In addition, the suggested CNN accelerator with 864 PEs delivers a peak throughput of 587.52 GOPs and an adequate energy efficiency of 142.95 GOPs/W. Comparison of aforementioned implementation results with the literature has shown that our CNN accelerator delivers 33.42% higher throughput and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.24\times $ </tex-math></inline-formula> better energy efficiency than the state-of-the-art work. Eventually, the field-programmable gate array (FPGA) prototype of the proposed CNN accelerator has been functionally validated using the real-world test setup for the detection of object from input image, using the GoogLeNet neural network.

Convolutional neural network (CNN) based machine learning requires a highly parallel as well as low power consumption (including leakage power) hardware accelerator. In this paper, we will present a digital ReRAM crossbar based CNN accelerator that can achieve significantly higher throughput and lower power consumption than state-of-arts. The CNN is trained with binary constraints on both weights and activations such that all operations become bitwise. With further use of 1-bit comparator, the bitwise CNN model can be naturally realized on a digital ReRAM-crossbar device. A novel sneak-path-free ReRAM-crossbar is further utilized for large-scale realization. Simulation experiments show that the bitwise CNN accelerator on the digital ReRAM crossbar achieves 98.3% and 91.4% accuracy on MNIST and CIFAR-10 benchmarks, respectively. Moreover, it has a peak throughput of 792GOPS at the power consumption of 6.3mW, which is 18.86 times higher throughput and 44.1 times lower power than CMOS CNN (non-binary) accelerators.

Technological advancements have had a significant impact on the education sector, including the application of Convolutional Neural Networks (CNN) for facial image analysis. This research aims to implement CNN to measure students' learning concentration levels. The FER2013 dataset, which includes seven emotion classifications and comprises 28,709 images for training data, is used as the database. The data is processed through rescaling and augmentation to prepare the CNN model. The model consists of several convolutional layers, pooling layers, and fully connected layers designed to extract crucial features from facial images. Evaluation results demonstrate a very high accuracy of 94.95% on training data, indicating that the model effectively recognizes complex patterns within the data. Although there is a higher loss value of 157% and a decreased accuracy of 62.75% on validation data, this suggests that the model possesses a strong foundational capability and can still be improved through further adjustments. With high accuracy in training and promising validation results, the model shows substantial potential for real-world application, where it can assist teachers in understanding students' emotional responses in real-time. The implementation of CNN aids educators in comprehending students' emotional responses and adapting their teaching methods more effectively, thereby creating a more conducive learning environment and enhancing students' academic and social development. These findings also open opportunities for further research to improve the performance and generalization of the model on unseen data, making this technology an increasingly reliable tool in education

Deep convolutional neural network (DCNN) is an advanced technology in image recognition. Because of extreme computing resource requirements, DCNN implementation with software alone cannot achieve real-time requirement. Therefore, the need to implement DCNN accelerator hardware is increasing. In this paper, we present a field programmable gate array (FPGA)-based hardware accelerator design of DCNN targeting handwritten Hangul character recognition application. Also, we present design optimization techniques in SDAccel environments for searching the optimal FPGA design space. The techniques we used include memory access optimization and computing unit parallelism, and data conversion. We achieved about 11.19 ms recognition time per character with Xilinx FPGA accelerator. Our design optimization was performed with Xilinx HLS and SDAccel environment targeting Kintex XCKU115 FPGA from Xilinx. Our design outperforms CPU in terms of energy efficiency (the number of samples per unit energy) by 5.88 times, and GPGPU in terms of energy efficiency by 5 times. We expect the research results will be an alternative to GPGPU solution for real-time applications, especially in data centers or server farms where energy consumption is a critical problem.

We empirically evaluate an undervolting technique, i.e., underscaling the\ncircuit supply voltage below the nominal level, to improve the power-efficiency\nof Convolutional Neural Network (CNN) accelerators mapped to Field Programmable\nGate Arrays (FPGAs). Undervolting below a safe voltage level can lead to timing\nfaults due to excessive circuit latency increase. We evaluate the\nreliability-power trade-off for such accelerators. Specifically, we\nexperimentally study the reduced-voltage operation of multiple components of\nreal FPGAs, characterize the corresponding reliability behavior of CNN\naccelerators, propose techniques to minimize the drawbacks of reduced-voltage\noperation, and combine undervolting with architectural CNN optimization\ntechniques, i.e., quantization and pruning. We investigate the effect of\nenvironmental temperature on the reliability-power trade-off of such\naccelerators. We perform experiments on three identical samples of modern\nXilinx ZCU102 FPGA platforms with five state-of-the-art image classification\nCNN benchmarks. This approach allows us to study the effects of our\nundervolting technique for both software and hardware variability. We achieve\nmore than 3X power-efficiency (GOPs/W) gain via undervolting. 2.6X of this gain\nis the result of eliminating the voltage guardband region, i.e., the safe\nvoltage region below the nominal level that is set by FPGA vendor to ensure\ncorrect functionality in worst-case environmental and circuit conditions. 43%\nof the power-efficiency gain is due to further undervolting below the\nguardband, which comes at the cost of accuracy loss in the CNN accelerator. We\nevaluate an effective frequency underscaling technique that prevents this\naccuracy loss, and find that it reduces the power-efficiency gain from 43% to\n25%.\n

Randomly switching neurons ON/OFF while training and inference process is an interesting characteristic of biological neural networks, that potentially results in inherent adaptability and creativity expressed by human mind. Dropouts inspire from this random switching behaviour and in the artificial neural network they are used as a regularization techniques to reduce the impact of over-fitting during the training. The energy-efficient digital implementations of convolutional neural networks (CNN) have been on the rise for edge computing IoT applications. Pruning larger networks and optimization for performance accuracy has been the main direction of work in this field. As opposed to this approach, we propose to build a near-sensor analogue CNN with high-density memristor crossbar arrays. Since several active elements such as amplifiers are used in analogue designs, energy efficiency becomes a main challenge. To address this, we extend the idea of using dropouts in training to also the inference stage. The CNN implementations require a subsampling layer, which is implemented as a mean pooling layer in the design to ensure lower energy consumption. Along with the dropouts, we also investigate the effect of non-idealities of memristor and that of the network.

Background: Memristor-based implementations of Convolutional Neural Networks (CNNs) offer the potential to reduce power consumption and improve computational efficiency significantly. However, the non-ideal characteristics of memristive devices, such as variability and limited switching behavior, can introduce nonlinearity and degrade network performance. It is therefore essential to simulate memristor-based deep learning systems and evaluate the impact of these imperfections before proceeding with hardware-level deployment. Objective: This study aims to design a lightweight memristor-based MobileNetV2 model for smoking behavior detection and analyze the influence of memristor non-idealities on the network’s detection accuracy. Methods: A lightweight MobileNetV2 network was constructed and trained on a self-developed dataset for smoking behavior detection, achieving a peak accuracy of 92.25%. Based on the trained model, a memristor-based MobileNetV2 architecture was implemented using the Mem- Torch simulation framework. Two memristor models, VTEAM and Data-Driven Verilog-A RRAM, were employed to replace the convolutional and fully connected layers with memristor crossbar arrays. The impact of non-ideal memristor properties, including conductance variability and resistance state faults, was systematically evaluated. Results: Simulation results demonstrated that the memristor-based MobileNetV2 network achieved detection accuracies of 92.11% with the VTEAM model and 91.82% with the Verilog-A RRAM model. As device non-idealities increased, the detection accuracy declined significantly, indicating a strong correlation between memristor imperfections and the performance of CNN- based target detection tasks. Conclusion: Memristor non-idealities substantially affect the performance of neuromorphic CNNs. This study provides targeted analyses and proposes mitigation strategies, such as differential encoding and improved fabrication techniques, to support the practical deployment of memristor- based neural networks in energy-efficient AI applications.

Effective disease control in crops is essential for ensuring food security, as numerous diseases can significantly reduce agricultural productivity. Leaf disease is a crucial determinant of crop yield and quality. Potatoes are widely regarded as a vital staple in diets around the world. Potato blight, often known as late blight, is a severe disease impacting Solanaceae plants, particularly the potato. The oomycete Phytophthora infestans serves as the causal agent, posing a significant threat to potato crops by reducing yields and leading to financial losses. Identifying potato diseases is essential for ensuring food security and minimizing economic losses in agriculture. Numerous studies have explored the applications of machine learning and deep learning technologies for detecting blight diseases in affected crops. The proposed methodology outlined in our study offers a promising and effective means to enhance the accuracy of potato late blight classification. Advancements in technology can significantly improve the identification and treatment of diseases. Deep learning algorithms have been effectively employed in a range of practical applications. In our investigation, convolutional neural network models are used to analyze images of leaves. The primary aim of this initiative is to gain insights into and enhance the capacity to recognize diseases in their early phases. After the input images undergo preprocessing, the target region is distinctly isolated. CNN examines images to identify unique features, including edges, colours, and textures. This study evaluates the effectiveness of the proposed model through a diverse dataset of potato leaf images, making essential contributions to the domain of automated disease detection systems. The results of this study could enhance techniques for agricultural management and strengthen the sustainability of agriculture. This study introduces an improved CNN model designed to identify the blight disease state in potato leaf images, providing a valuable tool for farmers. The findings may aid in the evaluation and implementation of Convolutional Neural Network (CNN) models for classifying proximal sensing images of potato plant leaves. Additional investigations could leverage this modelling framework and have the opportunity to examine various other factors to identify leaf infections at an earlier stage for crop protection.

ERDSE: efficient reinforcement learning based design space exploration method for CNN accelerator on resource limited platform

Convolutional neural networks are promising tool for pattern recognition. Implementations of convolutional neural networks require a significant number of computations during training and processing. In this paper we propose to use Winograd method to reduce computational complexity in convolutional layers of convolutional neural networks. Theoretical analysis of the convolutional layer’s computational complexity of convolutional neural networks showed that the use of the Winograd method can reduce the computational complexity compared with direct method by 26,75%, 17,01%, with filter sizes of 3×3, 5×5, respectively. A promising area of further research is the hardware implementation of convolutional layers with Winograd calculations on modern computing devices, such as FPGA and ASIC.