SNA: A Siamese Network Accelerator to Exploit the Model-Level Parallelism of Hybrid Network Structure

This paper presents Timeloop, an infrastructure for evaluating and exploring the architecture design space of deep neural network (DNN) accelerators. Timeloop uses a concise and unified representation of the key architecture and implementation attributes of DNN accelerators to describe a broad space of hardware topologies. It can then emulate those topologies to generate an accurate projection of performance and energy efficiency for a DNN workload through a mapper that finds the best way to schedule operations and stage data on the specified architecture. This enables fair comparisons across different architectures and makes DNN accelerator design more systematic. This paper describes Timeloop's underlying models and algorithms in detail and shows results from case studies enabled by Timeloop, which provide interesting insights into the current state of DNN architecture design. In particular, they reveal that dataflow and memory hierarchy co-design plays a critical role in optimizing energy efficiency. Also, there is currently still not a single architecture that achieves the best performance and energy efficiency across a diverse set of workloads due to flexibility and efficiency trade-offs. These results provide inspiration into possible directions for DNN accelerator research.

This article presents an energy-harvester-powered ultra-low power (ULP) vibration-based condition monitoring (VbCM) chip with a digital compute-in-memory (CIM)-based deep neural network (DNN) accelerator. The VbCM chip achieves end-to-end signal processing, including a piezoelectric (PZ)-based accelerometer sensor and its readout circuit (RoC), a digital CIM-based DNN accelerator and a ULP radio system. To perform the targeted rotating bearing anomaly detection task with best-in-class accelerator power consumption and energy efficiency, the chip integrates several technologies from the algorithm level to the hardware level, including: 1) Compressing deep neural network (C-DNN) to reduce the network size through a feature compressor module (FCM); 2) Signal chain characteristic adaptation from direct training of the time-domain feature extractor (TD-FEx); 3) ULP 13T SRAM CIM bitcell to perform high energy efficiency in-cell multiplication; and 4) ripple counter (RCNT)-based accumulation scheme to improve the overall energy efficiency of the accelerator under a 0.35 V supply voltage. The VbCM chip achieves 90.7 nW total power with an inference FR of 20 frames/s, and the DNN accelerator’s energy efficiency achieves 18.8 fJ/MAC. With all the weight parameters statically stored in the CIM macros, the accelerator can operate with an inference accuracy of 91.3% while consuming 24.7 nW.

Energy-efficient Resistive RAM (ReRAM) based deep neural network (DNN) accelerator suffers from severe Stuck-At-Fault (SAF) problem that drastically degrades the inference accuracy. The SAF problem gets even worse in realistic ReRAM devices with low cell resolution. To address the issue, we propose a fault-resilient DNN accelerator based on realistic ReRAM devices. We first analyze the SAF problem in a realistic ReRAM device and propose a 3-stage offline fault-resilient compilation and lightweight online compensation. The proposed work enables the reliable execution of DNN with only 5% area and 0.8% energy overhead from the ideal ReRAM-based DNN accelerator.

The conventional Deep Neural Networks (DNNs) accelerator is usually designed based on array-based processing element computing. By using the array-based computation, memory access can be reduced efficiently based on a specific dataflow. However, the computing flexibility of the contemporary DNN accelerators is usually restricted because they support the monotonous computing dataflow. Hence, the computing efficiency will be degraded under the different hyperparameters of the target DNN (e.g., kernel size, layers, etc.) Because of the high flexibility and scalability of Network on Chip (NoC) interconnection, the NoC-based DNN design methodology becomes an attractive design paradigm, which reduces the design complexity of the DNN accelerator implementation. However, the current NoC-based DNN designs usually assume that the entire DNN model can be mapped to the target NoC. In this way, the area overhead will be larger with respect to the increasing DNN scale. To solve this problem, we propose a dynamic mapping algorithm, called: dense mapping. The dense mapping is used to map the neuron operations to the NoC as long as the available computing resources are enough for the neuron operations. Besides, an input sharing mechanism is proposed to reuse input data. In this way, we can not only process a DNN model on a small-scale NoC but also decrease the number of memory access by using the proposed input sharing mechanism. Compared with the related work, the proposed approaches help to reduce 60.5% number of memory access and improve 96.5% throughput due to fewer memory accesses.

FPGAs have been rapidly adopted for acceleration of Deep Neural Networks (DNNs) with improved latency and energy efficiency compared to CPU and GPU-based implementations. High-level synthesis (HLS) is an effective design flow for DNNs due to improved productivity, debugging, and design space exploration ability. However, optimizing large neural networks under resource constraints for FPGAs is still a key challenge. In this paper, we present a series of effective design techniques for implementing DNNs on FPGAs with high performance and energy efficiency. These include the use of configurable DNN IPs, performance and resource modeling, resource allocation across DNN layers, and DNN reduction and re-training. We showcase several design solutions including Long-term Recurrent Convolution Network (LRCN) for video captioning, Inception module for FaceNet face recognition, as well as Long Short-Term Memory (LSTM) for sound recognition. These and other similar DNN solutions are ideal implementations to be deployed in vision or sound based IoT applications.

This paper presents an optimization framework named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NeuroSpector</i> that systematically analyzes the dataflow of deep neural network (DNN) accelerators and rapidly identifies optimal execution methods. The proposed methodology is demonstrated to work effectively with a variety of accelerator architectures and DNN workloads. It has been a baffling challenge to devise scheduling schemes for neural accelerators to maximize energy efficiency and performance. The challenge lies in that hardware specifications associated with multi-dimensional DNN data create an enormous number of possible scheduling options that can be exerted on accelerators. Related work suggested various techniques to solve the challenge encompassing brute-force search of massive solution spaces pruned by user constraints, solving the objective functions of system models, learning-based optimization, etc. However, each suggested technique was devised only for a specific accelerator model. Therefore, we find that they are not adaptively applicable to different accelerators and DNN workloads in that they produce hit-or-miss results with 100.1% greater energy and cycles on average compared to optimal scheduling schemes obtained from fully comprehensive brute-force searches. In contrast, NeuroSpector identifies efficient execution methods for various accelerators and workloads with only 1.5% differences on average to the optimal scheduling solutions. The optimization strategy of NeuroSpector is based on an observation that optimal executions are strongly correlated with minimizing data movements to the lower-level memory hierarchy of accelerators rather than maximizing the utilization of processing elements. Thus, NeuroSpector prioritizes optimizing lower-level components in the accelerator hierarchy, which is proven highly effective for various accelerators and DNN workloads.

The crossbar structure of the nonvolatile memory enables highly parallel and energy-efficient analog matrix-vector-multiply (MVM) operations. To exploit its efficiency, existing works design a mixed-signal deep neural network (DNN) accelerator, which offloads low-precision MVM operations to the memory array. However, they fail to accurately and efficiently support the low-precision networks due to their naive ADC designs. In addition, they cannot be applied to the latest technology nodes due to their premature RRAM-based memory array. In this work, we present 3D-FPIM, an energy-efficient and robust mixed-signal DNN acceleration system 3D-FPIM is a full-stack 3D NAND flash-based architecture to accurately deploy low-precision networks. We design the hardware stack by carefully architecting a specialized analog-to-digital conversion method and utilizing the three-dimensional structure to achieve high accuracy, energy efficiency, and robustness. To accurately and efficiently deploy the networks, we provide a DNN retraining framework and a customized compiler. For evaluation, we implement an industry-validated circuit-level simulator. The result shows that 3D-FPIM achieves an average of 2.09x higher performance per area and 13.18x higher energy efficiency compared to the baseline 2D RRAM-based accelerator.

A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks

Deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance in classification, natural language processing (NLP), and regression tasks. However, there is still a great gap between DCNNs and the human brain in terms of computation efficiency. Inspired by neural synaptic plasticity and stochastic computing (SC), we propose neural synaptic plasticity-inspired computing (NSPC) to simulate the human brain's neural network activity for inference tasks with simple logic gates. The multiplication and accumulation (MAC) is transformed by the wire connectivity in NSPC, which only requires bundles of wires and small width adders. To this end, the NSPC imitates the structure of neural synaptic plasticity from a circuit wires connection perspective. Furthermore, from the principle of NSPC, we use a data mapping method to convert the convolution operations to matrix multiplications. Based on the methodology of NSPC, fully-pipelined and low latency architecture is designed. The proposed NSPC accelerator exhibits high hardware efficiency while maintaining a comparable network accuracy level. The NSPC based DCNN accelerator (NSPC-CNN) processes DCNN at 1.5625M images/s with a power dissipation of 15.42 W and an area of 36.4 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . The NSPC based deep neural network (DNN) accelerator (NSPC-DNN) that implements three fully connected layers DNN consumes only 6.6 mm2 area and 2.93 W power, and achieves a throughput of 400M images/s. Compared with conventional fixed-point implementations, the NSPC-CNN achieves 2.77× area efficiency, 2.25× power efficiency; the proposed NSPC-DNN exhibits 2.31× area efficiency and 2.09× power efficiency.

The acceleration of deep neural networks (DNNs) on field-programmable gate arrays (FPGAs) has gained widespread adoption due to their high performance and energy efficiency. However, the shared hardware nature of FPGAs introduces new security vulnerabilities, especially in multi-tenant cloud and IoT environments. Among these, remote voltage side-channel attacks remain largely underexplored. In this paper, we present a novel attack paradigm that enables remote extraction of sensitive information through voltage fluctuation analysis on Xilinx 7-series FPGAs. Unlike traditional side-channel approaches requiring physical access or external sensors, our method leverages the embedded XADC module to non-invasively capture power-induced voltage variations caused by victim DNN computations. To effectively extract useful information from voltage fluctuations, we develop a customized processing pipeline tailored to the inherently low signal-to-noise ratio present in remote measurements. This pipeline employs dynamic thresholding, image-centric quantization, and multistage denoising to extract meaningful patterns from raw voltage traces. Applying this method, we successfully reconstruct input images with an average Normalized Cross-Correlation (NCC) of 73.86% against the original MNIST inputs, demonstrating high visual similarity and validating the feasibility of the attack. Our findings reveal critical security flaws in the shared power delivery and sensing infrastructure of modern FPGAs. This work underscores the need for hardware-level isolation and mitigation strategies, and expands the attack surface model for future FPGA-based cloud and edge deployments.

Data multiplexed and hardware reused architecture for deep neural network accelerator

Machine learning has surged in popularity in recent years, with deep neural networks (DNNs) becoming a cornerstone of applications such as autonomous driving, augmented reality, and natural language processing, owing to their remarkable accuracy and scalability. As the demand for greater portability and flexibility grows, there is a strong push to optimize these applications for resource-constrained edge devices like smartphones and IoT devices.However, deploying DNN models on such devices presents significant challenges, primarily due to the large size of these models, which is a key contributor to their high performance. For instance, the DeepSpeech 2 model outperforms its predecessor, DeepSpeech 1, but its size is eight times larger, encompassing approximately 68 million parameters. Similarly, other networks designed for various tasks, including convolutional neural networks like VGG and transformer-based networks like BERT and GPT, also exhibit substantial sizes. For example, the GPT-3 model, a predecessor to ChatGPT, consists of an astonishing 175 billion parameters. Consequently, fitting DNN models onto resource-limited edge devices is a formidable challenge. My thesis addresses these challenges with a three-part approach: * High-Performance DNN Accelerators: I will develop DNN accelerators that excel in high performance, low latency, and energy efficiency through an algorithm and hardware co-design across diverse FPGA architectures (Chapter 2). * Real-Time DNN Execution: I aim to achieve exceptional accuracy and real-time execution for various large-scale DNNs on off-the-shelf mobile devices, IoT devices, and other platforms (Chapter 3). * CompVQC Framework: I will explore a systematic framework, CompVQC, to reduce the circuit length of quantum neural networks (QNNs) through model compression, which can achieve up to a 20% improvement in accuracy. This reveals that compilation-aware compression can enhance the robustness of QNNs on near-term noisy quantum devices (Chapter 4). The field of AI has recently undergone a significant transformation, ushering in the era of generative AI, where models like ChatGPT and Stable Diffusion create realistic and creative content for a wide range of applications. As generative AI becomes more prevalent, it is crucial to adapt algorithm-hardware co-design strategies to meet the specific needs of these models, particularly on edge devices. In the final chapter (Chapter 5), I will introduce my ongoing and future work focused on activating generative AI on the edge.--Author's abstract

Deep Neural Networks (DNNs) are the current state of the art for various tasks such as object detection, natural language processing and semantic segmentation. These networks are massively parallel, hierarchical models with each level of hierarchy performing millions of operations on a single input. The enormous amount of parallel computation makes these DNNs suitable for custom acceleration. Custom accelerators can provide real time inference of DNNs at low power thus enabling widespread embedded deployment. In this paper, we present Snowflake, a high efficiency, low power accelerator for DNNs. Snowflake was designed to achieve optimum occupancy at low bandwidths and it is agnostic to the network architecture. Snowflake was implemented on the Xilinx Zynq XC7Z045 APSoC and achieves a peak performance of 128 G-ops/s. Snowflake is able to maintain a throughput of 98 FPS on AlexNet while averaging 1.2 GB/s of memory bandwidth.

Off-chip memory, such as DRAM, its access energy cost is orders of magnitude higher than other operations such as multiply and accumulate, thereby dominating the system energy consumption. Therefore, optimizing the access of the off-chip memory is crucial to further improve the energy efficiency of the deep neural network (DNN) accelerator. Towards this, this brief proposed an adaptive scheduling algorithm to minimize the DRAM access. Compared with the previous works, it can not only dynamically determine the data partition and the data type that will be reused, but also considered the constraints between adjacent layer, that is, if the output feature map of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i$ </tex-math></inline-formula> th layer is divided into N parts, the output feature map of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i+1$ </tex-math></inline-formula> th layer can only be divided into N parts or write back to the off-chip memory. Therefore, a minimize and realizable memory access solution can be obtained. Choosing 3 popular networks UNet, VGG-16 and MobileNet as benchmarks, the experiment results show that our scheduling algorithm can achieve a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$34.46\% \sim 93.42\%$ </tex-math></inline-formula> reduction in energy consumption of DRAM access and a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$34.34\% \sim 93.37\%$ </tex-math></inline-formula> reduction in DRAM access latency when compared to previous works.

With the advent of Deep Neural Network (DNN) accelerators, permanent faults are increasingly becoming a serious challenge for DNN hardware accelerator, as they can severely degrade DNN inference accuracy. The State-of-the-art works address this issue by adding homogeneous redundant Processing Elements (PEs) to the DNN accelerator’s central computing array, or bypassing faulty PEs directly. However, such designs induce inference loss, extra hardware cost, and performance overhead. Moreover, current designs are able to only deal with a limited number of faults due to costs. In this paper, we propose FSA, a Fault-tolerant Systolic Array-based DNN accelerator with the goal of maintaining DNN inference accuracy in the presence of permanent faults. The key feature of the proposed FSA is a unified re-computing module (RCM) that dynamically recalculates the required DNN computations that are supposed to be accomplished by faulty PEs with minimal latency and power consumption. Simulation results show that the proposed FSA reduces inference accuracy loss by 46%, improves execution time by 23%, and reduces energy consumption by 35% on average, as compared to existing designs.