A configurable SIMD architecture with explicit datapath for intelligent learning

SIMD (Single Instruction Multiple Data) architectures are widely used in application domains like the wireless communication, video and audio processing, and control engineering. The abundant data parallelism makes the SIMD architecture the proper match in data processing and performance improvement. However, there are also critical inefficiencies in current SIMD architectures. To understand such inefficiency, we carry out a deep investigation in the main components of Long Term Evolution (LTE) protocol, which is an important wireless communication protocol. Performance investigation is taken on a cycle-accurate simulator, featuring the main characteristics of existing SIMD architectures. Based on the investigation, we locate the inefficiencies in two aspects: the data communication operations among different processing units and the support for matrix-style computations. We have also carried out studies with enhanced SIMD architectures in the above two aspects. The overall performance of SIMD architectures can be greatly improved.

Due to the high computational complexity of the full-search algorithm for video coding, techniques to accelerate the execution of the full-search algorithm are needed for real-time implementations of current video compression standards. In this paper, a technique of using eight-bit partial sums of four luminance values is proposed to reduce the computational complexity of the original full-search algorithm without loss of its accuracy. Furthermore, since the proposed technique is suitable for the implementation on the single instruction multiple data (SIMD) architectures, the byte-type data-parallelism of the SIMD architecture can be utilized to further accelerate the execution of the proposed technique. Simulation results for the benchmark video test sequences demonstrate that the proposed technique can significantly accelerate the execution of the full-search algorithm on SIMD architectures without loss of its accuracy.

Software defined radio (SDR) systems require programmable architectures that provide high performance combined with high energy efficiency. Wide single instruction, multiple data (SIMD) architectures could potentially satisfy these demands, but only few publications examine the interdependence between the SIMD width and the performance of typical SDR algorithms. In this paper, we present a detailed analysis of three SDR algorithm classes on a newly developed scalable SIMD architecture, which combines SIMD processing with long instruction word (LIW) execution. Although radix-2 and mixed-radix fast Fourier transform, sphere decoding, and low-density parity-check decoding all benefit from SIMD processing, our results show that there are different constraints on the performance.

This article discusses a MATLAB-to-C vectorizing compiler that exploits custom instructions, for example, for Single Instruction Multiple Data (SIMD) processing and instructions for complex arithmetic present in Application-Specific Instruction Set Processors (ASIPs). Custom instructions are represented via specialized intrinsic functions in the generated code, and the generated code can be used as input to any C/C++ compiler supporting the target processor. Furthermore, the specialized instruction set of the target processor is described in a parameterized way using a target processor-independent architecture description approach, thus allowing the support of any processor. The compiler has been used for the generation of application code for two different ASIPs for several benchmarks. The code generated by the compiler achieves a speedup between 2× --74× and 2× --97× compared to the code generated by the MathWorks MATLAB-to-C compiler. Experimental results also prove that the compiler efficiently exploits SIMD custom instructions achieving a 3.3 factor speedup compared to cases where no SIMD processing is used. Thus the compiler can be employed to reduce the development time/effort/cost and time to market through raising the abstraction of application design in an embedded systems/system-on-chip development context.

Energy efficiency has become one of the most important topics in computing. To meet the ever increasing demands of the mobile market, the next generation of processors will have to deliver a high compute performance at an extremely limited energy budget. Wide single instruction, multiple data (SIMD) architectures provide a promising solution, as they have the potential to achieve high compute performance at a low energy cost. We propose a configurable wide SIMD architecture that utilizes explicit datapath techniques to further optimize energy efficiency without sacrificing computational performance. To demonstrate the efficiency of the proposed architecture, multiple instantiations of the proposed wide SIMD architecture and its automatic bypassing counterpart, as well as a baseline RISC processor, are implemented. Extensive experimental results show that the proposed architecture is efficient and scalable in terms of area, performance, and energy. In a 128-PE SIMD processor, the proposed architecture is able to achieve an average of 206 times speed-up and reduces the total energy dissipation by 48.3 % on average and up to 94 %, compared to a reduced instruction set computing (RISC) processor. Compared to the corresponding SIMD architecture with automatic bypassing, an average of 64 % of all register file accesses is avoided by the 128-PE, explicitly bypassed SIMD. For total energy dissipation, an average of 27.5 %, and maximum of 43.0 %, reduction is achieved.

Convolutional Neural Networks (CNNs) are one of the most successful deep machine learning technologies for processing image, voice and video data. CNNs require large amounts of processing capacity and memory, which can exceed the resources of low power mobile and embedded systems. Several designs for hardware accelerators have been proposed for CNNs which typically contain large numbers of Multiply Accumulate (MAC) units. One approach to reducing data sizes and memory traffic in CNN accelerators is "weight sharing", where the full range of values in a trained CNN are put in bins and the bin index is stored instead of the original weight value. In this paper we propose a novel MAC circuit that exploits binning in weight-sharing CNNs. Rather than computing the MAC directly we instead count the frequency of each weight and place it in a bin. We then compute the accumulated value in a subsequent multiply phase. This allows hardware multipliers in the MAC circuit to be replaced with adders and selection logic. Experiments show that for the same clock speed our approach results in fewer gates, smaller logic, and reduced power.

The aim of this study is to present the steps of optimization the H.264 / AVC video decoder for MIPS32 DSP ASE SIMD (Single-instruction, multiple-data) processor architecture. In particular, SIMD architecture can be very efficient in the implementation of video applications which have simple operations on 8-bit or 16-bit samples. Besides already mentioned architecture it is also important to get familiar with the H.264 / AVC video code standards. The parts of decoder that were subject of optimization were detected as the biggest consumers of CPU resources. After optimization, overall system speedup gain was 36%.

In this work, carry-free redundant arithmetic based fused multiply-accumulate (MAC) units are designed. In the first design, a regular redundant carry-save MAC unit is designed using well known carry-save techniques. In the second design, a hybrid design is proposed to exploit fast carry chains of the FPGA together with double carry-save output encoding. The proposed scheme exploits fast-carry chains of the FPGA structure, and, multi-operand adders are divided into smaller blocks to increase the performance. The outputs of the multi-operand adders are not merged and the results are kept in double carry-save format where extra redundancy reduces critical path delay. Designed MAC units have 16×16-bit multiplier with 40-digit accumulate output for recursive multiply-add operations. The designs are synthesized on Altera <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TM</sup> Stratix III FPGAs and provide superior performance compared to conventional pipelined carry-propagate multiply-accumulate units. The fusion in the arithmetic structure provides best performance compared to conventional pipelined multiplier based structures, hard multiplier based MAC units, and carry free redundant arithmetic based MAC structures as well.

Modern Artificial Intelligence (AI) systems deploy Convolution Neural Networks (CNN) as they offer very high accuracy. Computational complexity of CNNs necessitates hardware acceleration, especially in mobile phones and other hand held devices due to stringent power and area budget. In this paper, we propose a systolic dataflow based accelerator architecture with the aim of improving energy efficiency as well as scalability. We demonstrate that a prototype of our proposed accelerator comprising 4096 Multiply-Accumulate (MAC) units is capable of achieving 8.89–9.54 TOPs/W power efficiency when executing a number of state of the art CNN models. We also observe that the throughput of the accelerator scales almost linearly with power dissipated. This is attributed to low power consumption due to minimal non-compute overhead. Our proposed accelerator outperforms state of the art accelerators power efficiency by factors of 2.6 to 3.8× while executing various layers of Inception V3 model.

There is a rapidly growing demand for using intelligent cameras for various applications in surveillance and identification. Most of these applications have real-time demands and require huge processing capacity. Face recognition is one of those applications highly in demand. In this paper we show that we can run face recognition in real-time by implementing the algorithm on an architecture which combines a massively parallel processor with a high performance Digital Signal Processor. In this paper we focus on the INCA+ intelligent camera. It contains a CMOS sensor, a Single Instruction Multiple Data (SIMD) processor [1] and a Very Long Instruction Word (VLIW) processor. The SIMD processor enables high-performance pixel processing and detects the interesting (face) regions from the video. It sends the regions of interest to the VLIW processor, which performs the actual face recognition using a neural network. With this architecture we perform face recognition from a 5-persons database at more than 200 faces per second. The performance is better than most recent high-end professional systems [2].

This paper illustrates the implementation of Multiply-Accumulate unit (MAC) to improve performance using the Ancient Indian Vedic Mathematics. Real-time signal processing needs high speed and high yield Multiplier-Accumulator (MAC) unit that consumes less power, which is always a key to achieve a high performance digital signal processing system. Speed of the multiplier is essential to MAC unit. Vedic Mathematics is the ancient mathematics. There are sixteenth unique sutras in the Vedic mathematics. The Sutras saves effort and time required in solving the problems as compared to the formal methods. The Sutra “Urdhav Triyagbhyam” (Vertical and Crosswise) is used inside the MAC unit as a multiplier. The coding is done in VHDL, synthesis is done in Xilinx ISE series and the FPGA synthesis is done using Xilinx Spartan library. The results show that design of MAC unit using Vedic multiplication is much more efficient in terms of delay and speed compared to conventional multiplication.

We propose a high-speed and energy-efficient two-cycle multiply-accumulate (MAC) architecture that supports two's complement numbers, and includes accumulation guard bits and saturation circuitry. The first MAC pipeline stage contains only partial-product generation circuitry and a reduction tree, while the second stage, thanks to a special sign-extension solution, implements all other functionality. Place-and-route evaluations using a 65-nm 1.1-V cell library show that the proposed architecture offers a 31% improvement in speed and a 32% reduction in energy per operation, averaged across operand sizes of 16, 32, 48, and 64 bits, over a reference two-cycle MAC architecture that employs a multiplier in the first stage and an accumulator in the second. When operating the proposed architecture at the lower frequency of the reference architecture the available timing slack can be used to downsize gates, resulting in a 52% reduction in energy compared to the reference. We extend the new architecture to create a versatile double-throughput MAC (DTMAC) unit that efficiently performs either multiply-accumulate or multiply operations for N-bit, 1 × N/2-bit, or 2 × N/2-bit operands. In comparison to a fixed-function 32-bit MAC unit, 16-bit multiply-accumulate operations can be executed with 67% higher energy efficiency on a 32-bit DTMAC unit. © 2006 IEEE.

In this article, we demonstrated a high-throughput gate-level-pipelined 8-bit multiply-accumulate (MAC) unit with a long feedback loop using low-voltage rapid single-flux quantum (LV-RSFQ) logic. The long feedback loop in the MAC unit is an obstacle for high-throughput operation because the logic gates must wait for the delayed inputs from the feedback loop. The LV-RSFQ logic makes high-frequency operation even more difficult by larger and more variable feedback delay. We design the feedback loop by using counter-flow clocking and adding many D flip-flops to divide the long feedback loop into shorter paths. The target clock frequency of the MAC unit with a feedback loop was set to 30 GHz by the experimental results of the MAC unit without a feedback loop. We model the clock frequency and its circuit overhead in a feedback loop to design the feedback loop in the MAC unit achieving 30 GHz with a minimum overhead. The test chips are fabricated using the national institute of advanced industrial science and technology (AIST) 10-kA/cm <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\boldsymbol{^{2}}$</tex-math></inline-formula> Advanced Process 2. We have successfully obtained high-throughput 30-GHz operations in the LV-RSFQ MAC unit with a long feedback loop by using the model-based design. The maximum operating frequency of the MAC unit reaches 40 GHz.

As deep learning networks mature and improve classification performance, a significant challenge is their deployment in embedded settings. Modern network typologies, such as convolutional neural networks, can be very deep and impose considerable complexity that is often not feasible in resource bound, real-time systems. Processing of these networks requires high levels of parallelization, maximizing data throughput, and support for different network types, while minimizing power and resource consumption. In response to these requirements, in this paper, we present a low power FPGA based neural network accelerator named SCALENet: a SCalable Low power AccELerator for real-time deep neural Networks. Key features include optimization for power with coarse and fine grain scheduler, implementation flexibility with hardware only or hardware/software co-design, and acceleration for both fully connected and convolutional layers. The experimental results evaluate SCALENet against two different neural network applications: image processing, and biomedical seizure detection. The image processing networks, implemented on SCALENet, trained on the CIFAR-10 and ImageNet datasets with eight different networks, are implemented on an Arty A7 and Zedboard#8482; FPGA platforms. The highest improvement came with the Inception network on an ImageNet dataset with a throughput of 22x and decrease in energy consumption of 13x compared to the ARM processor implementation. We then implement SCALENet for time series EEG seizure detection using both a Direct Convolution and FFT Convolution method to show its design versatility with a 99.7% reduction in execution time and a 97.9% improvement in energy consumption compared to the ARM. Finally, we demonstrate the ability to achieve parity with or exceed the energy efficiency of NVIDIA GPUs when evaluated against Jetson TK1 with embedded GPU System on Chip (SoC) and with a 4x power savings in a power envelope of 2.07 Watts.

Convolutional neural networks (CNNs) have gained a huge attention for real-world artificial intelligence (AI) applications such as image classification and object detection. On the other hand, for better accuracy, the size of the CNNs’ parameters (weights) has been increasing, which in turn makes it difficult to enable on-device CNN inferences in resource-constrained edge devices. Though weight pruning and 5-bit quantization methods have shown promising results, it is still challenging to deploy large CNN models in edge devices. In this paper, we propose an encoding and hardware-based decoding technique which can be applied to 5-bit quantized weight data for on-device CNN inferences in resource-constrained edge devices. Given 5-bit quantized weight data, we employ arithmetic coding with range scaling for lossless weight compression, which is performed offline. When executing on-device inferences with underlying CNN accelerators, our hardware decoder enables a fast in-situ weight decompression with small latency overhead. According to our evaluation results with five widely used CNN models, our arithmetic coding-based encoding method applied to 5-bit quantized weights shows a better compression ratio by 9.6× while also reducing the memory data transfer energy consumption by 89.2%, on average, as compared to the case of uncompressed 32-bit floating-point weights. When applying our technique to pruned weights, we obtain better compression ratios by 57.5×–112.2× while reducing energy consumption by 98.3%–99.1% as compared to the case of 32-bit floating-point weights. In addition, by pipelining the weight decoding and transfer with the CNN execution, the latency overhead of our weight decoding with 16 decoding unit (DU) hardware is only 0.16%–5.48% and 0.16%–0.91% for non-pruned and pruned weights, respectively. Moreover, our proposed technique with 4-DU decoder hardware reduces system-level energy consumption by 1.1%–9.3%.