High-Accuracy Multiply-Accumulate (MAC) Technique for Unary Stochastic Computing

Stochastic Number Generators (SNG) plays a significant role in designing a stochastic computing system. SNGs make the stochastic system comfortable for computing in the stochastic domain. The challenges in developing the stochastic computing system are correlation and hardware area occupancy. By considering these phenomena, we have considered Linear Feedback Shift Register (LFSR) based SNG and S-box based SNG in this work. Our contributions to this paper are stochastic computation for activation functions using the SNGs mentioned above and stochastic computation for arithmetic components in the stochastic domain. By considering the two SNG methods, the difference in the computation for accuracy has been analyzed for stochastic activation function and stochastic arithmetic computation. The better SNG method will be used as SNG for stochastic convolutional neural network design using this analysis.

Magnetic skyrmion, topologically non-trivial spin texture, has been considered as promising information carrier in future electronic devices because of its nanoscale size, low depinning current density and high motion velocity. Despite the broad interests in skyrmion racetrack memory, researchers have been recently exploiting logic functions enabled by using the particle-like behaviors of skyrmions. These functions can be applied to unconventional computing, such as stochastic computing (SC), which treats data as probabilities and is superior to binary computing due to its simplicity of logic operation. In this work, we demonstrate SC implemented by skyrmionic logic devices. We propose a skyrmionic AND-OR logic device as a multiplier in the stochastic domain and two skyrmionic multiplexer (MUX) logic devices as stochastic adders. With the assist of voltage controlled magnetic anisotropy (VCMA), the precise control of skyrmions collision is not required in the skyrmionic AND-OR logic device, thus improving the operation robustness. In the two MUX logic devices, skyrmions can be driven by Zhang-Li torque or spin orbit torque (SOT). Particularly, we can flexibly regulate the skyrmion motion by VCMA or voltage controlled Dzyaloshinskii-Moriya Interaction (VCDMI) in the SOT case. Furthermore, 3-bit stochastic multiplier and adder are demonstrated by micromagnetic simulations. In addition, simulations in synthetic antiferromagnets (SAF) show that the performance of our skyrmionic logic gates can be optimized through advanced materials. Our work opens up perspective to implement SC using skyrmionic logic devices.

By introducing stochastic computing (SC), the hardware consumption of Deep Neural Network (DNN) can be reduced significantly. However, long SC sequence length is needed to maintain the accuracy, which results in extended computation time. To shorten the sequence length, we propose two high-accuracy SC computation units to improve the precision of SC-DNN, which are the rematching-based correlation-independent multiplier and the accumulator-based Rectified Linear Unit. Moreover, a length-adaptive method that uses variable lengths for different images is adopted to decrease the average sequence length. Software simulation shows that the SC design has just 0.01% accuracy loss with only the sequence length of 20 for MNIST dataset comparing to the binary system. The ASIC layout results demonstrate the area efficiency is improved by 16X comparing to the latest SC-DNN with similar accuracy loss.

Recently, CMOS invertible logic has been presented and is one of the new computing paradigms based on a probabilistic device model. It is designed based on stochastic computing that provides bidirectional operations between inputs and outputs and has been applied for several critical issues, such as integer factorization and machine learning (ML). This article presents an overview of CMOS invertible logic from principle to application. First, the principle is explained with a simple design example, and a design flow is introduced, as is an automatic design tool. Second, the hardware of CMOS invertible logic is designed using stochastic computing and then evaluated in two applications implemented on a field-programmable gate array (FPGA) or application-specific integrated circuits (ASICs). Finally, this article ends with future challenges.

Deep neural networks (DNNs) have been revolutionizing machine learning research and various applications by achieving better accuracy than human perception. However, a large amount of computational resources are needed due to their complex topologies, restricting the usage of DNNs in embedded systems and portable devices with limited areas and power budgets. This chapter considers stochastic computing (SC) as an alternative to binarized computing. SC uses a random bit sequence where the signal value is computed by counting the number of ones in the bit sequence. This allows the implementation of major arithmetic units like multiplication and addition by a simple logic. Consequently, SC can implement DNNs with essentially low hardware overhead and also offers good scalability. First, the chapter presents a detailed study of processing elements used in neural networks. Then, combining several building blocks, such as stochastic approximation of activation functions in SC, saturation arithmetic, and scaling, a new scheme is proposed for DNN inference and training in SC. At last, the chapter shows that SC-compatible DNNs can be effectively deployed on low-cost hardware. Deep neural network, optimization, accelerator, custom computing, stochastic computing

Stochastic Computing (SC) is a computing paradigm that allows for the\nlow-cost and low-power computation of various arithmetic operations using\nstochastic bit streams and digital logic. In contrast to conventional\nrepresentation schemes used within the binary domain, the sequence of bit\nstreams in the stochastic domain is inconsequential, and computation is usually\nnon-deterministic. In this brief, we exploit the stochasticity during switching\nof probabilistic Conductive Bridging RAM (CBRAM) devices to efficiently\ngenerate stochastic bit streams in order to perform Deep Learning (DL)\nparameter optimization, reducing the size of Multiply and Accumulate (MAC)\nunits by 5 orders of magnitude. We demonstrate that in using a 40-nm\nComplementary Metal Oxide Semiconductor (CMOS) process our scalable\narchitecture occupies 1.55mm$^2$ and consumes approximately 167$\\mu$W when\noptimizing parameters of a Convolutional Neural Network (CNN) while it is being\ntrained for a character recognition task, observing no notable reduction in\naccuracy post-training.\n

A complex process fault diagnosis method based on manifold distribution adaptation

Stochastic computing (SC) is a low-cost design technique that has great promise in applications such as image processing. SC enables arithmetic operations to be performed on stochastic bit-streams using ultra-small and low-power circuitry. However, accurate computations tend to require long run-times due to the random fluctuations inherent in stochastic numbers (SNs). We present novel techniques for SN generation that lead to better accuracy/run-time trade-offs. First, we analyze a property called progressive precision (PP) which allows computational accuracy to grow systematically with run-time. Second, borrowing from Monte Carlo methods, we show that SC performance can be greatly improved by replacing the usual pseudo-random number sources by low-discrepancy (LD) sequences that are predictably progressive. Finally, we evaluate the use of LD stochastic numbers in SC, and show they can produce significantly faster and more accurate results than existing stochastic designs.

In the fields of machine learning and image processing, cost-less circuits with low energy are required instead of extreme precision, and stochastic computing (SC), a type of approximate computing, is attracting attention. In SC, stochastic numbers (SNs), bit streams with values of the appearance rates of 1‘s, are used. SC enables calculations with simple circuits. To make the calculation results correct, duplication of an SN (gener-ating an SN with the same value) is required when using the SN with the same value. The conventional SN duplicator composed of a flip-flop (FF) has a problem that the output SN only depends on the input SN. Therefore, if the FF-based duplicator is used in a circuit with re-convergence paths, the output SN becomes erroneous. This paper proposes an SN duplicator, 2n RRR, that can output more independent output by its improved flexibility of bit re-arrangement. With this duplicator, the errors of the hyperbolic tangent function are reduced by up to 50% compared to the duplicator that we proposed previously. Also, up to more than 99.9% of the circuit area is reduced compared to the implementation of binary computing.

All applications in current trends need to use Machine Learning and Big Data in this huge data world.Machine learning is a type of artificial intelligence that allows software applications to become more accurate at predicting outcomes without being explicitly programmed to do so.Machine learning use historical data as input to predict new output values.Small Scale and even large developers cannot maintain Machine Learning Algorithms and other big data processing techniques all by themselves.To harvest, process and analyze the data we need a data processing engine.Our project is to provide them with a machine learning engine which runs on.a specified port on their servers which collects the data sets in real time.This which can be queried using a simple Query Language and analyze the database on the query.The purpose of our project is to create a Data Processing Engine for Big Data Analytics. Machine learning and Big Data plays a major role in the current huge data world. Our project is to maintain a dedicated machine learning and data processing engine, so that even small developers without the knowledge of machine learning and big data analysis can produce expected results based on Machine Learning to make their application smooth. All applications in current trends need to use Machine Learning and Big Data in this huge data world. Machine learning is a subset of artificial intelligence that allows software applications to become more accurate at predicting outcomes without being explicitly programmed to do so. Machine learning uses historical data as input to predict new output values. Small Scale and even large developers cannot maintain Machine Learning Algorithms and other big data processing techniques all by themselves. To harvest, process and analyze the data we need a data processing engine. This project is to provide them a data processing engine which runs on a specified port on their servers which collects the data sets in real time. This can be queried using a simple Query Language and analyze the data and produce respected results using ML based on the query

Neuromorphic computing is a promising technology for future machine learning research, which is envisioned to provide lower power consumption, fault tolerance, and massive parallelism. However, the neuromorphic network, i.e., spike neural network (SNN), suffers from a broader range of its application compared with artificial neural networks (ANN). In this work, we propose a stochastic computation SC-based hybrid ANN and SNN scheme, which supports both ANN and SNN computation in SC domain. The ANN computation is converted to SC, which is implemented with simple logic gates. Moreover, we propose a new two-stage stochastic computation that is compatible with the sequence computation style of SNN. Consequently, the logic resource for the hybrid SNN and ANN can be shared. Compared to the state-of-the-art hybrid scheme, the proposed SC-based ANN and SNN exhibit much lower hardware costs and higher hardware efficiency.

SummaryMany of the modern deep learning, machine learning, and artificial intelligence algorithms use adders, multipliers, and multiply‐accumulators (MACs) with mixed precisions. In general, fixed‐point and floating‐point adders, multipliers, and MACs are used in mixed‐precision hardware, such that the above algorithm can choose appropriate hardware for its processing. This paper proposes an efficient mixed‐precision MAC circuit that can perform fixed‐point and floating‐point operations. The proposed design uses Han‐Carlson adder with late carry or end‐around carry in its accumulator design; as a result, delay and energy of the circuit reduce by and respectively, when compared with the existing designs from the literature.

Hardware accelerators are playing an increasingly important role in the field of computer science and engineering, especially in accelerating deep learning and large-scale data processing. However, the traditional hardware accelerator design often relies on manual adjustment and experience accumulation, which is inefficient. Therefore, the use of machine learning technology to optimize hardware accelerator design has become a new trend. In this paper, a hardware accelerator optimization design method based on machine learning is proposed, including hardware design, algorithm optimization, system-level optimization, and deep learning model compression. Through the research and verification of the actual case, this research proves the effectiveness and feasibility of the hardware accelerator optimization design method based on machine learning. Compared with the traditional design method, this method can find better design scheme in shorter time and has higher performance prediction accuracy. Therefore, this method can significantly improve the design efficiency and performance of hardware accelerators. The results of this study are of great significance for promoting the development of hardware accelerator technology. It can promote scientific research and engineering applications in related fields to make greater progress, and promote the development of artificial intelligence, big data processing and other fields.

As core computational primitives underlying linear operations, such as matrix-vector and matrix-matrix multiplications, in signal processing, machine learning and other areas, inner products are primarily realized as multiply-accumulate operations and dominate computational workloads. Addressing hardware bottlenecks in scaling to high dimensions, online arith-metic minimizes interconnects and memory footprint through most-significant-digit- first (MSD F) processing. Conventional cas-caded online implementations, however, incur significant latency through hierarchical accumulation of online delays and synchro-nization overheads. We propose an integrated online inner prod-uct architecture that fuses multiplication-accumulation phases via digit recurrence relations, collapsing operator boundaries to reduce the initial latency by up to 50 % at near-equivalent resource cost. Implemented in an FPGA-based matrix-vector multiplier and validated with the Google's Page Rank algorithm, our approach achieves 13.4 x speedup for 32-dimension computation versus an Intel Core i9-12900K CPU, establishing a new paradigm for high-dimension linear system computation.

Nowadays, technologies based on machine learning (ML) are becoming a popular solution for building recommendation services on consumer electronics (CE) devices. Therefore, in this paper, we introduce i.data (interleaved data) processing scheme to more efficiently enable ML models on CE devices. The key idea of i.data is to interleave the training and inference steps in a parallel way so as to improve the GPU utilization and overall performance. We also compared i.data with Numpy and tf.data that are widely used to build ML models in various environments including CE and the internet of things (IoT). Our experimental results clearly show that i.data improves the performance compared with the traditional data processing approaches.