Computational Analysis of stochastic arithmetic computing and stochastic activation function

Stochastic number generator (SNG) is one important component of stochastic computing (SC). An SNG usually consists of a random number source (RNS) and a probability conversion circuit (PCC). The SNGs occupy a large portion of the total area and power of a stochastic circuit. Thus, it is critical to lower the area and power of the SNGs. The existing methods only focused on simplifying the RNSs inside the SNGs, such as sharing the RNSs and using emerging devices. However, how to reduce the area and power of PCCs is never studied. In this work, we explore this problem and propose a solution that can effectively reduce the area and power of PCCs. We also study the theoretical limit on the cost of SNG and find that our proposed design approaches the limit. The experimental results show that our design can gain up to 2× improvement in power-delay product over the traditional SNGs.

Stochastic Computing (SC) is an alternative way of computing with binary weighted words that can significantly reduce hardware resources. This technique relies on transforming information from a conventional binary system to the probability domain in order to perform mathematical operations based on probability theory, where smaller amounts of binary logic elements are required. Despite the advantage of computing with reduced circuitry, SC has a well known issue; the input interface known as stochastic number generator (SNG), is a hardware consuming module, which is disadvantageous for small digital circuits or circuits with several input data. Hence, in this work, efforts are dedicated to improving a classic weighted binary SNG (WBSNG). For this, one of the internal modules (weight generator) of the SNG was redesigned by detecting a pattern in the involved signals that helped to pose the problem in a different way, yielding equivalent results. This greatly reduced the number of logical elements used in its implementation. This pattern is interpreted with Boolean equations and transferred to a digital circuit that achieves the same behavior of a WBSNG but with less resources.

Stochastic neuromorphic computation (SNC) has the potential to enable a low power, error tolerant and scalable computing platform in comparison to its deterministic counterparts. However, the hardware implementation of complementary metal oxide semiconductor (CMOS)-based stochastic circuits involves conversion blocks that cost more than the actual processing circuits. The realization of the activation function for SNCs also requires a complicated circuit that results in a significant amount of power dissipation and area overhead. The inherent probabilistic switching behavior of nanomagnets provides an advantage to overcome these complexity issues for the realization of low power and area efficient SNC systems. This paper presents magnetic tunnel junction (MTJ)-based stochastic computing methodology for the implementation of a neural network. The stochastic switching behavior of the MTJ has been exploited to design a binary to stochastic converter to mitigate the complexity of the CMOS-based design. The paper also presents the technique for realizing stochastic sigmoid activation function using an MTJ. Such circuits are simpler than existing ones and use considerably less power. An image classification system employing the proposed circuits has been implemented to verify the effectiveness of the technique. The MTJ-based SNC system shows area and energy reduction by a factor of 13.5 and 2.5, respectively, while the prediction accuracy is 86.66%. Furthermore, this paper investigates how crucial parameters, such as stochastic bitstream length, number of hidden layers and number of nodes in a hidden layer, need to be set precisely to realize an efficient MTJ-based stochastic neural network (SNN). The proposed methodology can prove a promising alternative for highly efficient digital stochastic computing applications.

In this study, we propose a novel stochastic number generator architecture and prove that the resulting circuit can deliver independent stochastic numbers and improve the accuracy of the calculation results obtained using some recent conventional stochastic computing-based arithmetic circuits. This study is motivated by the increasingly important role of stochastic computing in various fields, such as the digital circuit design, where the stochastic number generators are responsible for a significant share of the hardware cost.

Stochastic computing (SC) is an attractive computing paradigm because of its fault tolerance and hardware efficiency. Unfortunately, existing SC systems suffer from large latency and inaccuracy problems. In this paper, we propose a new parallel stochastic computing (PSC) system which has higher computing accuracy and faster processing speed compared to existing SC systems. It uses the nibble serial data organization to reduce the latency and a combination of memory based weighted binary generator and data shuffling to improve the accuracy. Simulations on finite impulse response (FIR) filters show that the proposed system with 4 nibbles achieves 35% improvement in accuracy with about 3.5 times increase in processing speed compared to traditional SC. Like SC systems, PSC is also more tolerant of soft errors compared to conventional 2's complement implementations. For a 4-tap FIR filter a 10% injected error causes the root-mean-square-error (RMSE) of PSC to be 0.053 compared to RMSE of 0.188 for 2's complement implementation.

We consider the design of stochastic computing (SC) hardware based on spintronic devices. SC offers low-cost implementations of arithmetic operations and high degrees of error tolerance. When compared to charge-based devices, spin-based devices could be lower energy, nonvolatile, etc. However, spin-based devices can be fundamentally more error prone than charge-based devices. The marriage of SC architectures and spin-based devices has the potential to produce information processing systems that are robust, low energy, and nonvolatile. In this paper, we investigate SC hardware comprised of nanomagnetic logic (NML) devices. NML is a “beyond-CMOS” technology that uses bistable magnets to store, process, and move binary information. We introduce new NML circuit structures that exploit unique features of the technology to efficiently realize hardware components required for an SC system (e.g., random number generation). We also benchmark NML-based SC circuits against other implementations, and illustrate how features that are unique to NML (e.g., inherent pipelining) can lead to improved performance. Our results indicate NML SC implementations achieve smaller area footprints with reduced energy and delay when compared to CMOS equivalents.

Stochastic computing (SC) is an alternative computing paradigm that provides low-cost hardware implementation for complex operations. Many previous works adopted SC in several applications such as signal processing and pattern recognition, where these types of application possess a nature of successive computations. Successive SC computation affects correlation, which affects accuracy. This relationship is not studied well before. In this paper, we characterize correlation properties for SC functions to thoroughly understand the computation-correlation-accuracy relationship. We provide a measure ( δSCC) to identify the significance of variation of correlation due to computation. We also offer a general method to determine the correlation characteristics of any designed SC function.

Transient bit flips may cause unexpected behaviors in digital arithmetic circuits, and thus computation errors. It is generally believed that stochastic computing (SC) can tolerate bit flips due to its unique number representation format. In SC, a value is encoded by a random bit stream using the probability of 1s in the bit stream. Therefore, one bit flip changes the probability by a small number given a long enough bit stream, and the computation results might not change much. This is verified in previous works mostly through empirical studies. However, we found that bit flips actually change the probability of a stochastic bit stream by a predictable value. This is formulated by considering the bit flips as Bernoulli events. The distorted probability of the bit stream considering bit flips is derived and the SC bit flip models for various stochastic circuits are proposed. The error recovery based on the formulation is also proposed. The SC bit flip model and error recovery scheme are verified on basic SC elements such as the SC multipliers and an FSM-based stochastic circuit. They are then applied to a complex SC system computing a neural network. The results show that the accuracy of MNIST digit recognition can be recovered even with a high bit flip rate of 10%.

The stochastic computing (SC) has been proven to be an energy-efficient way to perform neural network. In this study, we propose a field-free voltage-controlled spintronics SC system based on the magnetic tunnel junction (MTJ). We observe a stochastic switching behavior of the MTJ and that it could be controlled by the voltage applied on the device. The voltage-controlled stochastic switching behavior is used to encode numbers ranging from 0 to 1 into a series of random bit-streams in the SC system. Furthermore, the handwritten recognition task is performed on the MTJ-based SC system, achieving a 95% maximum accuracy, which is comparable with the floating-point based neural network. Our work provides inspiration for the energy-efficient neural network systems.

Magnetic skyrmion, topologically non-trivial spin texture, has been considered as promising information carrier in future electronic devices because of its nanoscale size, low depinning current density and high motion velocity. Despite the broad interests in skyrmion racetrack memory, researchers have been recently exploiting logic functions enabled by using the particle-like behaviors of skyrmions. These functions can be applied to unconventional computing, such as stochastic computing (SC), which treats data as probabilities and is superior to binary computing due to its simplicity of logic operation. In this work, we demonstrate SC implemented by skyrmionic logic devices. We propose a skyrmionic AND-OR logic device as a multiplier in the stochastic domain and two skyrmionic multiplexer (MUX) logic devices as stochastic adders. With the assist of voltage controlled magnetic anisotropy (VCMA), the precise control of skyrmions collision is not required in the skyrmionic AND-OR logic device, thus improving the operation robustness. In the two MUX logic devices, skyrmions can be driven by Zhang-Li torque or spin orbit torque (SOT). Particularly, we can flexibly regulate the skyrmion motion by VCMA or voltage controlled Dzyaloshinskii-Moriya Interaction (VCDMI) in the SOT case. Furthermore, 3-bit stochastic multiplier and adder are demonstrated by micromagnetic simulations. In addition, simulations in synthetic antiferromagnets (SAF) show that the performance of our skyrmionic logic gates can be optimized through advanced materials. Our work opens up perspective to implement SC using skyrmionic logic devices.

Multiply-accumulate (MAC) operations are common in data processing and machine learning but costly in terms of hardware usage. Stochastic Computing (SC) is a promising approach for low-cost hardware design of complex arithmetic operations such as multiplication. Computing with deterministic unary bit-streams (defined as bit-streams with all 1s grouped together at the beginning or end of a bit-stream) has been recently suggested to improve the accuracy of SC. Conventionally, SC designs use multiplexer (MUX) units or OR gates to accumulate data in the stochastic domain. MUX-based addition suffers from scaling of data and OR-based addition from inaccuracy. This work proposes a novel technique for MAC operation on unary bit-streams that allows exact, non-scaled addition of multiplication results. By introducing a relative delay between the products, we control correlation between bit-streams and eliminate OR-based addition error. We evaluate the accuracy of the proposed technique compared to the state-of-the-art MAC designs. After quantization, the proposed technique demonstrates at least 37% and up to 100% decrease of the mean absolute error for uniformly distributed random input values, compared to traditional OR-based MAC designs. Further, we demonstrate that the proposed technique is practical and evaluate area, power and energy of three possible implementations.

This chapter begins by reviewing the sources of inaccuracy in stochastic computing, focusing on correlation, that is, dependencies among stochastic bit-streams. The measurement of correlation is considered, and the SCC metric is defined. The properties of correlation are then explored including some that have only been discovered recently. Correlation can be seen in two ways: either as corrupting a function f, or as changing f to a different, but potentially useful one. Therefore, to ensure that a stochastic circuit works as expected it is important to manage correlation appropriately. This can be done with correlation-controlling units, which must be used carefully to avoid unexpected functional changes and excessive hardware area or latency overhead. There are also cases where correlation has no effect at all (correlation insensitivity). Identifying such immunity to correlation can aid the design of stochastic circuits. Finally, design of stochastic number generators to provide specified levels of correlation is discussed.

Stochastic computing (SC) is a method of expressing a numerical value by the appearance ratio of 1 in a bitstream to perform an operation. Rapid single flux quantum (RSFQ) circuits can handle SC naturally and operate at ultra-high speed. RSFQ SC circuits are suitable for applications tolerating error. This paper presents a design method of RSFQ SC circuits to calculate polynomials. Using the proposed design method, an internal circuit is designed to feed bitstreams of coefficients from bitstreams of constant to reduce large stochastic number generators. The method inserts storage elements to reduce correlations of signals, and thereby reduce the calculation error of the circuit. Additionally, it minimizes the number of inserted D flip-flops utilizing the latching function of gate inputs of RSFQ logic gates.

The demand for cost-effective digital filters in near-sensor computing has aroused interest in stochastic computing (SC) as a promising alternative to traditional binary approaches. This study investigates in detail the error tolerance and hardware cost of SC-based digital filters of both the finite and infinite impulse response (FIR and IIR) types, focusing on their stochastic number generation and addition methods. We demonstrate that SC filters need only about half the area and power of binary designs under error-free conditions, though with a slight reduction of accuracy. Furthermore, these filters exhibit strong error resilience, with only 4X and 6X performance degradation under a 3% bit-flip rate, significantly outperforming their binary (non-SC) counterparts. We also compare the error tolerance of SC filters against binary filters equipped with triple modular redundancy (TMR). Our results indicate that for FIR filters, SC-based designs remain far more error-tolerant than binary FIR filters with TMR. However, in the IIR case, the binary designs with TMR outperform SC-based IIR filters at higher error rates. These findings highlight the possible trade-offs between SC and binary implementations, with SC offering more robust error tolerance, significant area and power savings and manageable accuracy trade-offs in most cases. This work also underscores SC’s potential as an efficient, resilient approach for low-cost digital filtering in edge devices and near-sensor computing applications.

Both binarized and stochastic computing-based neural networks exploit bit-wise operations to replace expensive full-precision multiplications with simple XNOR gates and thus, offer low-cost hardware implementation. In stochastic computing, arithmetic computations are performed on sequences of random bits which can approximate any real values. Stochastic computing-based neural networks benefit from approximate computing and promote fault-tolerant architectures against soft errors in noisy environments. On the other hand, in binarized neural networks, real values are deterministically binarized using the sign function. As a result, any bit-flip in the binarized values dramatically changes the outcome of arithmetic computations and makes binarized neural networks more vulnerable against soft errors. In this paper, we compare these two neural networks against each other in terms of fault-tolerance and hardware complexity (i.e., area and energy efficiency).