A system design perspective on neuromorphic computer processors

Neuromorphic computing provides a promising solution to the von Neumann bottleneck and has received a lot of research attention. However, due to the limitation of static threshold activation of traditional neuromorphic devices, it is difficult to simulate the dynamic sparsity characteristics of biological neuron, resulting in more than 90% computational redundancy in fully connected neural networks architectures based on classical devices, which has become a key issue for efficient neuromorphic computing. Here, a dynamically adaptive activation neuron‐transistor based on asymmetric electrodes and indium gallium zinc oxide thin films is proposed, overcoming the static activation limitations of conventional neuromorphic devices. The device achieves a dynamic adaptive activation similar to biometric neuron via gate voltage or UV irradiation, achieving a wide range of activation times (65 ms–13.5 s) and adjustable activation thresholds (2.5–7.7 V). Leveraging this device, a dynamic sparse spiking neural network (DS‐SNN) is constructed that enables in situ Hadamard‐based weight pruning/regeneration. Applied to autonomous driving object detection, the DS‐SNN achieves 85% accuracy with 42% sparsity, outperforming dense convolutional/spiking neural networks (320k /140k weights) while utilizing only ≈80k parameters. This hardware‐algorithm co‐design establishes a new paradigm for energy‐efficient edge‐computing electronics, exploring 3D integration of large‐scale neuromorphic processors.

A parallel approach to accelerate neural network hyperparameter selection for energy forecasting

Training deep learning networks is a difficult task due to computational complexity, and this is traditionally handled by simplifying network topology to enable parallel computation on graphical processing units (GPUs). However, the emergence of quantum devices allows reconsideration of complex topologies. We illustrate a particular network topology that can be trained to classify MNIST data (an image dataset of handwritten digits) and neutrino detection data using a restricted form of adiabatic quantum computation known as quantum annealing performed by a D-Wave processor. We provide a brief description of the hardware and how it solves Ising models, how we translate our data into the corresponding Ising models, and how we use available expanded topology options to explore potential performance improvements. Although we focus on the application of quantum annealing in this article, the work discussed here is just one of three approaches we explored as part of a larger project that considers alternative means for training deep learning networks. The other approaches involve using a high performance computing (HPC) environment to automatically find network topologies with good performance and using neuromorphic computing to find a low-power solution for training deep learning networks. Our results show that our quantum approach can find good network parameters in a reasonable time despite increased network topology complexity; that HPC can find good parameters for traditional, simplified network topologies; and that neuromorphic computers can use low power memristive hardware to represent complex topologies and parameters derived from other architecture choices.

The use of Graphics Processing Unit (GPU) for computational work has revolutionized how complex electromagnetic problems are solved. Complex problems which required supercomputers in the past for analysis can now be tackled and solved using personal computers by channeling the computational work towards GPUs instead of the traditional computer Central Processing Unit (CPU). Finite-Difference Time-Domain (FDTD) analysis, which is a computationally expensive method of solving electromagnetic problems is highly parallel in nature and can be readily executed in a GPU. We have developed an algorithm for three dimensional FDTD analysis of optical devices with micro and nano-structures using Compute Unified Device Architecture (CUDA). The developed algorithm exploits the benefits of multiple cores of GPU chips and boosts the speed of simulation without sacrificing its accuracy. We achieved a 25-fold speed up of simulation using CUDA compared to MATLAB code in CPU.

The process of using electronic circuits to replicate the neurobiological architectures seen in the nervous system is known as neuromorphic engineering, also referred to as neuromorphic computing. These technologies are essential for the future of computing, although most of the work in neuromorphic computing has been focused on hardware development. The execution speed, energy efficiency, accessibility and robustness against local failures are vital advantages of neuromorphic computing over conventional methods. Spiking neural networks are generated using neuromorphic computing. This chapter covers the basic ideas of neuromorphic engineering, neuromorphic computing, and its motivating factors and challenges. Deep learning models are frequently referred to as deep neural networks because deep learning techniques use neural network topologies. Deep learning techniques and their different architectures were also covered in this section. Furthermore, Emerging memory Devices for neuromorphic systems and neuromorphic circuits were illustrated.

In this paper, we use graphics processing units (GPU) to accelerate sparse and arbitrary structured neural networks. Sparse networks have nodes in the network that are not fully connected with nodes in preceding and following layers, and arbitrary structure neural networks have different number of nodes in each layers. Sparse Neural networks with arbitrary structures are generally created in the processes like neural network pruning and evolutionary machine learning strategies. We show that we can gain significant speedup for full activation of such neural networks using graphical processing units. We do a prepossessing step to determine dependency groups for all the nodes in a network, and use that information to guide the progression of activation in the neural network. Then we compute activation for each nodes in its own separate thread in the GPU, which allows for massive parallelization. We use CUDA framework to implement our approach and compare the results of sequential and GPU implementations. Our results show that the activation of sparse neural networks lends very well to GPU acceleration and can help speed up machine learning strategies which generate such networks or other processes that have similar structure.

Neuromorphic computing, where memory and computation occur in the same physical system, promises to overcome the von Neumann bottleneck by mimicking the massively parallel architecture of biological neural networks. Self-formed...

Brain-inspired neuromorphic computing has been extensively researched, taking advantage of increased computer power, the acquisition of massive data, and algorithm optimization. Neuromorphic computing requires mimicking synaptic plasticity and enables near-in-sensor computing. In synaptic transistors, how to elaborate and examine the link between microstructure and characteristics is a major difficulty. Due to the absence of interlayer shielding effects, defect-free interfaces, and wide spectrum responses, reducing the thickness of organic crystals to the 2D limit has a lot of application possibilities in this computing paradigm. This paper presents an update on the progress of 2D organic crystal-based transistors for data storage and neuromorphic computing. The promises and synthesis methodologies of 2D organic crystals (2D OCs) are summarized. Following that, applications of 2D OCs for ferroelectric non-volatile memory, circuit-type optoelectronic synapses, and neuromorphic computing are addressed. Finally, new insights and challenges for the field’s future prospects are presented, pushing the boundaries of neuromorphic computing even farther.

Neuromorphic computing offers energy‐efficient computations for large‐scale data processing compared to conventional von Neumann computing. The artificial synapse, a key element for learning and memory operations in neuromorphic computing, requires multi‐state characteristics and the capability to change and store its states. The implementation of hardware‐based artificial synapses using nonvolatile memory provides significant advantages in terms of energy consumption and circuit area compared to their currently employed CMOS‐based counterparts. In this regard, spintronic devices have emerged as a promising candidate due to their desirable properties for artificial synapses, including multilevel formability, non‐volatility, and outstanding writing performance. In this study, spintronic artificial synapses utilizing voltage‐controlled multilevel magnetic states and energy‐ and area‐efficient artificial neural network architectures associated with them are demonstrated. The multilevel states are created by gradually modulating the magnetic easy‐axis orientation from perpendicular to in‐plane and vice versa, which is achieved either by sequentially applying gate voltage pulses or by adjusting the pulse width of the gate voltage. Based on these spintronic artificial synapses, convolutional neural network (CNN) and spiking neural network (SNN) architectures are constructed, demonstrating high recognition accuracy for the MNIST dataset with improved energy efficiency and a reduced circuit area.

Efficient visual synthesis has played an indirect yet important role in the development of neural networks. Graphics processing units (GPUs) were initially designed for visual synthesis tasks such as rendering graphics. However, their parallel computing capabilities later made GPUs the ideal hardware for training neural networks. Since neural networks have greatly benefited from GPUs, a compelling question arises: How can neural networks, in turn, improve efficient visual synthesis? Existing visual synthesis methods still require improvement for higher efficiency. For example, Monte Carlo rendering takes over 15 minutes to generate a megapixel frame at a high-quality setting on a GPU, as computing thousands of samples per pixel is time-consuming yet essential for an offline path tracer. This thesis presents three technical contributions based on neural networks for efficient visual synthesis. We explore the performance improvement that neural networks can bring to different applications of visual synthesis, including graphics, 4D Gaussians, and videos. First, we propose speeding up Monte Carlo rendering by significantly reducing the number of pixels required for sampling. We introduce a neural network to predict future frames instead of rendering future frames. For each predicted future frame, certain pixels may not be accurately projected from existing frames due to challenging conditions, such as quick motion and large occlusion. Therefore, our method estimates a mask together with each subsequent frame. The estimated mask indicates the particular pixels requiring ray sampling to correct prediction results. Second, we introduce a feature splatting method based on 4D Gaussians, named Spacetime Gaussians. It is composed of three important components for dynamic view synthesis in a volumetric rendering way. The proposed Spacetime Gaussian is formulated by adapting 3D Gaussians with temporal opacity and parametric motion/rotation. This simple yet clean formulation enhances Spacetime Gaussian to capture static, dynamic, as well as transient content within a scene. Besides a new representation, we develop splatted feature rendering by replacing spherical harmonics with neural features. The neural features improve the rendering of appearance, which varies with viewpoint and time, while keeping the model compact. Additionally, with the guidance of training error and coarse depth, we sample new Gaussians in areas where current pipelines struggle to converge. Last, we apply neural visual synthesis to the general video prediction task. To produce high-quality results, we explore a trending architecture, Vision Transformers (ViTs). ViTs have made great progress in many low-level computer vision tasks like frame interpolation. However, video prediction is more

The data processing efficiency of traditional computers is suffering from the intrinsic limitation of physically separated processing and memory units. Logic-in-memory and brain-inspired neuromorphic computing are promising in-memory computing paradigms for improving the computing efficiency and avoiding high power consumption caused by extra data movement. However, memristors that can conduct digital memcomputing and neuromorphic computing simultaneously are limited by the difference in the information form between digital data and analogue data. In order to solve this problem, this paper proposes a flexible low-dimensional memristor based on boron nitride (BN), which has ultralow-power non-volatile memory characteristic, reliable digital memcomputing capabilities, and integrated ultrafast neuromorphic computing capabilities in a single in situ computing system. The logic-in-memory basis, including FALSE, material implication (IMP), and NAND, are implemented successfully. The power consumption of the proposed memristor per synaptic event (198 fJ) can be as low as biology (fJ level) and the response time (1 μs) of the neuromorphic computing is four orders of magnitude shorter than that of the human brain (10 ms), paving the way for wearable ultrahigh efficient next-generation in-memory computing architectures.

To understand the mechanism of information processing by a biological neural network, computer simulation of a large-scale spiking neural network is an important method. However, because of a high computation cost of the simulation of a large-scale spiking neural network, the simulation requires high performance computing implemented by a supercomputer or a computer cluster. Recently, hardware for parallel computing such as a multi-core CPU and a graphics card with a graphics processing unit (GPU) is built in a gaming computer and a workstation. Thus, parallel computing using this hardware is becoming widespread, allowing us to obtain powerful computing power for simulation of a large-scale spiking neural network. However, it is not clear how much increased performance the parallel computing method using a new GPU yields in the simulation of a large-scale spiking neural network. In this study, we compared computation time between the computing methods with CPUs and GPUs in a simulation of neuronal models. We developed computer programs of neuronal simulations for the computing systems that consist of a gaming graphics card with new architecture (the NVIDIA GTX 1080) and an accelerator board using a GPU (the NVIDIA Tesla K20C). Our results show that the computing systems can perform a simulation of a large number of neurons faster than CPU-based systems. Furthermore, we investigated the accuracy of a simulation using single precision floating point. We show that the simulation results of single precision were accurate enough compared with those of double precision, but chaotic neuronal response calculated by a GPU using single precision is prominently different from that calculated by a CPU using double precision. Furthermore, the difference in chaotic dynamics appeared even if we used double precision of a GPU. In conclusion, the GPU-based computing system exhibits a higher computing performance than the CPU-based system, even if the GPU system includes data transfer from a graphics card to host memory.

In deep learning, one of the popular subclasses of multilayer perceptron is the Convolutional Neural Networks (CNNs), optimized for image and semantic segmentation tasks. With a large number of floating-point operations and relatively less data transfer in the training phase, CNNs are well suited to be handled by parallel architectures. The high accuracy of CNNs comes at the cost of consequential compute and memory demands. In this article, we present a comparative analysis of a pre-trained CNN framework for recognizing a handwritten digit on different processing platforms. The performance of the neural network as the function of the parallel processing platforms offers parametric insights and factors that influence the inference on state-of-the-art Graphics Processing Units (GPUs) and systolic arrays such as Eyeriss and Tensor Processing Unit (TPU). Through inference time analysis, we observed that the systolic arrays can outperform upto 58.7 times better than a Turing architecture powered GPU. We show that while the efficiency with which the available resources are utilized is higher in the existing GPUs (upto 32%) than the TPUs, the efficiency of application-specific systolic arrays can be on par with that of GPUs. We present the results obtained from three different customized systolic array-based platforms which can be adopted by the designers to decide the hardware optimization goal.

Introduction to hardware accelerator systems for artificial intelligence and machine learning

With the advent of Big Data, traditional digital computing is struggling to cope with intricate tasks related to data classification or pattern recognition. To mitigate this limitation, software-based neural networks are implemented, but they are run in conventional computers whose operation principle (with separate memory and data-processing units) is highly inefficient compared to the human brain. Brain-inspired in-memory computing is achieved through a wide variety of methods, for example, artificial synapses, spiking neural networks, or reservoir computing. However, most of these methods use materials (e.g., memristor arrays, spintronics, phase change memories) operated with electric currents, resulting in significant Joule heating effect. Tuning magnetic properties by voltage-driven ion motion (i.e., magnetoionics) has recently emerged as an alternative energy-efficient approach to emulate functionalities of biological synapses: potentiation/depression, multilevel storage, or transitions from short-term to long-term plasticity. In this perspective, the use of magnetoionics in neuromorphic applications is critically reviewed, with emphasis on modulating synaptic weight through: 1) control of magnetization by voltage-induced ion retrieval/insertion; and 2) control of magnetic stripe domains and skyrmions in gated magnetic thin films adjacent to solid-state ionic supercapacitors. The potential prospects in this emerging research area together with a forward-looking discussion on future opportunities are provided.