Spiking neural networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANNs): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate BP using spike-time dependent plasticity with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based back-prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. The composite neuron encodes information in the form of stochastic spike-trains and converts BP weight updates into temporally and spatially local spike coincidence updates compatible with hardware-friendly resistive processing units. Furthermore, we characterize the quantization effect of discrete spike-based weight update to show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a winner take all rule. Our SNN with a two-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.

Magnetic textures are promising candidates for unconventional computing due to their non-linear dynamics. We propose to investigate the rich variety of seemingly trivial lamellar magnetic phases, e.g. helical, spiral, stripy phase, or other one-dimensional soliton lattices. These are the natural stray field-free ground states of almost every magnet. The order parameters of these phases may be of potential interest for both classical and unconventional computing, which we refer to as helitronics. For the particular case of a chiral magnet and its helical phase, we use micromagnetic simulations to demonstrate the working principles of all-electrical (i) classical binary memory cells and (ii) memristors and artificial synapses, based on the orientation of the helical stripes.

Abstract Biological intelligence imparts organisms with the ability to overcome a number of key challenges such as adapting to dynamic environments, learning from experience, and making complex decisions, even within a daunting set of constraints (e.g. extremely limited energy). Interestingly, we are encountering several analogous challenges and constraints as artificial intelligence (AI) begins to move from the cloud to the edge in the ever-growing internet-of-things (IoT). Neuromorphic computing is poised to play a critical role in moving AI to the edge, as it enables the implementation of state-of-the-art machine learning algorithms (e.g. deep neural networks) on hardware platforms with limited resources (energy, precision, I/O, etc.). This NCE focus issue on Extreme Edge Computing brings together a variety of works that are aimed at designing neuromorphic computing for AI at-the-edge. The collection includes four original research articles and one topical review paper, which are briefly summarized below

The impressive performance of artificial neural networks has come at the cost of high energy usage and CO2 emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally detrimental stochastic effects in magnetic domain-wall motion in nanowires. We demonstrate functional binary stochastic synapses alongside a gradient learning rule that allows their training with applicability to a range of stochastic systems. The rule, utilising the mean and variance of the neuronal output distribution, finds a trade-off between synaptic stochasticity and energy efficiency depending on the number of measurements of each synapse. For single measurements, the rule results in binary synapses with minimal stochasticity, sacrificing potential performance for robustness. For multiple measurements, synaptic distributions are broad, approximating better-performing continuous synapses. This observation allows us to choose design principles depending on the desired performance and the device’s operational speed and energy cost. We verify performance on physical hardware, showing it is comparable to a standard neural network.

The computation of the brain relies on the highly efficient communication among billions of neurons. Such efficiency derives from the brain’s plastic and reconfigurable nature, enabling complex computations and maintenance of vital functions with a remarkably low power consumption of only ∼20 W. First efforts to leverage brain-inspired computational principles have led to the introduction of artificial neural networks that revolutionized information processing and daily life. The relentless pursuit of the definitive computing platform is now pushing researchers towards investigation of novel solutions to emulate specific brain features (such as synaptic plasticity) to allow local and energy efficient computations. The development of such devices may also be pivotal in addressing major challenges of a continuously aging world, including the treatment of neurodegenerative diseases. To date, the neuroelectronics field has been instrumental in deepening the understanding of how neurons communicate, owing to the rapid development of silicon-based platforms for neural recordings and stimulation. However, this approach still does not allow for in loco processing of biological signals. In fact, despite the success of silicon-based devices in electronic applications, they are ill-suited for directly interfacing with biological tissue. A cornucopia of solutions has therefore been proposed in the last years to obtain neuromorphic materials to create effective biointerfaces and enable reliable bidirectional communication with neurons. Organic conductive materials in particular are not only highly biocompatible and able to electrochemically transduce biological signals, but also promise to include neuromorphic features, such as neuro-transmitter mediated plasticity and learning capabilities. Furthermore, organic electronics, relying on mixed electronic/ionic conduction mechanism, can be efficiently coupled with biological neural networks, while still successfully communicating with silicon-based electronics. Here, we envision neurohybrid systems that integrate silicon-based and organic electronics-based neuromorphic technologies to create active artificial interfaces with biological tissues. We believe that this approach may pave the way towards the development of a functional bidirectional communication between biological and artificial ‘brains’, offering new potential therapeutic applications and allowing for novel approaches in prosthetics.

The growing number of edge devices in everyday life generates a considerable amount of data that current AI algorithms, like artificial neural networks, cannot handle inside edge devices with limited bandwidth, memory, and energy available. Neuromorphic computing, with low-power oscillatory neural networks (ONNs), is an alternative and attractive solution to solve complex problems at the edge. However, ONN is currently limited with its fully-connected recurrent architecture to solve auto-associative memory problems. In this work, we use an alternative two-layer bidirectional ONN architecture. We introduce a two-layer feedforward ONN architecture to perform image edge detection, using the ONN to replace convolutional filters to scan the image. Using an HNN Matlab emulator and digital ONN design simulations, we report efficient image edge detection from both architectures using various size filters (3 × 3, 5 × 5, and 7 × 7) on black and white images. In contrast, the feedforward architectures can also perform image edge detection on gray scale images. With the digital ONN design, we also assess latency performances and obtain that the bidirectional architecture with a 3 × 3 filter size can perform image edge detection in real-time (camera flow from 25 to 30 images per second) on images with up to 128 × 128 pixels while the feedforward architecture with same 3 × 3 filter size can deal with 170 × 170 pixels, due to its faster computation.

Smart electronic circuits that support neuromorphic computing on the hardware level necessitate materials with memristive, memcapacitive, and neuromorphic- like functional properties; in short, the electronic response must depend on the voltage history, thus enabling learning algorithms. Here we demonstrate volatile ferroelectric switching of Sn2P2S6 at room temperature and see that initial polarization orientation strongly determines the properties of polarization switching. In particular, polarization switching hysteresis is strongly imprinted by the original polarization state, shifting the regions of non-linearity toward zero-bias. As a corollary, polarization switching also enables effective capacitive switching, approaching the sought-after regime of memcapacitance. Landau–Ginzburg–Devonshire simulations demonstrate that one mechanism by which polarization can control the shape of the hysteresis loop is the existence of charged domain walls (DWs) decorating the periphery of the repolarization nucleus. These walls oppose the growth of the switched domain and favor back-switching, thus creating a scenario of controlled volatile ferroelectric switching. Although the measurements were carried out with single crystals, prospectively volatile polarization switching can be tuned by tailoring sample thickness, DW mobility and electric fields, paving way to non-linear dielectric properties for smart electronic circuits.

Hebbian plasticity in winner-take-all (WTA) networks is highly attractive for neuromorphic on-chip learning, owing to its efficient, local, unsupervised, and on-line nature. Moreover, its biological plausibility may help overcome important limitations of artificial algorithms, such as their susceptibility to adversarial attacks, and their high demands for training-example quantity and repetition. However, Hebbian WTA learning has found little use in machine learning, likely because it has been missing an optimization theory compatible with deep learning (DL). Here we show rigorously that WTA networks constructed by standard DL elements, combined with a Hebbian-like plasticity that we derive, maintain a Bayesian generative model of the data. Importantly, without any supervision, our algorithm, SoftHebb, minimizes cross-entropy, i.e. a common loss function in supervised DL. We show this theoretically and in practice. The key is a ‘soft’ WTA where there is no absolute ‘hard’ winner neuron. Strikingly, in shallow-network comparisons with backpropagation, SoftHebb shows advantages beyond its Hebbian efficiency. Namely, it converges in fewer iterations, and is significantly more robust to noise and adversarial attacks. Notably, attacks that maximally confuse SoftHebb are also confusing to the human eye, potentially linking human perceptual robustness, with Hebbian WTA circuits of cortex. Finally, SoftHebb can generate synthetic objects as interpolations of real object classes. All in all, Hebbian efficiency, theoretical underpinning, cross-entropy-minimization, and surprising empirical advantages, suggest that SoftHebb may inspire highly neuromorphic and radically different, but practical and advantageous learning algorithms and hardware accelerators.

In principle, sparse neural networks should be significantly more efficient than traditional dense networks. Neurons in the brain exhibit two types of sparsity; they are sparsely interconnected and sparsely active. These two types of sparsity, called weight sparsity and activation sparsity, when combined, offer the potential to reduce the computational cost of neural networks by two orders of magnitude. Despite this potential, today’s neural networks deliver only modest performance benefits using just weight sparsity, because traditional computing hardware cannot efficiently process sparse networks. In this article we introduce Complementary Sparsity, a novel technique that significantly improves the performance of dual sparse networks on existing hardware. We demonstrate that we can achieve high performance running weight-sparse networks, and we can multiply those speedups by incorporating activation sparsity. Using Complementary Sparsity, we show up to 100× improvement in throughput and energy efficiency performing inference on FPGAs. We analyze scalability and resource tradeoffs for a variety of kernels typical of commercial convolutional networks such as ResNet-50 and MobileNetV2. Our results with Complementary Sparsity suggest that weight plus activation sparsity can be a potent combination for efficiently scaling future AI models.

Computation-in-memory using memristive devices is a promising approach to overcome the performance limitations of conventional computing architectures introduced by the von Neumann bottleneck which are also known as memory wall and power wall. It has been shown that accelerators based on memristive devices can deliver higher energy efficiencies and data throughputs when compared with conventional architectures. In the vast multitude of memristive devices, bipolar resistive switches based on the valence change mechanism (VCM) are particularly interesting due to their low power operation, non-volatility, high integration density and their CMOS compatibility. While a wide range of possible applications is considered, many of them such as artificial neural networks heavily rely on vector-matrix-multiplications (VMMs) as a mathematical operation. These VMMs are made up of large numbers of multiplication and accumulation (MAC) operations. The MAC operation can be realised using memristive devices in an analog fashion using Ohm’s law and Kirchhoff’s law. However, VCM devices exhibit a range of non-idealities, affecting the VMM performance, which in turn impacts the overall accuracy of the application. Those non-idealities can be classified into time-independent (programming variability) and time-dependent (read disturb and read noise). Additionally, peripheral circuits such as analog to digital converters can introduce errors during the digitalization. In this work, we experimentally and theoretically investigate the impact of device- and circuit-level effects on the VMM in a VCM crossbars. Our analysis shows that the variability of the low resistive state plays a key role and that reading in the RESET direction should be favored to reading in the SET direction.