Deep photonic reservoir computing based on a distributed feedback laser array

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention since many applications require high-speed operations. However, numerous processing elements and complex interconnections are usually required, leading to a large area occupation and a high power consumption. Stochastic computing has shown promising results for area-efficient hardware implementations, even though existing stochastic algorithms require long streams that exhibit long latency. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture uses integer stochastic streams and a modified Finite State Machine-based tanh function to improve the performance and reduce the latency compared to existing stochastic architectures for DNN. The simulation results show the negligible performance loss of the proposed integer stochastic DNN for different network sizes compared to their floating point versions.

Hardware implementation of artificial neural networks (ANNs) allows exploiting the inherent parallelism of these systems. Nevertheless, they require a large amount of resources in terms of area and power dissipation. Recently, Reservoir Computing (RC) has arisen as a strategic technique to design recurrent neural networks (RNNs) with simple learning capabilities. In this work, we show a new approach to implement RC systems with digital gates. The proposed method is based on the use of probabilistic computing concepts to reduce the hardware required to implement different arithmetic operations. The result is the development of a highly functional system with low hardware resources. The presented methodology is applied to chaotic time-series forecasting.

Electrical stimulation can be used to activate paralyzed muscles for the purpose of restoring motor functions such as stepping in individuals with spinal cord injury. Due to the variability observed in the responses of electrically stimulated muscles, several adaptive control techniques have been developed. The pattern shaper (PS) is an adaptive neural network that has been tested in a software implementation and has been shown to be capable of automatically generating cyclic patterns that are customized for an individual. The results of these tests are encouraging, but implementation of the real-time algorithm currently requires a dedicated PC. The purpose of this research is to develop a hardware implementation of a digital PS neural network to generate the electrical signals to stimulate muscles in individuals with spinal cord injury. We have implemented the pattern shaper algorithm in hardware by mapping the digital logic circuit to a field programmable gate array (FPGA), developed a user interface to input data to the FPGA from a computer, and constructed a wire-wrapped board to implement the PS in hardware for use in clinical tests. This hardware implementation is a step towards the development of low-power, portable, adaptive controller that can be used in electrical stimulation systems.

We present a method to implement trained neural networks on digital programmable hardware. The neural network is trained off-line using a microprocessor system and suitable software. Thereafter the trained neural network is transferred to the hardware implementation. In order to reduce the costs, the network nodes are implemented as multiplexed structures requiring a certain number of clocks for the execution of each iteration. However, in order to increase the overall processing speed the pipelining structure can be also used. It only delays the response with a few microseconds, depending on the clock rate and network complexity. The hardware implementation of neural network using programmable logic devices allows boosting the processing speed, compared to the software implementation, to alternatively reduce the overall costs and provide similar flexibility as the software approach.

Photonic reservoir computing (RC) enables us to introduce artificial neural networks (ANNs) as a deployable hardware-based processing unit beyond the von Neumann architecture. Its unique approach to consider only the output layer weights as adaptive, reduces training complexity and enables the use of ANNs in time-sensitive areas like optical communications. In this paper, we numerically investigate and evaluate the requirements of a photonic RC using an active silicon micro-ring resonator as a photonic integrated circuit (PIC) based reservoir system to compensate for transmission impairments such as chromatic dispersion and nonlinear effects occurring in optical transmission systems. We compare the performance of this RC with results from digital signal processing using Kramers-Kronig reception of distorted PAM-4 symbols from a 56 GBaud WDM transmission experiment over 100 km uncompensated SSMF.

In this paper we present a unified framework for extreme learning machines and reservoir computing (echo state networks), which can be physically implemented using a single nonlinear neuron subject to delayed feedback. The reservoir is built within the delay-line, employing a number of “virtual” neurons. These virtual neurons receive random projections from the input layer containing the information to be processed. One key advantage of this approach is that it can be implemented efficiently in hardware. We show that the reservoir computing implementation, in this case optoelectronic, is also capable to realize extreme learning machines, demonstrating the unified framework for both schemes in software as well as in hardware.

This letter presents the first on-chip bio-inspired reservoir computer (RC) prototype implemented in a 65-nm CMOS. The RC comprises 50 time-multiplexed neurons, and each neuron embeds a strong nonlinearity in a feedback loop. The RC applies a nonlinear transformation to the input and projects it to high-dimensional space, thus allowing linear separation by a simple logistic-regression (LR) layer implemented off-chip. We demonstrate real-time stress detection from electrocardiogram (ECG) signals using the RC. The RC achieves 93% classification accuracy which is 6% better than the state-of-the-art digital classifiers. Operating at 40 kHz, the prototype consumes 27.5 nJ/classification which is $7\times $ lower than the state-of-the-art ECG processors performing similar complexity classification task.

Reservoir computing is a versatile approach for implementing physically Recurrent Neural networks which take advantage of a reservoir, consisting of a set of interconnected neurons with temporal dynamics, whose weights and biases are fixed and do not need to be optimized. Instead, the training takes place only at the output layer towards a specific task. One important requirement for these systems to work is nonlinearity, which in optical setups is usually obtained via the saturation of the detection device. In this work, we explore a distinct approach using a photorefractive crystal as the source of the nonlinearity in the reservoir. Furthermore, by leveraging on the time response of the photorefractive media, one can also have the temporal interaction required for such architecture. If we space out in time the propagation of different states, the temporal interaction is lost, and the system can work as an extreme learning machine. This corresponds to a physical implementation of a Feed-Forward Neural Network with a single hidden layer and fixed random weights and biases. Some preliminary results are presented and discussed.

Implementation of convolutional neural network and 8-bit reservoir computing in CMOS compatible VRRAM

The Reservoir Computing (RC) area has gain relevance during the last years. An RC model is a neural network with a recurrent hidden subnetwork (named reservoir) composed by randomly initialized hidden-hidden weights over the edges, and the rest of the model parameters are trained using linear regression. In this paper, we study the effect of training a selected subset of hidden-hidden weights and the linear regression parameters. For the training, we use a popular evolutionary technique named Particle Swarm Intelligence. The proposed method is inspired in Extreme Learning machines, where only a good selection of adjustable parameters is trained. Empirical analysis was made over well-known simulated time-series benchmarks and a real-world problems.

This chapter covers the implementation of deep learning neural networks and memristive systems. In particular, deep memristive convolutional neural network (CNN) implementation is illustrated. In addition, the main issues and challenges of deep neural network implementation are discussed.

We present a method to improve the performance of a reservoir computer by keeping the reservoir fixed and increasing the number of output neurons. The additional neurons are nonlinear functions, typically chosen randomly, of the reservoir neurons. We demonstrate the interest of this expanded output layer on an experimental opto-electronic system subject to slow parameter drift which results in loss of performance. We can partially recover the lost performance by using the output layer expansion. The proposed scheme allows for a trade-off between performance gains and system complexity.

We present our latest results on silicon photonics neuromorphic information processing based on techniques like reservoir computing. We will discuss aspects like scalability, novel architectures for enhanced power efficiency, as well as all-optical readout. Additionally, we will touch upon new machine learning techniques to operate these integrated readouts.

We report a photonic implementation of the Reservoir Computer (PhRC) for sensing application. The kernel of the PhRC will be in the form of chaotic microcavity on a photonic crystal cavity platform designed to discriminate different analytes. The discrimination is achieved by recognising the unique temporal signal signature arising from the chaotic kernel in the presence of different analyte. The unique temporal signature is obtained exploiting the sensitive to the initial-condition response of a billiard shaped microcavity. This is noted that the new discrimination approach reported is performed directly on the temporal signal, in contrast to the conventional spectral fingerprinting.

Deep Neural Networks (DNNs) are often associated with a large number of data-parallel computations. Therefore, data-centric computing paradigms, such as Processing in Memory (PIM), are being widely explored for DNN acceleration applications. A recent PIM architecture, developed and commercialized by the UPMEM company, has demonstrated impressive performance boost over traditional CPU-based systems for a wide range of data-parallel applications. However, the application domain of DNN acceleration is yet to be explored on this PIM platform. In this work, we present successful implementations of DNNs on the UPMEM PIM system. We explore multiple operation mapping schemes with different optimization goals and accelerate two CNN algorithms using these schemes. Based on the data achieved from the physical implementation of the DNNs on the UPMEM system, we compare the performance of our DNN implementation with several other recently proposed PIM architecture.