Statistical Memristor Model and Its Applications in Neuromorphic Computing

The Sixth International Symposium on Neural Networks (ISNN 2009)

An In2O3 synaptic transistor based on lithium doped Gd2O3 (LiGdO) solid electrolyte was fabricated by a solution process, its basic synaptic functions were realized and its application in neuromorphic computing was demonstrated.

The process of using electronic circuits to replicate the neurobiological architectures seen in the nervous system is known as neuromorphic engineering, also referred to as neuromorphic computing. These technologies are essential for the future of computing, although most of the work in neuromorphic computing has been focused on hardware development. The execution speed, energy efficiency, accessibility and robustness against local failures are vital advantages of neuromorphic computing over conventional methods. Spiking neural networks are generated using neuromorphic computing. This chapter covers the basic ideas of neuromorphic engineering, neuromorphic computing, and its motivating factors and challenges. Deep learning models are frequently referred to as deep neural networks because deep learning techniques use neural network topologies. Deep learning techniques and their different architectures were also covered in this section. Furthermore, Emerging memory Devices for neuromorphic systems and neuromorphic circuits were illustrated.

There is a huge need for embedded machine learning for portable devices and smart sensors to power the next generation of Internet of Things (IoT). Implementation of neural networks involve large number of arithmetic and memory operations. Realization of the arithmetic blocks with conventional digital circuits will inherit a trade-off between accuracy of calculation and area required. On the other hand, existing analog building blocks for neural networks suffer from inaccuracies related to process variation and large power consumption. Moreover, the efficiency of computation-memory interface is degrading since memory bandwidth increment is poor compared to computation throughput growth in CMOS technology. Though the recent large scale neuromorphic circuits have used localized random access memory for reducing memory operations, this local memory size will not be scalable with increasing size of datasets. Here, we explore novel CMOS and post-CMOS circuits to realize ultra-low power neuromorphic circuit by co-design of algorithm and hardware. The solutions also overcome the issue of increasing bandwidth gap between memory operations and computation. First, a deep neural network with 2 convolution layers and 2 fully connected layers, is chosen and tuned for hardware implementation. The network has few tunable parameters (_ 40000) and is 40 times faster in training than usual deep neural networks with 4 layers. We propose a compact, single transistor element for realizing the connections inside the neural network- called ’synapse’. These synapses perform the computations involved by virtue of mismatch inherent in their fabrication. We use a current mirror array with n-input lines and m output lines, to perform ’n x m’ Multiply and Accumulation operation. The resultant neuromorphic circuit can emulate multi-layered artificial vision system. A circuit fabricated in 0.35_m CMOS process is characterized and a behavioral model is simulated for the deep neural network. Here, the learning is done offline. The inputs to the network may vary according to environmental conditions, for which we need an adaptive neural network. Hence we propose a second solution where the neuromorphic circuit can adapt it’s parameters in real time. With the advent of novel nanoscale devices with physical properties well matched to neural network enabling computations at energies much lower than CMOS, the research also focuses on use of a post-CMOS spintronic device-a Domain-Wall magnet for obtaining a low power spike timing dependent plastic (STDP) synapse for online learning. The spin-mode signals are injected across small potential (_ 50mV) through multiple layers of ferromagnetic and non-magnetic layers. Here we discuss the implementation of a spiking neural network with synapses which can be trained according to STDP learning rule. A detailed study with the help of device circuit co-simulation is done. Possible use of this synapse in online, real time learning spiking neural networks is also illustrated in this thesis.

Artificial synaptic simulating pain-perceptual nociceptor and brain-inspired computing based on Au/Bi3.2La0.8Ti3O12/ITO memristor

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

Human brain is capable of optimizing information flow and processing without energy-intensive data shuttling between processor and memory. At the core of this unique capability are billions of neurons connected through trillions of synapses—basic processing units of the brain. The action potentials or “spikes” based temporal processing using the regulated flow of ions across ion channels in neuron cells allows sparse and efficient transmission of data in the brain. Emerging systems based on confined fluidic systems have provided a framework for a new type of neuromorphic computing with lower energy consumption, hardware-level plasticity, and multiple information carriers that emulate natural processes and mechanisms of human brain. These systems mimic neuronal architectures by harnessing and modulating ion transport along artificial channels. The spikes-induced ion-to-surface interactions within these fluidic systems enables the control of ionic conductivity to achieve synaptic plasticity for the realization of brain-inspired functionalities such as memory effect and signal transmission. Herein, this review provides an overview of recent advances in fluidic devices such as memristors and other computing components, covering their basic operations, materials and architectures, as well as applications in neuromorphic computing. The review concludes with a brief outline of the challenges that these emerging technologies face and an outlook for the development of fluidic-based brain-inspired computing.

Resistive switching devices, particularly memristors, have attracted considerable interest due to their promising applications in neuromorphic computing and data storage. However, achieving high performance and reliability remains a significant challenge, especially in the optimization of their ferroelectric and switching properties. In this study, we report a substantial enhancement of both resistive switching and ferroelectric properties in NaNbO3 (NNO)/PbTiO3 (PTO) multilayers, facilitated by interfacial modifications of the electronic structure induced by defects and strain. The well-defined interfaces and strain gradients within the PTO layers lead to substantial alterations in local electronic properties, including Ti 3d orbital hybridization and oxygen octahedral tilting. These structural modifications enhance charge trapping dynamics, resulting in an ON/OFF ratio of 104, compared with 102 in single-layer NNO films. The synergistic effects of enhanced polarization and electronic state modulation are shown to optimize both the ferroelectric and resistive switching behaviors, highlighting the pivotal role of interface engineering in achieving high-performance memristive devices.

In the emerging fields of flexible electronics and bioelectronics, protein-based materials have attracted widespread attention due to their biocompatibility, biodegradability, and processability. Among these materials, silk fibroin (SF), a protein derived from natural silk, has demonstrated significant potential in biomedical applications such as medical sensing and bone tissue engineering, as well as in the development of advanced biosensors. This is primarily due to its highly ordered β-sheet structure, mechanical properties, and processability. Furthermore, SF-based memristors provided a material choice for producing flexible wearable, and even implantable bioelectronic devices, which are expected to advance intelligent health monitoring, electronic skin (e-skin), brain-computer interface (BCI), and other frontier bioelectronic technologies. This review systematically summarizes the latest research progress in SF-based memristors concerning structural design, performance optimization, device integration, and application prospects, particularly highlighting their potential applications in neuromorphic computing and memristive sensors. Concurrently, we objectively analyzed the challenges currently faced by SF-based memristors and prospectively discussed their future development trends. This review provides a theoretical foundation and technological roadmap for biomaterials-based memristor devices, aiming to realize applications in flexible electronics and bioelectronics.

The application of artificial neural networks to the estimation of bioprocess variables will be discussed. In fermentation processes, direct on-line measurements of primary process variables usually are unavailable. The state of the cultivation, therefore, has to be inferred from measurements of secondary variables and any previous knowledge of process dynamics. This research investigates the learning, recall and generalization characteristics of neural networks trained to model the nonlinear behavior of a fermentation process. Two different neural network methodologies are discussed, namely, feed-forward and recurrent neural networks, which differ in their treatment of time dependence. The neural networks are trained by backpropagation using a conjugate gradient technique, which provides a dramatic improvement in the convergence speed. The objective is to use environmental and physiological information available from on-line sensors to estimate concentrations of species in a bioreactor. Results of the neural network estimators are presented, based on experimental data available from the ethanol production by Zymomonas mobilis fermentation. The feed-forward and recurrent neural network methodologies are demonstrated to perform suitably as unmeasurable state estimators. Both networks offer comparable abilities of recall, but recurrent networks perform better than feed-forward networks in generalization.

An investigation into the application of neural networks, fuzzy logic, genetic algorithms, and rough sets to automated knowledge acquisition for classification problems

This chapter is devoted to atmospheric and oceanic satellite remote sensing (RS) NN applications. Two major RS problems, forward problem and inverse problem (or satellite retrievals), are introduced and discussed. Applications of forward models (FM) for solving the forward problem in the process of direct assimilation of satellite measurements and for variational retrievals, as well as applications of retrieval algorithms, solutions of the inverse problem, for assimilation of geophysical parameters in data assimilation systems, are discussed. Correspondingly, two neural network (NN) applications, NN FMs and NN retrieval algorithms, are introduced. An intelligent NN retrieval system, which incorporates an automatic quality control of satellite retrievals, is introduced. Previously developed RS NN applications are reviewed. Theoretical considerations are illustrated with real-life applications of the NN approach to the Special Sensor Microwave Imager (SSM/I). SSM/I NN FM and SSM/I NN retrieval algorithms are introduced, discussed, and compared with FMs and retrieval algorithms developed using other techniques elsewhere. Advantages and limitations of NN FMs and retrieval algorithms are discussed. An example of QuikSCAT wind vector retrievals is used to demonstrate great potential of using the NN technique to go beyond the standard point-wise retrieval paradigm. This chapter contains an extensive list of references giving extended background and further detail to the interested reader on each examined topic. It can serve as a textbook and an introductory reading for students and beginning and advanced investigators interested in learning how to apply the NN emulation technique in different RS applications.KeywordsForward ModelSatellite MeasurementSurface Wind SpeedRetrieval AlgorithmNeural Network AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

CFPA: Congestion aware, fault tolerant and process variation aware adaptive routing algorithm for asynchronous Networks-on-Chip

This paper explores the use and applications of neural networks in the construction of a text markup program. This research paper describes various tasks that require textual data analysis and discusses the problems, issues, and solutions that accompany them. Interest in neural networks has increased in recent years, and they are find­ing applications in a wide variety of fields, such as business, medicine, engineering, geology, and physics. Neural networks have made great strides in forecasting, planning, and management. There are several reasons for this situation. Neural networks are very powerful modeling systems capable of creating complex dependencies. Neural networks can be widely used in areas such as text/speech recognition, semantic search, decision support/expert systems, inventory forecasting, data storage systems and content analysis. The object of the work is the process of functioning of neural networks and an algorithm for text markup recognition. The purpose of the scientific paper is the application of neural networks in the construction of the program for the markup of the text. In order to achieve the goal, the following tasks were put forward: a) study existing neural networks, b) choose a neural network to cre­ate a model and study its structure, c) convert input data to feed it into a neural network model. Representation and analysis of symbolic structures in neural networks seems to be an interesting and useful direction in neural network theory. In this paper, we have reviewed some neural network architectures that may merit further consideration in these circumstances. In what follows, we will focus on specific experiments in this area. Thus, this paper outlines a problem area for further research and testing.

Statistical yield improvement under process variations of multi-valued memristor-based memories