Biologically Plausible Class Discrimination Based Recurrent Neural Network Training for Motor Pattern Generation

Constructive training of recurrent neural networks using hybrid optimization

This thesis deals with recurrent neural networks, a particular class of artificial neural networks which can learn a generative model of input sequences. The input is mapped, through a feedback loop and a non-linear activation function, into a hidden state, which is then projected into the output space, obtaining either a probability distribution or the new input for the next time-step. This work consists mainly of two parts: a theoretical study for helping the understanding of recurrent neural networks framework, which is not yet deeply investigated, and their application to non-linear prediction problems, since recurrent neural networks are really powerful models suitable for solving several practical tasks in different fields. For what concerns the theoretical part, we analyse the weaknesses of state-of-the-art models and tackle them in order to improve the performance of a recurrent neural network. Firstly, we contribute in the understanding of the dynamical properties of a recurrent neural network, highlighting the close relation between the definition of stable limit cycles and the echo state property of an echo state network. We provide sufficient conditions for the convergence of the hidden state to a trajectory, which is uniquely determined by the input signal, independently of the initial states. This may help extend the memory of the network and increase the design options for the network. Moreover, we develop a novel approach to address the main problem in training recurrent neural networks, the so-called vanishing gradient problem. Our new method allows us to train a very simple recurrent neural network, making the gradient not to vanish even after many time-steps. Exploiting the singular value decomposition of the vanishing factors in the gradient and random matrices theory, we find that the singular values have to be confined in a narrow interval and derive conditions about their root mean square value. Then, we also improve the efficiency of the training of a recurrent neural network, defining a new method for speeding up this process. Thanks to a least square regularization, we can initialize the parameters of the network, in order to set them closer to the minimum and running fewer epochs of classical training algorithms. Moreover, it is also possible to completely train the network with our initialization method, running more iterations of it without losing in performance with respect to classical training algorithms. Finally, it is also possible to use it as a real-time learning algorithm, adjusting the parameters to the new data through one iteration of our initialization. In the last part of this thesis, we apply recurrent neural networks to non-linear prediction problems. We consider prediction of numerical sequences, estimating the following input choosing it from a probability distribution. We study an automatic text generation problem, where we need to predict the following character in order to compose words and sentences, and a path prediction of walking mobile users in the central area of a city, as a sequence of crossroads. Then, we analyse the prediction of video frames, discovering a wide range of applications related to the prediction of movements. We study the collision problem of bouncing balls, taking into account only the sequence of video frames without any knowledge about the physical characteristics of the problem, and the distribution over days of mobile user in a city and in a whole region. Finally, we address the state-of-the-art problem of missing data imputation, analysing the incomplete spectrogram of audio signals. We restore audio signals with missing time-frequency data, demonstrating via numerical experiments that a performance improvement can be achieved involving recurrent neural networks.

Multi-step prediction, predicting values multiple steps ahead, is a difficult problem because of the accumulation of error after successive predictions. Recurrent neural networks (RNN) have been applied successfully to multi-step prediction problems such as language modeling where training data are abundant and regular. However, applying RNN to problems in healthcare is still challenging as RNN requires large, feature complete training data in which intervals between time points are fixed which are not reflective of healthcare data. To overcome these problems, we propose strategies to train RNN using missing data which involve using the RNN to fill in the incomplete or missing time points online. The strategies enables the efficient use of training data as well as the adoption of RNN in problems where the variables in a multivariate time series are recorded at irregular intervals or at different frequencies, where there are a lot of missing data. We train RNN using the Alzheimer's Disease Neuroimaging Initiative (ADNI) database to model the monthly progression of Alzheimer's disease for seven years into the future which is a multi-step prediction problem. We show that RNN outperforms baseline models.

The main task of using neural networks is the prompt and accurate solution of various creative tasks, including the analysis and synthesis of news flows, while maintaining the continuity of learning. The result of such processing can be digests, filtered news streams, as well as event forecasts that allow for proactivity in management decisions. Known methods of news processing by neural networks and technical solutions that implement them do not fully provide a solution to the problems that arise in this area. It is necessary to expand their functionality, and improve the space-time signal binding in recurrent neural networks. When processing news flows, simultaneously with continuous training of recurrent neural networks, selection, recognition, restoration, prediction and synthesis of news should be carried out. To reduce the severity of the problem, a promising method of multifunctional processing of news flows is proposed using recurrent neural networks with a logical organization of layers and continuous learning. The method is based on the development of associative processing of textual information in streaming recurrent neural networks with controlled elements. The key features of this method are the multifunctional processing of information flows with changing laws of news appearance. The method provides for operational selection, recognition, restoration, forecasting and synthesis of news based on deep associative continuous processing of links between text elements. The neural network system that implements the proposed method differs from the known solutions by new elements, connections between them, as well as by the functions performed. The results of the experiments confirmed the extended functionality of the method. New features of processing news texts by streaming RNNs are revealed. The proposed solutions can be used to create a new generation of intelligent systems not only for word processing, but also for other types of information.

With the advantages of high modeling accuracy and large bandwidth, recurrent neural network (RNN) based inversion model control has been proposed for output tracking. However, some issues still need to be addressed when using the RNN-based inversion model. First, with limited number of parameters in RNN, it cannot model the low-frequency dynamics accurately, thus an extra linear model has been used, which can become an interference for tracking control at high frequencies. Moreover, the control speed and the RNN modeling accuracy cannot be improved simultaneously as the control sampling speed is restricted by the length of the RNN training set. Therefore, this article focuses on addressing these limitations of RNN-based inversion model control. Specifically, a novel modeling method is proposed to incorporate the linear model in a way that it does not affect the existing high-frequency control performance achieved by RNN. Additionally, an interpolation method is proposed to double the sampling frequency (compared to the RNN training sampling frequency). Analysis on the stability issues which may arise when the proposed new model is used for predictive control is presented along with the instructions on determining the parameters for ensuring the closed-loop stability. Finally, the proposed approach is demonstrated on a commercial piezo actuator, and the experiment results show that the tracking performances can be significantly improved.

Task-trained artificial recurrent neural networks (RNNs) provide a computational modeling framework of increasing interest and application in computational, systems, and cognitive neuroscience. RNNs can be trained, using deep-learning methods, to perform cognitive tasks used in animal and human experiments and can be studied to investigate potential neural representations and circuit mechanisms underlying cognitive computations and behavior. Widespread application of these approaches within neuroscience has been limited by technical barriers in use of deep-learning software packages to train network models. Here, we introduce PsychRNN, an accessible, flexible, and extensible Python package for training RNNs on cognitive tasks. Our package is designed for accessibility, for researchers to define tasks and train RNN models using only Python and NumPy, without requiring knowledge of deep-learning software. The training backend is based on TensorFlow and is readily extensible for researchers with TensorFlow knowledge to develop projects with additional customization. PsychRNN implements a number of specialized features to support applications in systems and cognitive neuroscience. Users can impose neurobiologically relevant constraints on synaptic connectivity patterns. Furthermore, specification of cognitive tasks has a modular structure, which facilitates parametric variation of task demands to examine their impact on model solutions. PsychRNN also enables task shaping during training, or curriculum learning, in which tasks are adjusted in closed-loop based on performance. Shaping is ubiquitous in training of animals in cognitive tasks, and PsychRNN allows investigation of how shaping trajectories impact learning and model solutions. Overall, the PsychRNN framework facilitates application of trained RNNs in neuroscience research.

We investigate how unsupervised training of recurrent neural networks (RNNs) and their deep hierarchies can benefit a supervised task like temporal pattern detection. The RNNs are fully and fast trained by unsupervised algorithms and only supervised feed-forward readouts are used. The unsupervised RNNs are shown to perform better in a rigorous comparison against state-of-art random reservoir networks. Unsupervised greedy bottom-up trained hierarchies of such RNNs are shown being capable of big performance improvements over single layer setups.

With recent advances in learning algorithms, recurrent networks of spiking neurons are achieving performance that is competitive with vanilla recurrent neural networks. However, these algorithms are limited to small networks of simple spiking neurons and modest-length temporal sequences, as they impose high memory requirements, have difficulty training complex neuron models and are incompatible with online learning. Here, we show how the recently developed Forward-Propagation Through Time (FPTT) learning combined with novel liquid time-constant spiking neurons resolves these limitations. Applying FPTT to networks of such complex spiking neurons, we demonstrate online learning of exceedingly long sequences while outperforming current online methods and approaching or outperforming offline methods on temporal classification tasks. The efficiency and robustness of FPTT enable us to directly train a deep and performant spiking neural network for joint object localization and recognition, demonstrating the ability to train large-scale dynamic and complex spiking neural network architectures. Memory efficient online training of recurrent spiking neural networks without compromising accuracy is an open challenge in neuromorphic computing. Yin and colleagues demonstrate that training a recurrent neural network consisting of so-called liquid time-constant spiking neurons using an algorithm called Forward-Propagation Through Time allows for online learning and state-of-the-art performance at a reduced computational cost compared with existing approaches.

Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization

Recurrent neural networks (RNNs) have emerged as a prominent tool for modeling cortical function, and yet their conventional architecture is lacking in physiological and anatomical fidelity. In particular, these models often fail to incorporate two crucial biological constraints: i) Dale’s law, i.e., sign constraints that preserve the “type” of projections from individual neurons, and ii) Structured connectivity motifs, i.e., highly sparse yet defined connections amongst various neuronal populations. Both constraints are known to impair learning performance in artificial neural networks, especially when trained to perform complicated tasks; but as modern experimental methodologies allow us to record from diverse neuronal populations spanning multiple brain regions, using RNN models to study neuronal interactions without incorporating these fundamental biological properties raises questions regarding the validity of the insights gleaned from them. To address these concerns, our work develops methods that let us train RNNs which respect Dale’s law whilst simultaneously maintaining a specific sparse connectivity pattern across the entire network. We provide mathematical grounding and guarantees for our approaches incorporating both types of constraints, and show empirically that our models match the performance of RNNs trained without any constraints. Finally, we demonstrate the utility of our methods for inferring multi-regional interactions by training RNN models of the cortical network to reconstruct 2-photon calcium imaging data during visual behaviour in mice, whilst enforcing data-driven, cell-type specific connectivity constraints between various neuronal populations spread across multiple cortical layers and brain areas. In doing so, we find that the interactions inferred by our model corroborate experimental findings in agreement with the theory of predictive coding, thus validating the applicability of our methods.

In order to improve the accuracy and stability of English translation, this paper proposes an English translation model based on recurrent neural network. Based on the end-to-end encoder-decoder architecture, a recursive neural network (RNN) English machine translation model is designed to promote machine autonomous learning features, transform the distributed corpus data into word vectors, and directly map the source language and target language through the recurrent neural network. Selecting semantic errors to construct the objective function during training can well balance the influence of each part of the semantics and fully consider the alignment information, providing a strong guidance for the training of deep recurrent neural networks. The experimental results show that the English translation model based on recurrent neural network has high effectiveness and stability. Compared with the baseline system, it has improved about 1.51–1.86 BLEU scores. Conclusion. The model improves the performance and quality of English machine translation model, and the translation effect is better.

Unintentional falls can cause severe injuries and even death, especially if no immediate assistance is given. A fall detection system aims to detect a fall as soon as it occurs, therefore issuing an automatic assistance request. Wearable embedded sensors are emerging as the most viable solution for continuous monitoring since they are more effective, less intrusive and less expensive than other systems. Tailoring a deep learning method to the requirements of microcontrollers entails matching very stringent constraints in terms of both memory and computational power. In addition, datasets acquired with wearable devices are relatively scarce and not necessarily devised for supervised learning. In this work, we discuss the design of a software architecture based on recurrent neural networks which can be effective for fall detection while running entirely onboard a wearable device. The well-known and publicly-available SisFall dataset was adopted and extended with fine-grained temporal annotations to cope with the supervised training of recurrent neural networks. We then show that an appropriate process of architectural minimization together with accurate hyperparameters selection leads to a workable model which compares favorably with other detection techniques. The embedding of the resulting architecture has been validated using a state-of-art hardware device.

In this paper we demonstrate a modified topology for long short-term memory recurrent neural networks that controls the shape of the squashing functions in gating units. We further propose an efficient training framework based on a mini-batch training on sequence level combined with a sequence chunking approach. The framework is evaluated on publicly available data sets containing English and French handwriting by utilizing a GPU based implementation. Speedups of more than 3x are achieved in training recurrent neural network models which outperform state of the art recognition results.

A real-coded genetic algorithm for training recurrent neural networks

A combination of Cuckoo Search algorithm and Recurrent Neural Network is proposed in this paper to track the maximum power from solar photovoltaic system under complex partial shading conditions with minimum tracking speed. Cuckoo Search is a meta-heuristic search algorithm which imitates the behavior of the cuckoo bird. The Recurrent Neural Network is a highly successful technique in nonlinear systems. It can interpolate and extrapolate the random data with high accuracy if the network is properly trained. There are two stages in the proposed hybrid technique. In stage 1 (Off-line stage), Cuckoo Search algorithm identifies the maximum power under different solar radiation patterns and converts them into control pulses. The control pulses are given as target and the solar radiation of individual photovoltaic module as input to train the Recurrent Neural Network, also the weights are optimized. In stage 2 (On-line stage), the trained Recurrent Neural Network is tested for real time conditions. The proposed hybrid algorithm is implemented in MATLAB/Simulink platform and the tracking performance is compared with the state-of-art methods.