Neuro Controller For Time Varying Dynamical System.

Deep neural network (DNN) architectures such as recurrent neural networks and transformers display outstanding performance in modeling sequential unstructured data. However, little is known about their merit to model customer churn with time-varying data. The paper provides a comprehensive evaluation of the ability of recurrent neural networks and transformers for customer churn prediction (CCP) using time-varying behavioral features in the form of recency, frequency, and monetary value (RFM). RFM variables are the backbone of CCP and, more generally, customer behavior forecasting. We examine alternative strategies for integrating time-varying and non-variant customer features in one network architecture. In this scope, we also assess hybrid approaches that incorporate the outputs of DNNs in conventional CCP models. Using a comprehensive panel data set from a large financial services company, we find recurrent neural networks to outperform transformer architectures when focusing on time-varying RFM features. This finding is confirmed when time-invariant customer features are included, independent of the specific form of feature integration. Finally, we find no statistical evidence that hybrid approaches (based on regularized logistic regression and extreme gradient boosting) improve predictive performance—highlighting that DNNs and especially recurrent neural networks are suitable standalone classifiers for CCP using time-varying RFM measures.

The Sixth International Symposium on Neural Networks (ISNN 2009)

Soccer video semantic analysis has attracted a lot of researchers in the last few years. Many methods of machine learning have been applied to this task and have achieved some positive results, but the neural network method has not yet been used to this task from now. Taking into account the advantages of Convolution Neural Network(CNN) in fully exploiting features and the ability of Recurrent Neural Network(RNN) in dealing with the temporal relation, we construct a deep neural network to detect soccer video event in this paper. First we determine the soccer video event boundary which we used Play-Break(PB) segment by the traditional method. Then we extract the semantic features of key frames from PB segment by pre-trained CNN, and at last use RNN to map the semantic features of PB to soccer event types, including goal, goal attempt, card and corner. Because there is no suitable and effective dataset, we classify soccer frame images into nine categories according to their different semantic views and then construct a dataset called Soccer Semantic Image Dataset(SSID) for training CNN. The sufficient experiments evaluated on 30 soccer match videos demonstrate the effectiveness of our method than state-of-art methods.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

This thesis deals with recurrent neural networks, a particular class of artificial neural networks which can learn a generative model of input sequences. The input is mapped, through a feedback loop and a non-linear activation function, into a hidden state, which is then projected into the output space, obtaining either a probability distribution or the new input for the next time-step. This work consists mainly of two parts: a theoretical study for helping the understanding of recurrent neural networks framework, which is not yet deeply investigated, and their application to non-linear prediction problems, since recurrent neural networks are really powerful models suitable for solving several practical tasks in different fields. For what concerns the theoretical part, we analyse the weaknesses of state-of-the-art models and tackle them in order to improve the performance of a recurrent neural network. Firstly, we contribute in the understanding of the dynamical properties of a recurrent neural network, highlighting the close relation between the definition of stable limit cycles and the echo state property of an echo state network. We provide sufficient conditions for the convergence of the hidden state to a trajectory, which is uniquely determined by the input signal, independently of the initial states. This may help extend the memory of the network and increase the design options for the network. Moreover, we develop a novel approach to address the main problem in training recurrent neural networks, the so-called vanishing gradient problem. Our new method allows us to train a very simple recurrent neural network, making the gradient not to vanish even after many time-steps. Exploiting the singular value decomposition of the vanishing factors in the gradient and random matrices theory, we find that the singular values have to be confined in a narrow interval and derive conditions about their root mean square value. Then, we also improve the efficiency of the training of a recurrent neural network, defining a new method for speeding up this process. Thanks to a least square regularization, we can initialize the parameters of the network, in order to set them closer to the minimum and running fewer epochs of classical training algorithms. Moreover, it is also possible to completely train the network with our initialization method, running more iterations of it without losing in performance with respect to classical training algorithms. Finally, it is also possible to use it as a real-time learning algorithm, adjusting the parameters to the new data through one iteration of our initialization. In the last part of this thesis, we apply recurrent neural networks to non-linear prediction problems. We consider prediction of numerical sequences, estimating the following input choosing it from a probability distribution. We study an automatic text generation problem, where we need to predict the following character in order to compose words and sentences, and a path prediction of walking mobile users in the central area of a city, as a sequence of crossroads. Then, we analyse the prediction of video frames, discovering a wide range of applications related to the prediction of movements. We study the collision problem of bouncing balls, taking into account only the sequence of video frames without any knowledge about the physical characteristics of the problem, and the distribution over days of mobile user in a city and in a whole region. Finally, we address the state-of-the-art problem of missing data imputation, analysing the incomplete spectrogram of audio signals. We restore audio signals with missing time-frequency data, demonstrating via numerical experiments that a performance improvement can be achieved involving recurrent neural networks.

What does it mean when we say that computers can ‘write’ and how are recent developments in neural networks and machine learning changing this capacity? This article examines the long-standing literary fear of authorship being replaced by machines while also interrogating the labour and credit implications that sit behind widely used structures of authorship in a technological age. The argument makes reference to one work of computer-generated writing – Johannes Helden & Hakan Jonson’s Evolution [2014] – and to one software paradigm (a character-based recurrent neural networks for language acquisition trained on the corpus of the journal Textual Practice). I here argue that unless we conceive more broadly of the criteria for ‘authorship’ as a labour function, and unless we take seriously the need to see textual production as social production, hybridized (but predominantly) machine identities will come to dominate a literary landscape.

Large scale artificial neural networks (ANNs) have been widely used in data processing applications. The recurrent neural network (RNN) is a special type of neural network equipped with additional recurrent connections. Such a unique architecture enables the recurrent neural network to remember the past processed information and makes it an expressive model for nonlinear sequence processing tasks. However, the large computation complexity makes it difficult to effectively train a recurrent neural network and therefore significantly limits the research on the recurrent neural network in the last 20 years. In recent years, the use of graphics processing units (GPUs) becomes a significant advance to speed up the training process of large scale neural networks by taking advantage of the massive parallelism capabilities of GPUs. In this paper, we propose an efficient GPU implementation of the large scale recurrent neural network and demonstrate the power of scaling up the recurrent neural network with GPUs. We first explore the potential parallelism of the recurrent neural network and propose a fine-grained two-stage pipeline implementation. Experiment results show that the proposed GPU implementation can achieve 2 ~ 11 x speed-up compared with the basic CPU implementation with the Intel Math Kernel Library. We then use the proposed GPU implementation to scale up the recurrent neural network and improve its performance. The experiment results of the Microsoft Research Sentence Completion Challenge demonstrate that the large scale recurrent network without class layer is able to beat the traditional class-based modest-size recurrent network and achieve an accuracy of 47%, the best result achieved by a single recurrent neural network on the same dataset.

Unlimited access to information and data sharing wherever and at any time for anyone and anything is a fundamental component of fifth-generation (5G) wireless communication and beyond. Therefore, it has become inevitable to exploit the super-high frequency (SHF) and millimeter-wave (mmWave) frequency bands for future wireless networks due to their attractive ability to provide extremely high data rates because of the availability of vast amounts of bandwidth. However, due to the characteristics and sensitivity of wireless signals to the propagation effects in these frequency bands, more accurate path loss prediction models are vital for the planning, evaluating, and optimizing future wireless communication networks. This paper presents and evaluates the performance of several well-known machine learning methods, including multiple linear regression (MLR), polynomial regression (PR), support vector regression (SVR), as well as the methods using decision trees (DT), random forests (RF), K-nearest neighbors (KNN), artificial neural networks (ANN), and artificial recurrent neural networks (RNN). RNNs are mainly based on long short-term memory (LSTM). The models are compared based on measurement data to provide the best fitting machine-learning-based path loss prediction models. The main results obtained from this study show that the best root-mean-square error (RMSE) performance is given by the ANN and RNN-LSTM methods, while the worst is for the MLR method. All the RMSE values for the given learning techniques are in the range of to dB. Furthermore, this work shows that the models (except for the MLR model) perform excellently in fitting actual measurement data for wireless communications in enclosed indoor environments since they provide R-squared and correlation values higher than and , respectively. The paper shows that these learning methods could be used as accurate and stable models for predicting path loss in the mmWave frequency regime.

Application of the diagonal recurrent wavelet neural network to solar irradiation forecast assisted with fuzzy technique

Physics-informed recurrent neural networks for linear and nonlinear flame dynamics

Long–short-term memory encoder–decoder with regularized hidden dynamics for fault detection in industrial processes

In the field of big data analytics, deep learning has garnered a lot of interest due to the fact that it is capable of doing feature extraction and classification. The categorization of big data has traditionally been carried out by academics via the use of Machine Learning methods; nonetheless, the process of feature extraction has traditionally been carried out manually. Deep learning is a method developed by researchers that allows for the extraction of features via the use of various algorithms. This research made use of an algorithm known as "Convolutional Neural Network," or CNN for short, in order to get a deeper comprehension of the dependability of big data analytics. Primary research has been conducted in order to get an understanding of how hidden nodes and layers influence the accuracy of this neural network. The "Recurrent Neural Network" (also known as RNN) and the "Artificial Neural Network" (also known as ANN) are the two other neural network algorithms that have been selected for comparison with CNN in order to evaluate whether or not CNN is more effective than the other algorithms. In the first research, the reliability of CNN as well as other methods was analyzed in relation to the number of nodes, the number of hidden layers, the amount of training time, and the amount of validating time. Control variables were employed in both correlation and regression investigations. These variables included Hidden Nodes, Training time, Valid time, and Secret Layers. CNN, ANN, and RNN were significant variables. According to the findings, CNN is more dependable than other neural networks for analyzing huge amounts of data, scoring 92%, whereas other neural networks scored less than 90%. The inclusion of hidden nodes results in a significant increase in CNN's reliability.

Artificial neural networks allow the identification of black-box models. This paper proposes a method aimed at replicating the static and dynamic behavior of a DC-DC power converter based on a recurrent nonlinear autoregressive exogenous neural network. The method proposed in this work applies an algorithm that trains a neural network based on the inputs and outputs (currents and voltages) of a Buck converter. The approach is validated by means of simulated data of a realistic nonsynchronous Buck converter model programmed in Simulink and by means of experimental results. The predictions made by the neural network are compared to the actual outputs of the system, to determine the accuracy of the method, thus validating the proposed approach. Both simulation and experimental results show the feasibility and accuracy of the proposed black-box approach.

Research in the area of artificial neural networks has advanced at a rapid pace in recent times. The artificial neural network possesses a good ability to learn adaptively. The decision process in an artificial neural network is based on certain nonlinear operations. Such nonlinearities are useful: i) in improving the convergence speed (of the algorithm); ii) to provide more general nonlinear mapping between input-output signals; and iii) to reduce the effect of outliers in the measurements. One of the most successful artificial neural networks is the so-called feed forward neural network. In this chapter first the description of the feed forward neural network and its training algorithms is given. Next, parameter estimation using this approach is discussed. The presentation of training algorithms is such that it facilitates MATLAB implementation. Subsequently, recurrent neural networks are described. Several schemes based on recurrent neural networks are presented for parameter estimation of dynamical systems. Subsequently, the genetic algorithm is described and its application to parameter estimation considered.

The research paper analysed two types of artificial neural networks (ANN) for predicting the power output of a hydroelectric power plant. The first type is the feedforward artificial neural network (FF-ANN), while the second type is the recurrent neural network (RNN). Authors thoroughly analysed the steps taken in the process of implementing neural networks for this purpose, from data collection and preparation to training, evaluation, and results analysis. The input data includes net head, water flow through the turbine, upper water level, lower water level, and the temperature of the inlet cooling water. Based on the correlation coefficient of individual variables from the input layer with the output power, new ANN implementations were conducted. The results obtained from these variations were systematically analysed to achieve precise modelling, focusing on dynamic changes in water flow through the turbine.