Convergence of learning algorithms with constant learning rates

The conventional back-propagation algorithm is basically a gradient-descent method, it has the problems of local minima and slow convergence. A new generalized back-propagation algorithm which can effectively speed up the convergence rate and reduce the chance of being trapped in local minima is introduced. The new back-propagation algorithm is to change the derivative of the activation function so as to magnify the backward propagated error signal, thus the convergence rate can be accelerated and the local minimum can be escaped. In this letter, we also investigate the convergence of the generalized back-propagation algorithm with constant learning rate. The weight sequences in generalized back-propagation algorithm can be approximated by a certain ordinary differential equation (ODE). When the learning rate tends to zero, the interpolated weight sequences of generalized back-propagation converge weakly to the solution of associated ODE.

Based on various approaches, several different learing algorithms have been given in the literature for neural networks. Almost all algorithms have constant learning rates or constant accelerative parameters, though they have been shown to be effective for some practical applications. The learning procedure of neural networks can be regarded as a problem of estimating (or identifying) constant parameters (i.e. connection weights of network) with a nonlinear or linear observation equation. Making use of the Kalman filtering, we derive a new back-propagation algorithm whose learning rate is computed by a time-varying Riccati difference equation. Perceptron-like and correlational learning algorithms are also obtained as special cases. Furthermore, a self-organising algorithm of feature maps is constructed within a similar framework.

In this paper, a smoothing algorithm with constant learning rate is presented for training two kinds of fuzzy neural networks (FNNs): max-product and max-min FNNs. Some weak and strong convergence results for the algorithm are provided with the error function monotonically decreasing, its gradient going to zero, and weight sequence tending to a fixed value during the iteration. Furthermore, conditions for the constant learning rate are specified to guarantee the convergence. Finally, three numerical examples are given to illustrate the feasibility and efficiency of the algorithm and to support the theoretical findings.

Two multi-agent policy iteration learning algorithms are proposed in this work. The two proposed algorithms use the exponential moving average approach along with the Q-learning algorithm as a basis to update the policy for the learning agent so that the agent's policy converges to a Nash equilibrium policy. The first proposed algorithm uses a constant learning rate when updating the policy of the learning agent, while the second proposed algorithm uses two different decaying learning rates. These two decaying learning rates are updated based on either the Win-or-Learn-Fast (WoLF) mechanism or the Win-or-Learn-Slow (WoLS) mechanism. The WoLS mechanism is introduced in this article to make the algorithm learn fast when it is winning and learn slowly when it is losing. The second proposed algorithm uses the rewards received by the learning agent to decide which mechanism (WoLF mechanism or WoLS mechanism) to use for the game being learned. The proposed algorithms have been theoretically analyzed and a mathematical proof of convergence to pure Nash equilibrium is provided for each algorithm. In the case of games with mixed Nash equilibrium, our mathematical analysis shows that the second proposed algorithm converges to an equilibrium. Although our mathematical analysis does not explicitly show that the second proposed algorithm converges to a Nash equilibrium, our simulation results indicate that the second proposed algorithm does converge to Nash equilibrium. The proposed algorithms are examined on a variety of matrix and stochastic games. Simulation results show that the second proposed algorithm converges in a wider variety of situations than state-of-the-art multi-agent reinforcement learning algorithms.

Sensitivity analysis of neural adaptive filters with respect to the slope parameter of a neuron activation function is performed. The analysis is provided both for a feedforward neural adaptive filter and a recurrent perceptron. The slope affects stability and convergence characteristics of a filter via inherent relationship between the slope and the learning rate parameter. In addition, it determines character of an activation function, i.e. whether it is contractive or expansive mapping. Presented analysis shows that gradient-descent based learning algorithms with an adaptive learning rate significantly reduce sensitivity of a neural adaptive filter with respect to the slope parameter, when compared with learning algorithms with a constant learning rate. Experimental results on the test speech and HRV signals support the analysis.

The convergence of online learning algorithms is analyzed using the tools of the stochastic approximation theory, and proved under very weak conditions. A general framework for online learning algorithms is first presented. This framework encompasses the most common online learning algorithms in use today, as illustrated by several examples. The stochastic approximation theory then provides general results describing the convergence of all these learning algorithms at once.

A unified learning algorithm to extract principal and minor components

This paper addresses the questions of improving convergence performance for back propagation (BP) neural network. For traditional BP neural network algorithm, the learning rate selection is depended on experience and trial. In this paper, based on Taylor formula the function relationship between the total quadratic training error change and connection weights and biases changes is obtained, and combined with weights and biases changes in batch BP learning algorithm, the formula for self-adaptive learning rate is given. Unlike existing algorithm, the self-adaptive learning rate depends on only neural network topology, training samples, average quadratic error and error curve surface gradient but not artificial selection. Simulation results show iteration times is significant less than that of traditional batch BP learning algorithm with constant learning rate.

Tracking-by-detection methods have been widely studied and some promising results have been obtained. These methods use discriminative appearance models to train and update online classifiers. They also use a sliding window to detect samples which will then be classified. Then, the location of the sample with the maximum classifier response will be selected as the new location. Compressive tracking was recently proposed with an appearance model based on features extracted in the compressed domain. However, CT uses a fixed-size tracking box to detect samples, and this is unsuitable for practice applications. CT detects samples around the selected region of the previous frame within a fixed radius. Here, the classifier may become inaccurate if the selected region drifts. The fixed radius is also not suitable for tracking targets that experience abrupt acceleration changes. Furthermore, CT updates the classifier parameters with constant learning rate. If the target is fully occluded for an extended period, the classifier will instead learn the features of the cover object and the target will ultimately be lost. In this paper, we present a multi-scale compressive tracker. This tracker integrates an improved appearance model based on normalized rectangle features extracted in the adaptive compressive domain into the bootstrap filter. This type of feature extraction is efficient, and the computation complexity does not increase as the tracking regions become larger. The classifier response is utilized to generate particle importance weight and a re-sample procedure preserves samples according to weight. A 2-order transition model considers the target velocity to estimate the current position and scale status. In this way, the sampling is not limited to a fixed range. Here, feedback strategies are adopted to adjust learning rate for occlusion. Experimental results on various benchmark challenging sequences have demonstrated the superior performance of our tracker when compared with several state-of-the-art tracking algorithms.

While direct, model-free reinforcement learning often performs better than model-based approaches in practice, only the latter have yet supported theoretical guarantees for finite-sample convergence. A major difficulty in analyzing the direct approach in an online setting is the absence of a definitive exploration strategy. We extend the notion of admissibility to direct reinforcement learning and show that standard Q-learning with optimistic initial values and constant learning rate is admissible. The notion justifies the use of a greedy strategy that we believe performs very well in practice and holds theoretical significance in deriving finite-sample convergence for direct reinforcement learning. We present empirical evidence that supports our idea.

A new artificial intelligent peak power load forecaster based on non-fixed neural networks

An adaptive estimation method with exploration and exploitation modes for non-stationary environments

In this paper we have proposed a new way to achieve the optimum learning rate that can reduce the learning time of the multi layer feed forward neural network. The effect of optimum numbers of inner iterations and numbers of hidden nodes on learning time and recognition rate has been shown. The Principal Component Analysis and Multilayer Feed Forward Neural Network are applied in face recognition system for feature extraction and recognition respectively. The paper shows that the recognition rate and training time are dependent on numbers on hidden nodes. In this approach we have used variable learning rate and demonstrated its superiority over constant learning rate. We have used ORL database for all the experiments.

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

Artificial neural networks provide a neurocomputing approach for solving complex problems that might otherwise not have a tractable solution. Neural networks are usually trained using local gradient-based procedures. Such methods are frequently find sub-optimal solutions being trapped in local minima. And It will be difficult for the traditional neural network to solve such problems where the input/output data sets used to train a neural network may not be hundred percent precise but within certain range. A learning algorithm based on interval optimization is presented in this paper. The above disadvantages of the traditional learning algorithm can be settled by using this method.