A Fast Partitioning Algorithm and a Comparison of Binary Feedforward Neural Networks

A fast and robust algorithm for training feedforward neural networks (FNNs) by using a variable forgetting factor and U-D factorization-based recursive prediction error (RPE) method is proposed. In comparison with the backpropagation (BP) and RPE based learning algorithms, the proposed algorithm, called UD-RPE, can provide much more accurate learning results in fewer iterations with fewer hidden nodes and improve convergence rate and numerical stability (robustness). In addition, it is less sensitive to start-up parameters, such as initial weights and initial covariance matrix, and the randomness in the observed data. It also has good generalization ability and needs less learning time. Simulation results of nonlinear dynamic system modeling and identification show that the algorithm proposed here is an effective and efficient learning algorithm for FNNs.

This paper presents the development of several efficient learning algorithms for neural networks (ELEANNE). The ELEANNE 1 and ELEANNE 2 are two recursive least-squares learning algorithms, proposed for training single-layered neural networks with analog output. This paper also proposes a new optimization strategy for training single-layered neural networks, which provides the basis for the development of a variety of efficient learning algorithms. This optimization strategy is the source of the ELEANNE 3, a second-order learning algorithm for training single-layered neural networks with binary output. A simplified version of this algorithm, called ELEANNE 4, is also derived on the basis of some simplifying but reasonable assumptions. The two algorithms developed for single-layered neural networks provide the basis for the derivation of ELEANNE 5 and ELEANNE 6, which are proposed for training multilayered neural networks with binary output. The ELEANNE 7 is an efficient algorithm developed for training multilayered neural networks with either binary or analog output. >

In this paper a general class of fast learning algorithms for feedforward neural networks is introduced and described. The approach exploits the separability of each layer into linear and nonlinear blocks and consists of two steps. The first step is the descent of the error functional in the space of the outputs of the linear blocks (descent in the neuron space), which can be performed using any preferred optimization strategy. In the second step, each linear block is optimized separately by using a least squares (LS) criterion. To demonstrate the effectiveness of the new approach, a detailed treatment of a gradient descent in the neuron space is conducted. The main properties of this approach are the higher speed of convergence with respect to methods that employ an ordinary gradient descent in the weight space backpropagation (BP), better numerical conditioning, and lower computational cost compared to techniques based on the Hessian matrix. The numerical stability is assured by the use of robust LS linear system solvers, operating directly on the input data of each layer. Experimental results obtained in three problems are described, which confirm the effectiveness of the new method.

In order to solve the problem that the traditional long-term high-speed traffic forecasting algorithm is affected by the approximation ability of the function and easy to fall into the local mass value, we wrote a multivariate-based highway traffic forecasting algorithm scaling and convolutional networks. Because the feedforward wavelet neural network algorithm predicts the short-term traffic flow in different areas, it is necessary to examine the ability to predict the difference between different models. From the standard feedforward wavelet neural network algorithm using global optimization capabilities, we improve the wolf pack algorithm, improve the search accuracy of the algorithm, get the best solution of the estimated value of the work according to the search results when completing the research objectives, and get the ability to predict the work of the model. Feedforward neural network algorithm: we develop and obtain the best short-term high-speed traffic forecast values. The results are as follows: after using the author’s algorithm, the processing time increases by 1.5 seconds, but the average percentage of errors decreases by more than 50%, in fact the error and the root mean square error decreased by about 30%, and the smoothing coefficient increased by about 1%. The prediction of the author’s algorithm for short-term high-speed traffic is better than the wavelet neural network prediction algorithm, and the prediction accuracy and stability of the author’s algorithm are higher.

A new fast learning algorithm for a multilayer feedforward neural network is presented. The new algorithm, based on an innovative variable step size and gradient averaging, has excellent properties to overcome major drawbacks of the backpropagation algorithm and shows high convergence speed. The improved performance of the new algorithm in comparison with other algorithms is verified via computer simulations.

In this paper, the terminal attractor based back propagation learning algorithms for feedforward neural networks are examined. Through a rigorous mathematical analysis, a condition to guarantee the convergence of the algorithms is given. A simulation study is presented to demonstrate the effectiveness of the analysis.

The convergence of a class of stochastic pseudo-gradient algorithms driven by correlated data sequences is considered in this paper. The obtained results are applied to a learning algorithm for feedforward neural networks and sufficient conditions for its convergence are determined.

Recent study, the extended Kalman filter (EKF) is applied to the learning algorithm for a feedforward neural network and it is shown that the EKF based learning algorithm has good convergence performances. However, the feedforward neural network could not perform a dynamical signal processing such as time series pattern recognition. On the other hand, the recurrent neural network (RNN) could have dynamical characteristics because the RNN has feedback connections with time delay in the network. The EKF based learning algorithms for the RNN are also reported and some training properties became clear. In this paper, RNN training based on the EKF are applied to the time series signal processing. The connection weights of the RNN can be modified by using the filtering algorithm based on the EKF. Simulation results show that the RNN based on the EKF has good performances of the training for time series patterns generated from sine functions and for some pulse patterns.

Fractional calculus has been adopted in the modelling of many scientific processes and systems. Due to the inherent feature of long term memory of fractional derivatives, it has been used in the learning process of neural networks. A fractional order derivative based back propagation learning algorithm in neural networks is proposed in this paper. Specifically, Riemann-Liouville (R-L), Caputo (C) and Caputo Fabrizio (CF) fractional Derivative based on the back propagation algorithms in a three layer feed-forward neural network employed. To get a faster learning rate without oscillation, momentum factor is incorporated. The effect of fractional order and momentum factor is investigated and compared. The performance of these fractional derivatives based algorithms with integer derivatives based algorithm in terms of mean square error (MSE), particularly the salary based on years of experience is predicted. Results demonstrate that fractional derivative based learning algorithms outperform the integer derivatives.

In the paper, the research present situation and development in the wavelet neural network model and a novel learning algorithm for wavelet neural network based on extended Kalman filter are discussed. Based on combining the exceptional property of localization of the wavelet transform and characteristics of self-learning of neural networks, the non-line time series model and network architecture model which combines affine transform with revolving transform is discussed .A novel learning algorithm for wavelet neural network based on extended Kalman filter is proposed to predict the deformation of structure. In comparison with the WNN algorithm, the EKF learning algorithm has improved convergence and can provide much more accuracy learning results.

Presents an algorithm of dynamic node creation and weights learning for a hybrid feedforward neural network (HFNN) which consists of a linear model and a multilayer neural network. The algorithm is only based on linear least squares, and no iterative learning process is needed. According to the demand of model precision, the algorithm determines the best weights of network and the minimal number of hidden nodes automatically. Compared with the well-known backpropagation network, simulation results show that the new learning algorithm for the HFNN is efficient in model precision, rate of convergence and generalization ability.

Up to now, rich and varied information, such as networks, multimedia information, especially images and visual information, has become an important part of information retrieval, in which video and image information has been an important basis. In recent years, an effective learning algorithm for standard feed-forward neural networks (FNNs), which can be used classifier and called random weights networks (RWN), has been extensively studied. This paper addresses the image classification algorithms using the algorithm. A new algorithm of image classification based on the RWN and principle component analysis (PCA) is proposed. The proposed algorithm includes significant improvements in classification rate, and the extensive experiments are performed using challenging databases. Compared with some traditional approaches, the new method has superior performances on both classification rate and running time.

Extreme learning machine (ELM) is an easy-to use and effective learning algorithm of single-hidden layer feed-forward neural networks (SLFNs). The classical learning algorithm in neural network, e. g. backpropagation, requires setting several user-defined parameters and may get into local minimum. However, ELM only requires setting the number of hidden neurons and the activation function. It does not require adjusting the input weights and hidden layer biases during the implementation of the algorithm, and it produces only one optimal solution. Therefore, ELM has the advantages of fast learning speed and good generalization performance. In this paper, ELM is introduced in predicting reservoir permeability. By comparing to SVM, we analyze its feasibility and advantages in reservoir permeability prediction. The experimental results show that ELM has similar accuracy compared to SVR, but it has obvious advantages in parameter selection and learning speed.

Linear programming algorithm for neural networks

A fast and robust learning algorithm for feedforward neural networks