Identification of MIMO systems using radial basis function networks with hybrid learning algorithm

This paper examines the architecture of back-propagation neural networks used as approximators by addressing the interrelationship between the number of training pairs and the number of input, output, and hidden layer nodes required for a good approximation. It concentrates on nets with an input layer, one hidden layer, and one output layer. It shows that many of the currently proposed schemes for selecting network architecture for such nets are deficient. It demonstrates in numerous examples that overdetermined neural networks tend to give good approximations over a region of interest, while underdetermined networks give approximations which can satisfy the training pairs but may give poor approximations over that region of interest. A scheme is presented that adjusts the number of hidden layer nodes in a neural network so as to give an overdetermined approximation. The advantages and disadvantages of using multiple output nodes are discussed. Guidelines for selecting the number of output nodes are presented.

The paper demonstrates that radial basis function network (RBFN) with adaptive centers and width can be used effectively for identification of nonlinear dynamic system. The proposed RBFN is trained by hybrid learning algorithm, which uses conjugate gradient optimization algorithm to obtain the center and width of each radial basis function and the least squares method to obtain the weights. To avoid capturing a local optimum, regularization error energy function is used and the centers of basis functions are initialized using a fuzzy C-means clustering method. Simulation results reveal that the identification schemes based on RBFN gives considerably better performance and show faster learning in comparison to previous methods

Long short-term memory (LSTM) is widely applied in both academic and industrial fields. However, there is no reliable criterion on selecting hyperparameters of LSTM. Currently, although some widely used classic methods such as random search and grid search have obtained success to some extent, the problems in local optimum and convergence still exist. In this research, we propose to use grey wolf optimizer (GWO) to search for the hyperparameters of LSTM. Through the method, the superiority of metaheuristic in global optimization and the strength of LSTM in predicting are combined. In this model, number of hidden layer nodes and learning rate of LSTM are set as preys, and grey wolf pack has a simple but efficient mechanism to search for the optimal hyperparameters. The benchmark tests on several basic functions were utilized, and the results were verified by a comparative study with random search, support vector regression and several other regression methods. Specifically, we applied this algorithm in predicting the degradation trend of the airborne fuel pump. As a result, the ergodicity and convergence of the algorithm are proved mathematically based on Markov processes theory. The benchmark tests show that the GWO-LSTM model holds for predicting data with low overall slope and high partial fluctuation. The application in airborne fuel pump shows that, trained by dataset with 5700 points, the proposed model could predict sequence of 300 points with root mean square error 0.617 after 30 iterations of optimizing, which is 2.512 previously. The result further demonstrates that the proposed algorithm is applicable to make prediction with high accuracy. Overall, the effectiveness of GWO-LSTM model is verified from theoretical proof to benchmark tests and then to actual product application.

In this paper, a soft computing technique is proposed to select an initial structure of neuro-fuzzy networks (NFNs) for the noise data with outlier regression. That is, the proposed soft computing technique is fusion of neural network, fuzzy logic and support vector regression (SVR). Because the SVR approach is equivalent to solving a linear constrained quadratic programming problem under the fixed structure of SVR, the number of hidden nodes and adjustable parameters are easily obtained. That is, the SVR approach is used to obtain the initial structures of the NFNs in the noise data with outlier. Then, an annealing robust learning algorithm (ARLA) uses as the learning algorithm of NFNs. It is applied to adjust the parameters of promise parts and the fuzzy singleton of consequence parts in the NFNs. At the same time, the proposed method is applied to approximate a nonlinear model with outliers. Simulation results are provided to show the validity and applicability of the proposed soft computing technique.

This article addresses the robust estimation of the output layer linear parameters in a radial basis function network (RBFN). A prominent method used to estimate the output layer parameters in an RBFN with the predetermined hidden layer parameters is the least-squares estimation, which is the maximum-likelihood (ML) solution in the specific case of the Gaussian noise. We highlight the connection between the ML estimation and minimizing the Kullback-Leibler (KL) divergence between the actual noise distribution and the assumed Gaussian noise. Based on this connection, a method is proposed using a variant of a generalized KL divergence, which is known to be more robust to outliers in the pattern recognition and machine-learning problems. The proposed approach produces a surrogate-likelihood function, which is robust in the sense that it is adaptive to a broader class of noise distributions. Several signal processing experiments are conducted using artificially generated and real-world data. It is shown that in all cases, the proposed adaptive learning algorithm outperforms the standard approaches in terms of mean-squared error (MSE). Using the relative increase in the MSE for different noise conditions, we compare the robustness of our proposed algorithm with the existing methods for robust RBFN training and show that our method results in overall improvement in terms of absolute MSE values and consistency.

In this paper, based on least trimmed squares-support vector regression (LTS-SVR), a robust radial basis function network (RRBFN) is proposed in the modeling problem to deal with training data sets that may contain outliers and noises. There are two stages in the proposed RRBFN approach. In stage I, outliers and large noises will be trimmed via the LTS-SVR procedure so that the influences of outliers and large noises can be reduced. In other words, with the use of the LTS-SVR procedure, the proposed approach can obtain an appropriate initial structure for the RRBFN to avoid possible overfitting phenomena. It can also be found that with this procedure it may result in a fast convergence speed. After stage I, the remaining data of the training data set are directly used to adjust the parameters of the RRBFN through the gradient-descent kind of learning algorithm. Hence, it does not need to take extra time to compute the weights of the robust cost function like that considered in the M-estimate kind of approaches. From our simulation results, the performances of the proposed system are superior to that of using the annealing robust RBFN and that of using the Wilcoxon generalized RBFN when training data contain outliers and noises.

Generalized [formula omitted]-sparse algorithms for constructing fault tolerant RBF networks

Three learning phases for radial-basis-function networks

This research work focuses on the development of meta-cognitive sequential learning algorithms in Radial Basis Function (RBF) network classiers, and their application to the early diagnosis of neurodegenerative diseases. The important issues in existing sequential learning algorithms are proper selection of training samples, nding minimal network structure and selection of an appropriate learning strategy. In addition, the random sequence of sample arrival inuences the performance signicantly. It has been reported in human learning that best learning strategies employ meta-cognition (metacognition means cognition about cognition) to address fundamental problems of what-tolearn, when-to-learn and how-to-learn. This thesis develops such meta-cognitive sequential learning algorithms in RBF network for classication problems. We call a RBF network employing meta-cognitive algorithm as `meta-cognitive RBF network' (McRBFN). McRBFN is developed based on Nelson and Narens model of meta-cognition for human learning. Accordingly, McRBFN has two components, namely cognitive and meta-cognitive components. A RBF network with evolving structure is the cognitive component and a self-regulatory learning mechanism is its meta-cognitive component. List of Figures 1.1 Nelson and Narens model of meta-cognition . . . . . . . . . . . . . . . .

A hybrid algorithm for determining Radial Basis Function (RBF) networks is proposed. Evolutionary learning is applied to the non-linear problem of determining RBF network architecture (number of hidden layer nodes, basis function centres and widths) in conjunction with supervised gradient-based learning for tuning connection weights. A direct encoding of RBF hidden layer node basis function centres and widths is employed. The genetic operators utilised are adapted from those used in recent work on evolution of fuzzy inference systems. A parsimonious allocation of training sets and training epochs to evaluation of candidate networks during evolution is proposed in order to accelerate the learning process.

Radial basis function (RBF) network can be viewed as a fuzzy rule base with specified membership functions and fuzzy inference operations. However, there is a trade-off between the approximation performance of RBF network and the number of hidden neurons. To tackle this problem, this paper proposes a dynamic RBF (DRBF) network with a constructive learning. This DRBF network not only can create the new hidden neurons, but also can prune the insignificant hidden neurons. Then, an adaptive dynamic RBF fuzzy neural control (ADRFNC) system, including a neural controller and a saturation compensator, is developed. The neural controller uses a DRBF network to on-line mimic an ideal controller and the saturation compensator is designed to dispel the approximation error introduced by the neural controller. Finally, the proposed ADRFNC system is applied to a chaotic circuit and a DC motor control system. Simulation and experimental results show the proposed ADRFNC system can achieve a favorable tracking performance when the controller’s parameters have been learned and the network structure has been constructed by the proposed learning algorithm.

In hydrological research, accurate rainfall data is the primary subject for the minimization of potential loss of life and property that is mainly caused by floods. However, there is a difficulty in getting precise rainfall data for poorly gauged locations, especially in mountainous areas. Weather radar instruments can be the remedy accompanied by some errors. And, these errors should be removed before the implementation of this product. This paper presents the results of the research on radar rainfall estimate errors with support vector regression (SVR) method using the observed rain gauge data. The paper depicts the methodological base of the algorithm that covers additive and multiplicative corrections and the results of practical implementations considering the locations of gauge measurements. The preliminary results show that the SVR has a location-oriented performance. The multiplicative and additive correction factors show decreasing and polynomial trends respectively, as the distance from the radar location increase. Another particular outcome is that the SVR shows better results for the stations located in the mid-range (mainly for 40-60 km) contrary to the nearest ones. Since the systematic error in the radar data is nonlinear, the SVR method would show a promising result with a combination of other optimization techniques.

The characteristics of hydraulic automatic gage control System(AGC) of cold rolling mill is analyzed. the principle of identification using Radial Basis Function(RBF) network is introduced. The RBF network structure, train algorithm are given in detail. Finally, the nonlinear model of HAGC is established with RBF network. To improving the accuracy of identification. the input and output data is filtered with wavelet denoising technique. The experimental results indicate that the combination of wavelet denoising and RBF network can provide an efficient method for system identification.

Stochastic configuration network (SCN) is a powerful prediction model whose performance is significantly influenced by the configuration of the network parameters. To improve the prediction accuracy of the network, a cooperative stochastic configuration network (CSCN) based on a novel differential evolutionary sparrow search algorithm (DESSA), termed as DESSA-CSCN, is proposed. In the CSCN, the number of hidden layer nodes is adaptively adjusted according to the number of iterations, and the parameters of hidden nodes are cooperatively optimized by using a population-based metaheuristic algorithm. A sparse matrix is introduced to mitigate parameter overfitting caused by the increased number of hidden layer nodes. During parameter optimization, the fitness function is constructed by using the supervision mechanism of the SCN, and the DESSA is utilized as the metaheuristic algorithm to update the weights and biases. In order to verify the effectiveness of the DESSA-CSCN, several simulation experiments have been conducted. The performance of the DESSA is evaluated by the CEC2017 test suit, and the simulation results show that the DESSA exhibits better convergence accuracy and can jump out of local optima more effectively than other algorithms. The performance of the DESSA-CSCN is evaluated by 4 datasets from KEEL, and the simulation results indicate that the DESSA-CSCN achieves better prediction accuracy and faster prediction speed than other models.

This article explores the use of single trial EEG signals to predict the voluntary movements of single hand and two hands. During single-hand movements, three kinds of task, grasping, releasing, and holding were considered. The tasks considered during two-hand movements are left and right grasping, left and right releasing, and holding. The subject performs the tasks spontaneously without waiting for and responding to any external cues. In addition, a neural adaptive noise canceller is developed that accomplishes eye blinks suppression. The neural adaptive filter is here implemented by means of a three-layer feed-forward neural network. The feature vectors are formed from the three channels (Fz, C3, and F3). We employ the multilayer perceptron (MLP) with back-propagation learning algorithm and Radial Basis Function (RBF) network with stochastic gradient learning rule for discriminating different patterns of the EEG signals. In the classical approach to RBF and MLP network implementation, the number of hidden units is predetermined. It, usually, results in too many hidden units. To overcome this drawback, we develop an enhanced resource-allocating network (RAN) for discriminating the EEG patterns. These networks start with no hidden units and grow by allocating new hidden units based on the novelty in the EEG signals, which arrive sequentially. The results of this analysis show that the neural networks would be able to detect the movements of a single hand and two hands with an average classification accuracy of 98.82% and 96.40%, respectively. Moreover, the RAN provides a reduction in the training epochs as compared to the MLP and RBF networks. This work represents a promising approach to control prosthesis device.