Configuring radial basis function network using fractal scaling process with application to chaotic time series prediction

Introduces a novel algorithm for adjusting the structure of a Radial Basis Function Network (RBFN). It has been shown that Radial Basis Function Networks (RBFNs) are capable of universal approximation. This suggests the possibility of using these neural models to identify the chaotic systems. However, when one deals with the observable from some process (e.g., biological systems) whose mathematical formulation and the total number of variables may not be known exactly, structure construction and adjustment of the artificial neural network remain as an open question. In this work, the authors introduce the chaotic dynamics in the structure construction of the RBFN. It can be seen that the radial basis functions establish a partition of the embedding space into regions in each of which it is possible to approximate the dynamics with a basis function. On account of the fact that the attractor of the chaotic systems is a fractal object, the authors use the fractal scaling process for partitioning the strange attractor into self-similar structures. Accordingly, the number of input variables, the number of basis function, and the scaling parameter of the basis function can be specified by the fractal scaling process. This work represents a promising approach to the modeling and prediction of chaotic time series.

Radial Basis Function (RBF) Network is popularly used for solving pattern recognition problems. The training of RBF Network is faster compared to multi layer perceptron using error backpropagation. However, the RBF Network uses the pseudo inverse matrix to calculate weights from the hidden layer to the output layer. Thus calculation cost increases when the number of data and the number of hidden units increase. In addition, in RBF Network the decision of optimum number of hidden units is difficult. It is also more prone to overtraining, needing repeated train and test cycles to ascertain a proper number of the Network hidden units, so that generalization performance is good. In this work, we propose a technique to set up RBF network parameters which is fast, as well as the number of hidden units are automatically determined. We start with training a Self-Organizing Maps (SOM), which is a unsupervised training, though our samples are labeled. SOM can find the distribution of data in multidimensional space, and map it on a two dimensional display. The results of SOM network is used to calculate the RBF parameters. It is shown by experiments that using the proposed method, RBF network parameters can be determined much faster compared to existing technique. Moreover, the recognition rate for the test data was higher, showing better generalization performance.

We propose a differential radial basis function (RBF) network termed RBF-DiffNet—whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF—to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction in the prediction error over the baseline RBF network (e.g., 41% reduction in the root mean squared scaled error on the M5 dataset, and 53% reduction in the mean absolute error on the logistic map); RBF-DiffNet also shows a comparable performance to the LSTM ensemble but requires 99% less computational time. Our proposed network consequently enables more accurate predictions—in the presence of observational noise—in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.

Three learning phases for radial-basis-function networks

A new learning method of radial basis function (RBF) network based on fuzzy C-means (FCM) clustering and ant colony optimization (ACO) is presented in the paper. Generally, the crucial problem of learning RBF network is the selection of the unit number, the centers and widths of the Gaussian radial basis function in hidden layer, and the weights between hidden layer and output layer. In this work, the centers are determined by FCM clustering, and the weights are determined by least mean squares (LMS) method. In addition, ACO is introduced to optimize two important parameters of FCM including the weighting exponent which impacts its performance, and the clustering number that is equal to the hidden unit number which impacts the generalization of RBF network, by minimizing an objective function integrated the generalization error with the hidden unit number of RBF network. Simulation results of identifying a nonlinear system illustrate the effectiveness of designing the RBF network with smaller structure but stronger generalization ability, comparing with K-means and orthogonal least squares (OLS) based learning methods.

In this paper, we construct predictive model of chaotic time series via Volterra series and characteristics of Hopfield neural network, which possess fast and parallel processing. We also induce weight formula of Hopfield neural network. It is shown through experiments that this model is valid.

View-based object representations using RBF networks

Radial basis function (RBF) network can be viewed as a fuzzy rule base with specified membership functions and fuzzy inference operations. However, there is a trade-off between the approximation performance of RBF network and the number of hidden neurons. To tackle this problem, this paper proposes a dynamic RBF (DRBF) network with a constructive learning. This DRBF network not only can create the new hidden neurons, but also can prune the insignificant hidden neurons. Then, an adaptive dynamic RBF fuzzy neural control (ADRFNC) system, including a neural controller and a saturation compensator, is developed. The neural controller uses a DRBF network to on-line mimic an ideal controller and the saturation compensator is designed to dispel the approximation error introduced by the neural controller. Finally, the proposed ADRFNC system is applied to a chaotic circuit and a DC motor control system. Simulation and experimental results show the proposed ADRFNC system can achieve a favorable tracking performance when the controller’s parameters have been learned and the network structure has been constructed by the proposed learning algorithm.

The radial basis function (RBF) network is a type of neural network that uses a radial basis function as its activation function (Ou, Oyang & Chen, 2005). Because of the better approximation capabilities, simpler network structure and faster learning speed, the RBF networks have attracted considerable attention in many science and engineering field. Horng (2010) used the RBF for multiple classifications of supraspinatus ultrasonic images. Korurek & Dogan (2010) used the RBF networks for ECG beat classifications. Wu, Warwick, Jonathan, Burgess, Pan & Aziz (2010) applied the RBF networks for prediction of Parkinson’s disease tremor onset. Feng & Chou (2011) use the RBF network for prediction of the financial time series data. In spite of the fact that the RBF network can effectively be applied, however, the number of neurons in the hidden layer of RBF network always affects the network complexity and the generalizing capabilities of the network. If the number of neurons of the hidden layer is insufficient, the learning of RBF network fails to correct convergence, however, the neuron number is too high, the resulting over-learning situation may occur. Furthermore, the position of center of the each neuron of hidden layer and the spread parameter of its activation function also affect the network performance considerably. The determination of three parameters that are the number of neuron, the center position of each neuron and its spread parameter of activation function in the hidden layer is very important.

This article explores the use of single trial EEG signals to predict the voluntary movements of single hand and two hands. During single-hand movements, three kinds of task, grasping, releasing, and holding were considered. The tasks considered during two-hand movements are left and right grasping, left and right releasing, and holding. The subject performs the tasks spontaneously without waiting for and responding to any external cues. In addition, a neural adaptive noise canceller is developed that accomplishes eye blinks suppression. The neural adaptive filter is here implemented by means of a three-layer feed-forward neural network. The feature vectors are formed from the three channels (Fz, C3, and F3). We employ the multilayer perceptron (MLP) with back-propagation learning algorithm and Radial Basis Function (RBF) network with stochastic gradient learning rule for discriminating different patterns of the EEG signals. In the classical approach to RBF and MLP network implementation, the number of hidden units is predetermined. It, usually, results in too many hidden units. To overcome this drawback, we develop an enhanced resource-allocating network (RAN) for discriminating the EEG patterns. These networks start with no hidden units and grow by allocating new hidden units based on the novelty in the EEG signals, which arrive sequentially. The results of this analysis show that the neural networks would be able to detect the movements of a single hand and two hands with an average classification accuracy of 98.82% and 96.40%, respectively. Moreover, the RAN provides a reduction in the training epochs as compared to the MLP and RBF networks. This work represents a promising approach to control prosthesis device.

In the radial basis function (RBF) network, selection of width parameter of the basis functions is considered. The effect of the width parameter on multivariable discrete function approximation is analysed in the Fourier domain and based on this, an appropriate selection method for the width parameter is presented. The method basically takes the orthogonal multiresolutional decomposition by wavelet transform of the function subject to approximation and to each decomposed component, a separate RBF network having a suitable width parameter matching to the appropriate resolution level, is employed. Each outcome from these multiresolutional RBF networks is added to that from the others and a final approximation result is obtained. A marked enhancement of approximation is achieved which implies an enhancement of the generalisation capability of the network performing robust estimation at the same time.

The prediction accuracy and generalization ability of neural/neurofuzzy models for chaotic time series prediction highly depends on employed network model as well as learning algorithm. In this study, several neural and neurofuzzy models with different learning algorithms are examined for prediction of several benchmark chaotic systems and time series. The prediction performance of locally linear neurofuzzy models with recently developed Locally Linear Model Tree (LoLiMoT) learning algorithm is compared with that of Radial Basis Function (RBF) neural network with Orthogonal Least Squares (OLS) learning algorithm, MultiLayer Perceptron neural network with error back-propagation learning algorithm, and Adaptive Network based Fuzzy Inference System. Particularly, cross validation techniques based on the evaluation of error indices on multiple validation sets is utilized to optimize the number of neurons and to prevent over fitting in the incremental learning algorithms. To make a fair comparison between neural and neurofuzzy models, they are compared at their best structure based on their prediction accuracy, generalization, and computational complexity. The experiments are basically designed to analyze the generalization capability and accuracy of the learning techniques when dealing with limited number of training samples from deterministic chaotic time series, but the effect of noise on the performance of the techniques is also considered. Various chaotic systems and time series including Lorenz system, Mackey-Glass chaotic equation, Henon map, AE geomagnetic activity index, and sunspot numbers are examined as case studies. The obtained results indicate the superior performance of incremental learning algorithms and their respective networks, such as, OLS for RBF network and LoLiMoT for locally linear neurofuzzy model.

In order to forecast missile storage reliability better, the paper researched a forecasting method based on neural network which is with the ability of actualizing multi-nonlinear mapping from input to output, and discussed steps of forecasting based on back propagation (BP) network and radial basis function (RBF) network respectively. At last, the storage reliability of one type ship-to-ship missile is forecasted based on BP network and RBF network respectively. The results show that both of the BP and RBF are suitable for Missile storage reliability forecasting, and the precision of the train goal is better by using RBF network. RBF network is more suitable for dealing with this problem.

The present paper proposes a novel non-intrusive model reduction approach based on Proper Orthogonal Decomposition (POD), the Discrete Empirical Interpolation Method (DEIM) and Radial Basis Function (RBF) networks to efficiently predict production of oil and gas reservoirs. Provided a representative set of training reservoir scenarios, either POD or DEIM allows for effectively projecting input parameters (e.g., permeability, porosity), states (e.g., pressure, saturations) and outputs (e.g., well production curves) into a much lower dimension that retains the main features contained in the simulation system. In this work, these projections are applied across multiple levels to be able to collapse a large number of spatio-temporal correlations. It is observed that these projections can be effectively performed at a large extent regardless of the underlying geological complexity and operational constraints associated with the reservoir model. The RBF network provides a powerful means for developing learning functions from input-output relationships described by the reservoir dynamics entailed by multiple combinations of inputs and controls. In order to achieve a high degree of predictability from the resulting reduced model, the RBF network exploits locality by a means of Gaussian basis functions that are maximal at the sampled point and decrease monotonically with distance. Compared to multilayer perceptron networks (i.e., traditional artificial neural networks) RBF networks require less training and are less sensitive to the presence of noise in the data. In this regard, POD or DEIM acts as a data filter that additionally aids at designing a more compact RBF network representation suitable for targeting fast reservoir predictions. Numerical results show significant accelerations with respect to running the original simulation model on a set of field-motivated cases.

This paper proposes a generic criterion that defines the optimum number of basis functions for radial basis function (RBF) neural networks. The generalization performance of an RBF network relates to its prediction capability on independent test data. This performance gives a measure of the quality of the chosen model. An RBF network with an overly restricted basis gives poor predictions on new data, since the model has too little flexibility. By contrast, an RBF network with too many basis functions also gives poor generalization performance since it is too flexible and fits too much of the noise on the training data. Bias and variance are complementary quantities, and it is necessary to assign the number of basis function optimally in order to achieve the best compromise between them. In this paper we use Stein's unbiased risk estimator to derive an analytical criterion for assigning the appropriate number of basis functions. Two cases of known and unknown noise have been considered and the efficacy of this criterion in both situations is illustrated experimentally. The paper also shows an empirical comparison between this method and two well known classical methods, cross validation and the Bayesian information criterion, BIC.