Efficient Training Algorithms for the Probabilistic RBF Network

We propose a differential radial basis function (RBF) network termed RBF-DiffNet—whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF—to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction in the prediction error over the baseline RBF network (e.g., 41% reduction in the root mean squared scaled error on the M5 dataset, and 53% reduction in the mean absolute error on the logistic map); RBF-DiffNet also shows a comparable performance to the LSTM ensemble but requires 99% less computational time. Our proposed network consequently enables more accurate predictions—in the presence of observational noise—in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.

The radial basis function (RBF) network is a type of neural network that uses a radial basis function as its activation function (Ou, Oyang & Chen, 2005). Because of the better approximation capabilities, simpler network structure and faster learning speed, the RBF networks have attracted considerable attention in many science and engineering field. Horng (2010) used the RBF for multiple classifications of supraspinatus ultrasonic images. Korurek & Dogan (2010) used the RBF networks for ECG beat classifications. Wu, Warwick, Jonathan, Burgess, Pan & Aziz (2010) applied the RBF networks for prediction of Parkinson’s disease tremor onset. Feng & Chou (2011) use the RBF network for prediction of the financial time series data. In spite of the fact that the RBF network can effectively be applied, however, the number of neurons in the hidden layer of RBF network always affects the network complexity and the generalizing capabilities of the network. If the number of neurons of the hidden layer is insufficient, the learning of RBF network fails to correct convergence, however, the neuron number is too high, the resulting over-learning situation may occur. Furthermore, the position of center of the each neuron of hidden layer and the spread parameter of its activation function also affect the network performance considerably. The determination of three parameters that are the number of neuron, the center position of each neuron and its spread parameter of activation function in the hidden layer is very important.

This research work focuses on the development of meta-cognitive sequential learning algorithms in Radial Basis Function (RBF) network classiers, and their application to the early diagnosis of neurodegenerative diseases. The important issues in existing sequential learning algorithms are proper selection of training samples, nding minimal network structure and selection of an appropriate learning strategy. In addition, the random sequence of sample arrival inuences the performance signicantly. It has been reported in human learning that best learning strategies employ meta-cognition (metacognition means cognition about cognition) to address fundamental problems of what-tolearn, when-to-learn and how-to-learn. This thesis develops such meta-cognitive sequential learning algorithms in RBF network for classication problems. We call a RBF network employing meta-cognitive algorithm as `meta-cognitive RBF network' (McRBFN). McRBFN is developed based on Nelson and Narens model of meta-cognition for human learning. Accordingly, McRBFN has two components, namely cognitive and meta-cognitive components. A RBF network with evolving structure is the cognitive component and a self-regulatory learning mechanism is its meta-cognitive component. List of Figures 1.1 Nelson and Narens model of meta-cognition . . . . . . . . . . . . . . . .

Generalized [formula omitted]-sparse algorithms for constructing fault tolerant RBF networks

This paper presents a method for analog circuit fault diagnosis by using neural networks. This method exploits a DC approach for constructing a dictionary in fault diagnosis using the neural network's classification capability. Also, Radial Basis Function (RBF) and backward error propagation (BEP) networks are considered and compared for analog fault diagnosis. The primary focus of the paper is to provide robust diagnosis using a mechanism to deal with the problem of component tolerance and reduce testing time. Simulation results show that the radial basis function network with reasonable dimension has double precision in fault classification but its classification is local, while the backward error propagation network with reasonable dimension has single precision in fault classification but its classification is global.

In the radial basis function (RBF) network, selection of width parameter of the basis functions is considered. The effect of the width parameter on multivariable discrete function approximation is analysed in the Fourier domain and based on this, an appropriate selection method for the width parameter is presented. The method basically takes the orthogonal multiresolutional decomposition by wavelet transform of the function subject to approximation and to each decomposed component, a separate RBF network having a suitable width parameter matching to the appropriate resolution level, is employed. Each outcome from these multiresolutional RBF networks is added to that from the others and a final approximation result is obtained. A marked enhancement of approximation is achieved which implies an enhancement of the generalisation capability of the network performing robust estimation at the same time.

An electronic nose and modular radial basis function network classifiers for recognizing multiple fragrant materials

A new approach is presented to enhance fuzzy modeling using multidimensional fuzzy sets directly in the fuzzy model in place of decomposed fuzzy sets by projection. The antecedent fuzzy sets in the form of multivariable functions are represented in continuous form. This is accomplished by using a multi input and multi output neural network, which is in particular radial basis functions (RBF) network providing the required multivariable function approximation properties in a fuzzy model. The inputs of the network are fuzzy model inputs, outputs are the membership function values as to the multivariable fuzzy sets involved in the model. Thus, each output is restricted to one multivariable fuzzy set. The RBF network is trained according to the multidimensional fuzzy sets which are identified after fuzzy clustering of the data. Based on this, the linear model parameters are determined by the method of least squares in a straightforward manner. The transparency issues can be handled by means of projection process to have information about the shape and locations of the membership functions pertinent to each variable at the input. The accurate fuzzy modeling having been thus guaranteed the information on the linear model parameters are used to establish the multivariable fuzzy rules with their respective validity regions. The redundancy of the number of fuzzy sets can be circumvented by classical cluster merging algorithms available. In this approach, the fuzzy rules are determined from the local linear models, however in contrast with the conventional fuzzy modeling, the model outputs are determined by the multivariable fuzzy sets. To obtain regular sets, RBF network is trained by means of orthogonal least squares (OLS) method so that at the same time the convergence issues of general neural network training is eliminated.

Using general regression with local tuning for learning mixture models from incomplete data sets

This paper demonstrates that a radial basis function (RBF) network can be used to estimate or predict future head-positions from samples of the current and past positions. The results, shown for a network trained and tested over a recorded head position time series, imply that the technique is a viable method for reducing latency in the head-slaved computer generated imagery of a flight simulator. The procedure for building the network-based estimator is straightforward, using example-based learning to associate certain transformations on the current position to the value of position at a later time, The success of the approach is found to be dependent on the degree to which the training data represent the full range of motion found in the simulator. A single-output network is shown to accurately estimate rotational position (azimuth) over all motion types of interest, a range from low through high acceleration motion. It is also shown that performance can be improved by adding data variations to the network training set. The network's accuracy is evaluated for several different prediction intervals, from 150 msec through 350 msec, with the best results found to be at 200 msec and below. Finally, the results for a single-output network are extended to a two-output network, demonstrating prediction of both elements of the look-angle (azimuth, elevation) in one RBF network.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In this paper, we propose a novel complex-valued RBF networks approach for dynamic behavioral modeling of nonlinear power amplifier with memory. The complex recursive orthogonal least squares (ROLS) algorithm is applied to train complex radial basis function (RBF) network when modeling, so as to save large memory and computational efforts. Using the information available from the trained network with complex ROLS algorithm, the effective centers of the network can be obtained by adopting backward selection algorithm, which achieve acceptable accuracy with significant reduction of network structure. The model has been validated using measured and simulated data. The results of simulation and experimentation show that this complex RBF network model requires a significantly reduced complexity in the analysis and training procedures, when driven with WCDMA signals, than previously published neural-network-based PA models.

Approximation methods such as response surface method and artificial neural network (ANN) method are widely used to alleviate the computation costs in structural reliability analysis. However most of the ANN methods proposed in the literature suffer various drawbacks such as poor choice of parameter setting, poor generalization and local minimum. In this study, a support vector machine-based radial basis function (RBF) network method is proposed, in which the improved RBF model is used to approximate the limit state function and then is connected to a reliability method to estimate failure probability. Since the learning algorithm of RBF network is replaced by the support vector algorithm, the advantage of the latter, such as good generalization ability and global optimization are propagated to the former, thus the inherent drawback of RBF network can be defeated. Numerical examples are given to demonstrate the applicability of the improved RBF network method in structural reliability analysis, as well as to illustrate the validity and effectiveness of the proposed method.

In this paper, we propose a new Expectation-Maximization (EM) algorithm which speeds up the training of feedforward networks with local activation functions such as the Radial Basis Function (RBF) network. The core of the conventional EM algorithm for supervised learning of feedforward networks consists of decomposing the observations into their individual output units and then estimating the parameters of each unit separately. In previously proposed approaches, at eac h E-step the residual is decomposed equally among the units or proportionally to the weights of the output layer. However, this approach tends to slow down the training of networks with local activation units. To overcome this drawback in this paper we use a new E-step which applies a soft decomposition of the residual among the units. Inparticular, the residual is decomposed according to the probability of each RBF unit given each input-output pattern. It is shown that this variant not only speeds up the training in comparison with other EM-type algorithms, but also provides better results than a global gradient-descent technique since it has the capability of avoiding some unwanted minima of the cost function.

A segmentation procedure using a radial basis function network (RBFN), coupled with an active contour (AC) model based on a cubic splines formulation is presented for the detection of the gray-white matter boundary in axial MMRI (T1, T2 and PD). A RBFN classifier has been previously introduced for MMRI segmentation, with good generalization at a rate of 10% misclassification over white and gray matter pixels on the validation set. The coupled RBFN and AC model system incorporates the posterior probability estimation map into the AC energy term as a restriction force. The RBFN output is also employed to provide an initial contour for the AC. Furthermore, an adaptation strategy for the network weights, guided by a feedback from the contour model adjustment at each iteration, is described. In order to compare the algorithm's performance, the segmentations using the adaptive, as well as the non-adaptive schemes were computed. It was observed that the major differences are located around deep circonvolutions, where the result of the adaptive process is superior than that obtained with the non-adaptive scheme, even in moderate noise conditions. In summary, the RBFN provides a good initial contour for the AC, the coupling of both processes keeps the final contour within the desired region and the adaptive strategy enhances the contour location.

Estimation for the number of components in a mixture model using stepwise split-and-merge EM algorithm