Correntropy Induced Metric Penalized Sparse RLS Algorithm to Improve Adaptive System Identification

Recursive least squares (RLS)-based algorithms are a class of fast online training algorithms for feedforward multilayered neural networks (FMNNs). Though the standard RLS algorithm has an implicit weight decay term in its energy function, the weight decay effect decreases linearly as the number of learning epochs increases, thus rendering a diminishing weight decay effect as training progresses. In this paper, we derive two modified RLS algorithms to tackle this problem. In the first algorithm, namely, the true weight decay RLS (TWDRLS) algorithm, we consider a modified energy function whereby the weight decay effect remains constant, irrespective of the number of learning epochs. The second version, the input perturbation RLS (IPRLS) algorithm, is derived by requiring robustness in its prediction performance to input perturbations. Simulation results show that both algorithms improve the generalization capability of the trained network.

This paper proposes a recursive least-squares (RLS) algorithm with multiple time-varying forgetting factors for on-line parameter estimation of an induction machine (IM). The regressive mathematical model of the IM is also introduced which is simple and appropriate for online parameter estimation. The estimator inputs using the proposed RLS algorithm are easily measurable variables such as the stator voltages and currents as well as the rotor speed of the IM. The simulation results obtained compare the estimated parameters with the IM parameters achieved using other RLS algorithms such as a standard RLS algorithm and a RLS with a constant forgetting factor. The comparison shows that the proposed RLS algorithm is better than others for on-line parameter estimation of the IM.

Envelope tracking has many important applications in electrical distribution systems for either flicker meters or flicker compensators. This paper presents a novel control technique for flicker mitigation. This technique is based on the instantaneous tracking of the measured voltage envelope. The ADAptive LINEar neuron (ADALINE) algorithm, and the Recursive Least Square (RLS) algorithm are introduced for the flicker envelope tracking. The ADALINE algorithm is characterized by a light computational demand, unlike that of existing techniques; the RLS algorithm outperforms the ADALINE algorithm by its fast convergence and robust tracking performance. Both the ADALINE and the RLS algorithms give accurate results even under rapid dynamic changes. The paper investigates the effects of different parameters on the performance of the ADALINE algorithm and that of the RLS algorithm. Extensive simulations of the proposed flicker tracking algorithms are conducted to evaluate their performance for the tracking and mitigation of voltage flicker. The ADALINE and the RLS algorithms are examined by tracking and mitigating the flicker produced by a resistance welder and an arc furnace in a simple distribution system simulated in the PSCAD/EMTDC package.

A new recursive least square (RLS) algorithm based on nonlinear principal component analysis (NPCA) is proposed to address the blind source separation (BSS) problem in non-stationary environment. Forgetting factor is introduced to improve the tracking ability in non-stationary environment. The Kalman filter is used to solve the NPCA problem since its outstanding tracking performance in non-stationary environments. Simulations using the real speech source signals are used to illustrate the performance of the new RLS algorithm in static and non-stationary environments. Results show that the new RLS algorithm has faster convergence rate and better tracking capacity compared with the stochastic gradient algorithm, and previous RLS algorithm.

In this paper, a method of updating filter coefficients is proposed in which the RLS (Recursive Least Squares) algorithm is applied to a noise canceler with a filter for crosstalk removal. The RLS algorithm has an advantage over the Least Mean Square algorithm by virtue of better convergence. When the filter coefficients are updated including the filter for crosstalk removal, the error signal becomes nonlinear in terms of filter coefficients. Hence, a nonlinear RLS algorithm is derived in which such nonlinearity is taken into account. It is shown that the convergence is further improved in comparison with the conventional RLS algorithm. Finally, a method is proposed to reduce the noise contained in the transmitted voice signals of telephone by means of the present noise canceler and its effectiveness is evaluated. © 2003 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 86(5): 36–44, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.10039

The recursive least square (RLS) algorithm is insensitive to the dispersions of the correlation matrix. Consequently, the usage of the RLS algorithm to update the adaptive recursive filter coefficients are attractive for the system modeling and identification fields. In this paper, an error function RLS (EFRLS) algorithm is derived for the adaptive recursive filter. Furthermore, a scaled version of the EFRLS algorithm is proposed where the computation complexity is significantly reduced. The scaled EFRLS algorithm can be realized in real time using the available digital signal processors. The convergence to the optimal solution of both the EFRLS algorithm and its scaled version is ensured while the general RLS algorithm fails to converge to the optimal solution for the multimodal performance criterion.

The Cerebellar Model Articulation Controller (CMAC) neural network is an associative memory that is biologically inspired by the cerebellum, which is found in the brains of animals. The standard CMAC uses the least mean squares algorithm to train the weights. Recently, the recursive least squares (RLS) algorithm was proposed as a superior algorithm for training the CMAC online as it can converge in just one epoch, and does not require tuning of a learning rate. However, the RLS algorithm was found to be very computationally demanding as its computational complexity is dependent on the square of the number of weights required which can be huge for the CMAC. Here, we show a more efficient RLS algorithm that uses inverse QR decomposition and additionally provides a regularized solution, improving generalization. However, while the inverse QR decomposition based RLS algorithm reduces computation time significantly; it is still not fast enough for use in CMACs greater than two dimensions. To further improve efficiency we show that by using kernel methods the CMAC computational complexity can be transformed to become dependent on the number of unique training data. Additionally, it is shown how modeling error can be improved through use of higher order basis functions.

This paper proposes a new method of concurrent SOC and SOH estimation using a combination of recursive least square (RLS) algorithm and particle swarm optimization (PSO). The RLS algorithm is equipped with multiple fixed forgetting factors (MFFF) which are optimized by PSO. The performance of the hybrid RLS-PSO is compared with the similar RLS which is optimized by single objective genetic algorithms (SOGA) as well as multi-objectives genetic algorithm (MOGA). Open circuit voltage (OCV) is treated as a parameter to be estimated at the same timewith internal resistance. Urban Dynamometer Driving Schedule (UDDS) is used as the input data. Simulation results show that the hybrid RLS-PSO algorithm provides little better performance than the hybrid RLS-SOGA algorithm in terms of mean square error (MSE) and a number of iteration. On the other hand, MOGA provides Pareto front containing optimum solutions where a specific solution can be selected to have OCV MSE performance as good as PSO.

Vanadium redox flow battery (VRB) energy storage system has been widely utilized in renewable energy applications such as wind power integration and green buildings. An online electrical model of VRB is needed to monitor and estimate battery states by using real time data since the internal parameters of battery are varying during operation. In this paper, an improved second-order Thevenin model is proposed to describe the operating characteristics of VRB. To overcome the drawback of recursive least squares (RLS) algorithm in tracking parameter identification, this paper proposes a method based on time-varying forgetting factor recursive least Squares (TFF-RLS) algorithm to identify model parameters. According to the data collected by the pulse charging experiment, the RLS algorithm and the TFF-RLS algorithm are used to identify the model parameters. Experiments are carried out on a 5kWh/3kWh VRB experimental platform in Wuhan University of Technology. Two online identification methods are compared. The results show that the TFF-RLS method demonstrates a better performance on tracking the variation of parameters dynamically, and improving estimation accuracy compared with the RLS method.

In this paper, we proposed a recurrent kernel recursive least square (RLS) algorithm for online learning. In classical kernel methods, the kernel function number grows as the number of training sample increases, which makes the computational cost of the algorithm very high and only applicable for offline learning. In order to make the kernel methods suitable for online learning where the system is updated when a new training sample is obtained, a compact dictionary (support vectors set) should be chosen to represent the whole training data, which in turn reduces the number of kernel functions. For this purpose, a sparsification method based on the Hessian matrix of the loss function is applied to continuously examine the importance of the new training sample and determine the update of the dictionary according to the importance measure. We show that the Hessian matrix is equivalent to the correlation matrix of the training samples in the RLS algorithm. This makes the sparsification method able to be easily incorporated into the RLS algorithm and reduce the computational cost futher. Simulation results show that our algorithm is an effective learning method for online chaotic signal prediction and nonlinear system identification.

In this paper, we propose an recurrent neural network (RNN) based step-size estimation for the recursive least squares (RLS) algorithm with application to acoustic echo cancellation (AEC). RLS-based AEC (as compared to the least mean square (LMS) based) has a better convergence rate and less distortion, which is an important advantage in acoustic speech recognition (ASR) usages. The optimal step size of the RLS, as derived in the literature, is impractical due to its dependence on the power of the desired speech. An RNN model is presented that learns the relationship between the reference signal and the microphone signal to the optimal step size. At inference mode, the trained RNN produces the step size in real-time, which is then fed to the RLS algorithm. For evaluation, we used two hours of recordings from the AEC-challenge and Wav2vec2 databases, and compared the proposed technique and other competing methods. Experiments show that the proposed technique has advantages in terms of ASR performance and other classic measurements.

This chapter presents recursive least-squares (RLS) algorithms, which are based on numerically robust Householder transformations. Section 7.1 introduces the conventional orthogonal Householder transform and provides its geometrical interpretation. The hyperbolic Householder and the row (orthogonal and hyperbolic) Householder transforms are also briefly described. In Section 7.2, the Householder RLS (HRLS) algorithm is presented, its relation with the other known square-root RLS algorithms is discussed, and two applications, where the HRLS algorithm has been successfully applied, are presented. By considering a block-by-block update approach to the RLS problem, the block exact Householder QRD-RLS algorithm is developed in Section 7.3, which constitutes a generalization of the conventional QRD-RLS algorithm. Section 7.4 presents an inverse QRD-RLS algorithm, which employs block updating via the application of appropriate row orthogonal Householder transformations. Finally, a sliding window block RLS algorithm, which comprises a pair of row Householder transforms, is introduced in Section 7.5.

Recursive implementations of the least squares algorithm start the computation with a known set of initial conditions. The information contained in new data samples is then used to update the old estimates. Therefore the initialization procedure is seen to be an integral part of recursive algorithms. A number of methods for initializing recursive least squares (RLS) algorithms have been proposed in the literature. The two most common methods are the fast exact initialization and the soft constrained initialization. This paper discusses the equivalence relationship between RLS algorithms that use soft constrained initialization and a widely used technique in statistics called ridge regression.

The authors propose a new discrete time blind deconvolution technique for linear channels driven by cyclostationary inputs. This problem arises in digital communications, seismic signal processing, and many other applications. In particular, homomorphic approaches are applied to the cyclic autocorrelation of the fractionally-spaced sampled output of the channel. First, the method identifies the differential cepstrum parameters of the complex channel by means of a recursive least squares (RLS) algorithm. The RLS algorithm is based on the special characteristics of a cyclic autocorrelation matrix and an appropriate matrix inversion lemma. Once the differential cepstrum parameters are recovered, then the impulse response of the channel/equalizer is obtained by simple recursive formulas. Only partial information is required, i.e., the cyclic period and the distribution of the input data. It is shown that the method can directly identify the characteristics of either the channel or its inverse, provided that an unknown channel satisfies a special condition. The method is evaluated by means of computer simulations and is found to perform efficiently. >

In this work, we propose an adaptive linear interference suppression scheme based on single-carrier frequency domain equalization (SC-FDE) for multiuser direct-sequence ultra-wideband (DS-UWB) systems. In a block-by-block based transmission, by inserting the cyclic prefix (CP) between each data block, the inter-block interference (IBI) can be ignored and the Toeplitz channel matrix can be expressed as an equivalent circulant channel matrix. With this feature, we propose a simple signal model, in which only a vector formed adaptive filter needs to be trained in the frequency domain. Based on the new signal model, an adaptive block conjugate gradient (BCG) algorithm is developed. Complexity analysis shows that the proposed BCG algorithm is significantly simpler than the recursive least squares (RLS) algorithm. The bit-error rate (BER) performance of the proposed scheme is compared with the least-mean-square (LMS) and RLS algorithm in a number of scenarios and the simulation results show fast convergence speed and excellent performance of the proposed BCG algorithm.