Ell _1-regularized recursive total least squares based sparse system identification for the error-in-variables

Online parameter identification is essential for the accuracy of the battery equivalent circuit model (ECM). The traditional recursive least squares (RLS) method is easily biased with the noise disturbances from sensors, which degrades the modeling accuracy in practice. Meanwhile, the recursive total least squares (RTLS) method can deal with the noise interferences, but the parameter slowly converges to the reference with initial value uncertainty. To alleviate the above issues, this paper proposes a co-estimation framework utilizing the advantages of RLS and RTLS for a higher parameter identification performance of the battery ECM. RLS converges quickly by updating the parameters along the gradient of the cost function. RTLS is applied to attenuate the noise effect once the parameters have converged. Both simulation and experimental results prove that the proposed method has good accuracy, a fast convergence rate, and also robustness against noise corruption.

Sparse recursive least squares (RLS) algorithms designed by introducing a sparse penalty (regularization) into the standard RLS cost function, have been proposed in the literature. Compared with the standard RLS, the sparse RLS achieves faster convergence and better performance under sparse systems. Even though, it includes in the updating equation an additional sparse term, which not only incurs extra complexity but also prevents the use of existing fast implementations such as the stable fast transversal filter (SFFT) algorithm. In this paper, we aim to reduce the complexity of the sparse RLS for promoting its practicability. To achieve the goal, the sparse updating term is analyzed and then approximated. With an approximated sparse updating term, the fast implementation is enabled for the sparse RLS, achieving complexity reduction. To demonstrate the feasibility of the proposed scheme, thel 0 -RLS (as a typical sparse RLS algorithm) coupled with an approximated sparse updating term is proposed, leading to the selective zero-attracting SFTF(SZA-SFTF) algorithm. The SZA-SFTF has a complexity of order $O(11N)$, compared with $O(N^{2})$ for the originall 0 -RLS. In term of performance, simulations of sparse system identification showed the SZA-SFTF considerably outperforms the standard SFTF and achieves close performance to the exactl 0 -RLS.

The paper presents a real-time approach for parameter identification used for diagnosing different types of faults in the exhaust gas recirculation (EGR) system of diesel engines. An important benefit of the proposed diagnostics method is its ability to detect and estimate a leak or a restriction in the EGR system. This is achieved by making use of a recursive-least-squares (RLS) method, as well as, a recursive formulation of a robust version of the RLS method, which we refer to as recursive total-least-squares (RTLS) method. The proposed approach of fault detection is successfully applied to diagnose low flow or high flow faults in an engine and is validated using experimental data.

Adaptive Fourier analyzers are used to estimate the coefficients of the sine and cosine terms of a noisy sinusoidal signal assuming the frequencies are known. The recursive least square (RLS) Fourier analyzer is a powerful algorithm that provides excellent performance. However, it is computationally very intensive. Furthermore, in real-life applications, the signal frequencies may differ from their assumed or supposed values. This difference, referred to as frequency mismatch (FM), may significantly deteriorate the performance of the RLS. In this paper, we first propose two fast RLS (FRLS) algorithms by utilizing the inherent characteristics of the estimation problem. The new FRLS algorithms perform almost the same as the RLS, while require considerably less computations. Next, the RLS as well as the proposed two FRLS algorithms are modified by incorporating a new adaptive scheme that alleviates the influence of the FM. Extensive simulations are provided to clarify our claims on the proposed FRLS algorithms, and to show that all the modified Fourier analyzers are capable of accommodating the FM very effectively.

The proportionate normalized least mean square (PNLMS) algorithm is used in sparse system identification for its simplicity and adaptively step-size adjusting scheme. However, the PNLMS has the noisy input and the non-Gaussian output noise problem. A bias Compensated PNLMF algorithm (called BCPNLMF) for identifying sparse system has been proposed to solve aforementioned issues. The BCPNLMF algorithm which takes advantage of the bias compensated and the proportionate scheme, can achieve better steady-state accuracy and faster convergence speed besides identify the system parameters in noisy input and output with non-Gaussian character environments. Simulation results carried out in sparse system identification confirm the remarkable performance of the BCPNLMF, compared with other well-known algorithms.

The recursive least square (RLS) algorithm is insensitive to the dispersions of the correlation matrix. Consequently, the usage of the RLS algorithm to update the adaptive recursive filter coefficients are attractive for the system modeling and identification fields. In this paper, an error function RLS (EFRLS) algorithm is derived for the adaptive recursive filter. Furthermore, a scaled version of the EFRLS algorithm is proposed where the computation complexity is significantly reduced. The scaled EFRLS algorithm can be realized in real time using the available digital signal processors. The convergence to the optimal solution of both the EFRLS algorithm and its scaled version is ensured while the general RLS algorithm fails to converge to the optimal solution for the multimodal performance criterion.

This paper considers the realization of high‐speed phase synchronization when there exists a frequency error and discusses the application of the recursive least square (RLS) algorithm to the burst signal demodulator. Especially, as a phase control algorithm for the coherent detection where there is an error in the carrier frequency, a method is proposed to represent the phase variation by the frequency error as the state transition function and a detailed algorithm is derived.It is shown that the theoretical convergence and the tracking performance of the algorithm agree with those of the ordinary RLS algorithm. The algorithm is applied to the frequency control circuit of the π/4‐QPSK differential detector and the phase control circuit of the 16‐QAM quasi‐coherent detector.It is demonstrated experimentally that the system has an excellent performance and that the initial convergence is complete in approximately ten symbols when there exists a frequency error.

This paper incorporates a correntropy-induced metric (CIM)-based sparsity constraint into variable step size normalized least mean square (VSSNLMS) algorithm for identification of sparse system corrupted by additive noise. The proposed CIM-VSSNLMS algorithm makes a good trade-off between convergence characteristics, filter stability, and steady state error. The proposed (CIM-VSSNLMS) algorithm incorporates a variable step size that accelerates the convergence characteristics and lowers the normalized misalignment (NMSA) error with respect to CIM-NLMS with constant value of step size. An expression for variant step size is derived under the stability condition of proposed algorithm. Finally, the implementation of the proposed algorithm is carried out in MATLAB software to manifest the improved estimated behavior in the identification of sparse system.

Sparse adaptive filtering algorithms are utilized to exploit potential sparse structure information as well as to mitigate noises in many unknown sparse systems. Sparse recursive least square (RLS) algorithms have been attracted intensely attentions due to their low-complexity and easy- implementation. Basically, these algorithms are constructed by standard RLS algorithm and sparse penalty functions (e.g., l_1-norm). However, existing sparse RLS algorithms do not exploit the sparsity efficiently. In this paper, an improved adaptive filtering algorithm is proposed by incorporating a novel correntropy induced metric (CIM) constraint into RLS, which is termed as RLS- CIM algorithm. Specifically, we adopt a well-known Gaussian kernel in CIM and further devise a novel variable kernel width to control the sparse penalty in different transient-error scenarios. Numerical simulation results are given to corroborate the proposed algorithm via mean square deviation (MSD).

System identification, which includes parameter identification and non-parameter identification, is to estimate its mathematical model based on the input and output observation in system. This paper discusses the system identification theory and establishes a transfer function of 1/4 vehicle’s second-order vibration system model. Through the discrete transfer function, the system’s difference equation can be obtained to identify the system in two ways, RLS (recursive least squares) and RELS (extended recursive least squares). Finally, the paper makes a comparative analysis about RLS and RELS in connection with the vehicle model. The results show that RELS method is more accurate and has stronger convergence than RLS method, which provides the basis for the researching of control system’s algorithm, simulation and making control strategy.

The newly proposed l 1 norm constraint zero-point attraction least mean square algorithm (ZA-LMS) demonstrates excellent performance on exact sparse system identification. However, ZA-LMS has less advantage against standard LMS when the system is near sparse. Thus, in this study, firstly the near sparse system (NSS) modelling by generalised Gaussian distribution is recommended, where the sparsity is defined accordingly. Second, two modifications to the ZA-LMS algorithm have been made. The l 1 norm penalty is replaced by a partial l 1 norm in the cost function, enhancing robustness without increasing the computational complexity. Moreover, the ZA item is weighted by the magnitude of estimation error which adjusts the ZA force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS (DWZA-LMS) algorithm is further proposed, which shows better performance on NSS identification. In addition, the mean-square performance of DWZA-LMS algorithm is analysed. Finally, computer simulations demonstrate the effectiveness of the proposed algorithm and verify the result of theoretical analysis.

In this thesis, four methods specifically designed for sparse systems estimation are originally developed and presented, which were called here: Relaxations method, Successive Expansions method, l 1 -norm Minimization method and Automatic Adjustment of the Regularization Factor method. The four proposed methods are based on the Least Squares (LS) Estimation method and incorporate techniques related to convex optimization and to the theory of compressive sensing. The simulation results show that the proposed methods herein present superior performance than the ordinary LS estimation method and the Recursive Least Squares (RLS) with convex regularization method (l 1 -RLS), in many cases achieving the same optimal performance presented by the LS Oracle method. Furthermore, the proposed methods demand lower computational cost than the l 1 -RLS method.

In this paper, we proposed a recurrent kernel recursive least square (RLS) algorithm for online learning. In classical kernel methods, the kernel function number grows as the number of training sample increases, which makes the computational cost of the algorithm very high and only applicable for offline learning. In order to make the kernel methods suitable for online learning where the system is updated when a new training sample is obtained, a compact dictionary (support vectors set) should be chosen to represent the whole training data, which in turn reduces the number of kernel functions. For this purpose, a sparsification method based on the Hessian matrix of the loss function is applied to continuously examine the importance of the new training sample and determine the update of the dictionary according to the importance measure. We show that the Hessian matrix is equivalent to the correlation matrix of the training samples in the RLS algorithm. This makes the sparsification method able to be easily incorporated into the RLS algorithm and reduce the computational cost futher. Simulation results show that our algorithm is an effective learning method for online chaotic signal prediction and nonlinear system identification.

A general zero attraction (GZA) proportionate normalized maximum correntropy criterion (GZA-PNMCC) algorithm is devised and presented on the basis of the proportionate-type adaptive filter techniques and zero attracting theory to highly improve the sparse system estimation behavior of the classical MCC algorithm within the framework of the sparse system identifications. The newly-developed GZA-PNMCC algorithm is carried out by introducing a parameter adjusting function into the cost function of the typical proportionate normalized maximum correntropy criterion (PNMCC) to create a zero attraction term. The developed optimization framework unifies the derivation of the zero attraction-based PNMCC algorithms. The developed GZA-PNMCC algorithm further exploits the impulsive response sparsity in comparison with the proportionate-type-based NMCC algorithm due to the GZA zero attraction. The superior performance of the GZA-PNMCC algorithm for estimating a sparse system in a non-Gaussian noise environment is proven by simulations.

Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-response (FIR) system identification and, finally, to infinite-impulse-response (IIR) system identification. For each case, we relate the condition number of the regressor matrix to the transient response and rate of convergence of the parameter estimates.