Kernel Adaptive Filters With Feedback Based on Maximum Correntropy

A robust adaptive filtering algorithm based on the convex combination of two adaptive filters under the maximum correntropy criterion (MCC) is proposed. Compared with conventional minimum mean square error (MSE) criterion-based adaptive filtering algorithm, the MCC-based algorithm shows a better robustness against impulsive interference. However, its major drawback is the conflicting requirements between convergence speed and steady-state mean square error. In this letter, we use the convex combination method to overcome the tradeoff problem. Instead of minimizing the squared error to update the mixing parameter in conventional convex combination scheme, the method of maximizing the correntropy is introduced to make the proposed algorithm more robust against impulsive interference. Additionally, we report a novel weight transfer method to further improve the tracking performance. The good performance in terms of convergence rate and steady-state mean square error is demonstrated in plant identification scenarios that include impulsive interference and abrupt changes.

The multikernel adaptive filters based on the minimum mean square error (MMSE) criterion have been proposed to improve the performance of the kernel least mean square (KLMS), efficiently. However, these multikernel methods suffer from large computational burden as well as instability in impulsive noises. To address these issues, a novel multikernel method is proposed by replacing the trace operation of a matrix with the inner product of two vectors, thus leading to higher computational efficiency, significantly. Based on the maximum correntropy criterion, the multikernel maximum correntropy (MKMC) algorithm is therefore proposed. To further reduce the complexity, an online vector quantization strategy is presented for MKMC to generate the quantized MKMC (QMKMC) algorithm. Monte Carlo simulations on different nonlinear examples validate the superiorities of the proposed two algorithms.

Kernel recursive maximum correntropy

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

In wireless sensor networks (WSN), measurements are always corrupted by outliers or impulsive noise. Cubature information filtering (CIF) is founded based on minimum mean square error (MMSE) criterion, which is not applicable to non-Gaussian noise. Hence, a novel robust CIF (RCIF) is derived based on maximum correntropy criterion (MCC) to enhance the robustness of state estimation in the local node. For the information fusion, weighted average consensus (WAC) based distributed RCIF (DRCIF) is founded to improve the stability of sensor networks and the accuracy of state estimation. The estimation error of DRCIF is proved to be bounded in mean square. Numerical simulations are provided to evaluate the effectiveness of proposed algorithms.

This paper investigates the kernel entropy based extended Kalman filter (EKF) as the navigation processor for the Global Navigation Satellite Systems (GNSS), such as the Global Positioning System (GPS). The algorithm is effective for dealing with non-Gaussian errors or heavy-tailed (or impulsive) interference errors, such as the multipath. The kernel minimum error entropy (MEE) and maximum correntropy criterion (MCC) based filtering for satellite navigation system is involved for dealing with non-Gaussian errors or heavy-tailed interference errors or outliers of the GPS. The standard EKF method is derived based on minimization of mean square error (MSE) and is optimal only under Gaussian assumption in case the system models are precisely established. The GPS navigation algorithm based on kernel entropy related principles, including the MEE criterion and the MCC will be performed, which is utilized not only for the time-varying adaptation but the outlier type of interference errors. The kernel entropy based design is a new approach using information from higher-order signal statistics. In information theoretic learning (ITL), the entropy principle based measure uses information from higher-order signal statistics and captures more statistical information as compared to MSE. To improve the performance under non-Gaussian environments, the proposed filter which adopts the MEE/MCC as the optimization criterion instead of using the minimum mean square error (MMSE) is utilized for mitigation of the heavy-tailed type of multipath errors. Performance assessment will be carried out to show the effectiveness of the proposed approach for positioning improvement in GPS navigation processing.

Autonomous Underwater Vehicles (AUVs) need accurate navigation and positioning data to carry out their operations in challenging marine environments. However, non-Gaussian environmental noise and sensor outliers in underwater environments degrade the positioning accuracy and stability of traditional algorithms. In such circumstances, the typical Unscented Kalman Filter (UKF) performs worse according to the Minimum Mean Square Error (MMSE) criteria. To reduce non-Gaussian environmental disturbances, a Maximum Correntropy Criterion (MCC) is created. This study extends the MCUKF by utilizing a mixture correntropy framework and an effective iterative optimization technique, and an improved Sage-Husa filtering mechanism is integrated into the iterative algorithm to address time-varying noise dynamics in complex scenarios. These changes increase the algorithm's resilience and flexibility in demanding underwater conditions. The algorithm's capability is confirmed using simulation and sea experiments, which show notable gains in Root Mean Square Error and navigation accuracy. Experimental results show that the proposed method outperforms other existing algorithms in terms of navigation accuracy performance in our simulations and experiments. To guarantee algorithm performance and resilience for the non-Gaussian time-varying measurement noise issue that AUVs encounter in intricate maritime settings, and to additionally enhance the precision and dependability of localization, this study uses the MCC-UKF to combat the environment's non-Gaussian noise, and the Levenberg-Marguardt iterative method to improve reliability of numbers and estimation precision, while the Maximum Mixture Correntropy Criterion (MMCC) serves the purpose of improve the filter flexibility and convergence speed, and finally, the improved Sage-Husa algorithm is combined to improve the robustness to time-varying noise. The following are the study's primary contributions. (1) In our study, we suggest an enhanced UKF algorithm that combines the MCC and iterated cost function, and adopt a Levenberg-Marquardt optimization strategy to accelerate the convergence. The mixture correntropy criterion was added in the construction of the LMIMC-UKF framework in order to achieve the synergistic enhancement of convergence performance and calculate accuracy. (2) Targeting the issue of time-varying environment noise interference, a fusion approach of iterative tuning and improved Sage-Husa filtering is suggested to achieve the systematic improvement of algorithm robustness. (3) By combining the Monte Carlo simulation with maritime measurement data, the algorithm's technical viability is confirmed, and the parameter sensitivity is quantitatively examined by methodical simulation tests. According to the results, the suggested algorithm performs noticeably better than the current maximum correntropy filtering techniques in terms of anti-interference capability, iteration speed, and navigation accuracy.

Improved proportionate-type sparse adaptive filtering under maximum correntropy criterion in impulsive noise environments

The kernel adaptive filters (KAFs) based on the minimum mean square error (MMSE) criterion in reproducing kernel Hilbert space (RKHS) improve the performance of linear adaptive filters but result in instability issues and large burdens of computation and memory in impulsive noises. To this end, a novel Nystrom kernel recursive generalized maximum correntropy (NKRGMC) with probability density rank-based quantization (PRQ) sampling (NKRGMC-PRQ) algorithm is proposed to improve filtering performance, robustness, and computational efficiency of the traditional KAFs in this letter. In a fixed dimensional network structure, the proposed NKRGMC-PRQ algorithm can achieve a comparable performance to KAFs with low computational complexity. Monte Carlo simulations are conducted to validate the superiorities of NKRGMC-PRQ in terms of filtering accuracy, computational complexity, and robustness.

As a new measure of similarity, the correntropy can be used as an objective function for many applications. In this letter, we study Bayesian estimation under maximum correntropy (MC) criterion. We show that the MC estimation is, in essence, a smoothed maximum a posteriori (MAP) estimation, including the MAP and the minimum mean square error (MMSE) estimation as the extreme cases. We also prove that under a certain condition, when the kernel size in correntropy is larger than some value, the MC estimation will have a unique optimal solution lying in a strictly concave region of the smoothed posterior distribution.

Unsupervised feature learning with deep networks has been widely studied in the recent years. Despite the progress, most existing models would be fragile to non-Gaussian noises and outliers due to the criterion of mean square error (MSE). In this paper, we propose a robust stacked autoencoder (R-SAE) based on maximum correntropy criterion (MCC) to deal with the data containing non-Gaussian noises and outliers. By replacing MSE with MCC, the anti-noise ability of stacked autoencoder is improved. The proposed method is evaluated using the MNIST benchmark dataset. Experimental results show that, compared with the ordinary stacked autoencoder, the R-SAE improves classification accuracy by 14% and reduces the reconstruction error by 39%, which demonstrates that R-SAE is capable of learning robust features on noisy data.

Performance analysis of the deficient length augmented CLMS algorithm for second order noncircular complex signals

The past decade has seen the rapid development of information theoretic learning and its applications in signal processing and machine learning. Specifically, minimum error entropy (MEE) and maximum correntropy criterion (MCC) have been widely studied in the literature. Although MEE and MCC are applied in many branches of knowledge and could outperform statistical criteria (such as mean square error), they have not been compared with each other from theoretical point of view. In some cases, MEE and MCC perform similarly to each other; however, under some conditions (e.g., in non-Gaussian environments), they act differently. This letter derives a new information theoretic relation between MEE and MCC, leading to better understanding of the theoretical differences, and illustrates the findings in a common example.

Maximum correntropy criterion (MCC) based adaptive filters are found to be robust against impulsive interference. This paper proposes a novel MCC based adaptive filter with variable step size in order to obtain improved performance in terms of both convergence rate and steady state error with robustness against impulsive interference. The optimal variable step size is obtained by minimizing the Mean Square Deviation (MSD) error from one iteration to the other. Simulation results in the context of a highly impulsive system identification scenario show that the proposed algorithm has faster convergence and lesser steady state error than the conventional MCC based adaptive filters.

Long-term prediction of chaotic time series is very difficult, for the chaos restricts predictability. In this paper a new method is studied to model and predict chaotic time series based on minimax probability machine regression (MPMR). Since the positive global Lyapunov exponents lead the errors to increase exponentially in modelling the chaotic time series, a weighted term is introduced to compensate a cost function. Using mean square error (MSE) and absolute error (AE) as a criterion, simulation results show that the proposed method is more effective and accurate for multistep prediction. It can identify the system characteristics quite well and provide a new way to make long-term predictions of the chaotic time series.