Robust feature learning by stacked autoencoder with maximum correntropy criterion

In this paper, a regularized correntropy criterion (RCC) for extreme learning machine (ELM) is proposed to deal with the training set with noises or outliers. In RCC, the Gaussian kernel function is utilized to substitute Euclidean norm of the mean square error (MSE) criterion. Replacing MSE by RCC can enhance the anti-noise ability of ELM. Moreover, the optimal weights connecting the hidden and output layers together with the optimal bias terms can be promptly obtained by the half-quadratic (HQ) optimization technique with an iterative manner. Experimental results on the four synthetic data sets and the fourteen benchmark data sets demonstrate that the proposed method is superior to the traditional ELM and the regularized ELM both trained by the MSE criterion.

The problem of forming images that are optimal with respect to a Mean Square Error (MSE) criterion, based on nite data, is considered. First, it is shown that the MSE criterion is consistent with the general goal of classifying images, in that decreasing the MSE guarantees a decrease in the probability of misclassifying an image. The problem of choosing sampling locations for image formation that optimize the MSE is then formulated. It is shown that this MSE minimization problem has a solution under certain conditions and necessary conditions for a minimum are obtained. The results are illustrated on a simple image formation problem.

In many deep learning based speech enhancement frameworks, the deep neural network (DNN) is usually trained under the mean square error (MSE) criterion to estimate the potential clean amplitudes. Yet, the MSE criterion is not directly connected with speech perception, which potentiates alternative training targets for further improvements in speech quality and intelligibility. This paper introduces maximizing the correlation between the enhanced speech and the reference, so as to further improve the speech quality and intelligibility of the enhanced speech synchronously. Based on the short-time Fourier transform amplitude spectra, the correlations are firstly calculated between the envelops along the time and frequency respectively. Then, the time and frequency envelop correlations are incorporated into the time-frequency correlation (TFC). Finally, the loss function is defined as 1 minus the TFC. In experiment, several loss functions containing the time and frequency envelop correlations are investigated. The system trained with the TFC loss outperforms the baseline trained with the MSE loss with promising improvements in both speech quality and intelligibility. Phoneme recognition results also confirm the effectiveness of maximizing the correlation for speech enhancement.

Timely and accurate traffic flow prediction is the core of an intelligent transportation system. Canonical long short-term memory (LSTM) networks are guided by the mean square error (MSE) criterion, so it can handle Gaussian noise in traffic flow effectively. The MSE criterion is a global measure of the total error between the predictions and the ground truth. When the errors between the predictions and the ground truth are independent and identically Gaussian distributed, the MSE-guided LSTM networks work well. However, traffic flow is often impacted by non-Gaussian noise, and can no longer maintain an identical Gaussian distribution. Then, a $ {\overline{\delta }}_{relax} $-LSTM network guided by mixed correlation entropy and variable center (MCVC) criterion is proposed to simultaneously respond to both Gaussian and non-Gaussian distributions. The abundant experiments on four benchmark datasets of traffic flow show that the $ {\overline{\delta }}_{relax} $-LSTM network obtained more accurate prediction results than state-of-the-art models.

To date, most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in this article, we develop a new Kalman-type filter, called minimum error entropy KF (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC-based KFs, the proposed filter is also an online algorithm with the recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the MEE extended KF (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

MSE superiority of Bayes and empirical Bayes estimators in two generalized seemingly unrelated regressions

In this paper, the Bayes estimator and the parametric empirical Bayes estimator (PEBE) of mean vector in multivariate normal distribution are obtained. The superiority of the PEBE over the minimum variance unbiased estimator (MVUE) and a revised James-Stein estimators (RJSE) are investigated respectively under mean square error (MSE) criterion. Extensive simulations are conducted to show that performance of the PEBE is optimal among these three estimators under the MSE criterion.

Sleep staging based on electroencephalogram (EEG) signal, as one of the vital bases of study on sleep diagnosis, has been under massive attention. With the spring up of deep learning these years, the idea of combining deep learning structure with automatic sleep staging has been an attractive topic. However, the labeling of sleep stages requires professional knowledge as well as plenty of time, which raise the barrier to evaluate this idea. In this study, the method of unsupervised feature learning based on a mass of unlabeled data and a small number of labeled data was proposed to accomplish sleep staging. The unsupervised feature learning structure was built based on a pair of symmetric convolutional neural networks, with the help of a shallow neural network classifier to classify sleep stages. The results showed that under the condition of the very few labeled data, sleep staging based on unsupervised feature learning can achieve similar accuracy to supervised feature learning, which provides a new direction for the application of deep learning method in dealing with data that is difficult to label or lack of prior knowledge.

Autonomous Underwater Vehicles (AUVs) need accurate navigation and positioning data to carry out their operations in challenging marine environments. However, non-Gaussian environmental noise and sensor outliers in underwater environments degrade the positioning accuracy and stability of traditional algorithms. In such circumstances, the typical Unscented Kalman Filter (UKF) performs worse according to the Minimum Mean Square Error (MMSE) criteria. To reduce non-Gaussian environmental disturbances, a Maximum Correntropy Criterion (MCC) is created. This study extends the MCUKF by utilizing a mixture correntropy framework and an effective iterative optimization technique, and an improved Sage-Husa filtering mechanism is integrated into the iterative algorithm to address time-varying noise dynamics in complex scenarios. These changes increase the algorithm's resilience and flexibility in demanding underwater conditions. The algorithm's capability is confirmed using simulation and sea experiments, which show notable gains in Root Mean Square Error and navigation accuracy. Experimental results show that the proposed method outperforms other existing algorithms in terms of navigation accuracy performance in our simulations and experiments. To guarantee algorithm performance and resilience for the non-Gaussian time-varying measurement noise issue that AUVs encounter in intricate maritime settings, and to additionally enhance the precision and dependability of localization, this study uses the MCC-UKF to combat the environment's non-Gaussian noise, and the Levenberg-Marguardt iterative method to improve reliability of numbers and estimation precision, while the Maximum Mixture Correntropy Criterion (MMCC) serves the purpose of improve the filter flexibility and convergence speed, and finally, the improved Sage-Husa algorithm is combined to improve the robustness to time-varying noise. The following are the study's primary contributions. (1) In our study, we suggest an enhanced UKF algorithm that combines the MCC and iterated cost function, and adopt a Levenberg-Marquardt optimization strategy to accelerate the convergence. The mixture correntropy criterion was added in the construction of the LMIMC-UKF framework in order to achieve the synergistic enhancement of convergence performance and calculate accuracy. (2) Targeting the issue of time-varying environment noise interference, a fusion approach of iterative tuning and improved Sage-Husa filtering is suggested to achieve the systematic improvement of algorithm robustness. (3) By combining the Monte Carlo simulation with maritime measurement data, the algorithm's technical viability is confirmed, and the parameter sensitivity is quantitatively examined by methodical simulation tests. According to the results, the suggested algorithm performs noticeably better than the current maximum correntropy filtering techniques in terms of anti-interference capability, iteration speed, and navigation accuracy.

In recent years, feature learning methods based on unsupervised learning get more and more attention.Until now, Unsupervised feature learning has been applied to solve many problems such as detection, recognition and classification.In this paper, we propose a deep feature learning method based on Sparse AutoEncoder to recognize vehicle types and to improve the classification accuracy rate.First we used Sparse AutoEncoder to generate the convolutional kernel and used the convolutional kernel to generate convolutional feature.Then pooling was applied.We repeated the network several times to construct a deep feature learning framework.To improve performance, we also combined the feature learned in different layer to form a new feature vector and applied PCA to reduce the dimension.Finally we used softmax to recognize the vehicle type.Adopting the local receive field, we can reduce the parameters.The experiment shows that our method can improve the performance a little.

This paper investigates the kernel entropy based extended Kalman filter (EKF) as the navigation processor for the Global Navigation Satellite Systems (GNSS), such as the Global Positioning System (GPS). The algorithm is effective for dealing with non-Gaussian errors or heavy-tailed (or impulsive) interference errors, such as the multipath. The kernel minimum error entropy (MEE) and maximum correntropy criterion (MCC) based filtering for satellite navigation system is involved for dealing with non-Gaussian errors or heavy-tailed interference errors or outliers of the GPS. The standard EKF method is derived based on minimization of mean square error (MSE) and is optimal only under Gaussian assumption in case the system models are precisely established. The GPS navigation algorithm based on kernel entropy related principles, including the MEE criterion and the MCC will be performed, which is utilized not only for the time-varying adaptation but the outlier type of interference errors. The kernel entropy based design is a new approach using information from higher-order signal statistics. In information theoretic learning (ITL), the entropy principle based measure uses information from higher-order signal statistics and captures more statistical information as compared to MSE. To improve the performance under non-Gaussian environments, the proposed filter which adopts the MEE/MCC as the optimization criterion instead of using the minimum mean square error (MMSE) is utilized for mitigation of the heavy-tailed type of multipath errors. Performance assessment will be carried out to show the effectiveness of the proposed approach for positioning improvement in GPS navigation processing.

Maximum correntropy criterion (MCC) based adaptive filters are found to be robust against impulsive interference. This paper proposes a novel MCC based adaptive filter with variable step size in order to obtain improved performance in terms of both convergence rate and steady state error with robustness against impulsive interference. The optimal variable step size is obtained by minimizing the Mean Square Deviation (MSD) error from one iteration to the other. Simulation results in the context of a highly impulsive system identification scenario show that the proposed algorithm has faster convergence and lesser steady state error than the conventional MCC based adaptive filters.

The study of the robust fatigue feature learning method for the driver’s operational behavior is of great significance for improving the performance of the real-time detection system for driver’s fatigue state. Aiming at how to extract more abstract and deep features in the driver’s direction operation data in the robust feature learning, this article constructs a fuzzy recurrent neural network model, which includes input layer, fuzzy layer, hidden layer, and output layer. The steering-wheel direction sensing time series sends the time series to the input layer through a fixed time window. After the fuzzification process, it is sent to the hidden layer to share the weight of the hidden layer, realize the memorization of the fatigue feature, and improve the feature depth capability of the steering wheel angle time sequence. The experimental results show that the proposed model achieves an average recognition rate of 87.30% in the fatigue sample database of real vehicle conditions, which indicates that the model has strong robustness to different subjects under real driving conditions. The model proposed in this article has important theoretical and engineering significance for studying the prediction of fatigue driving under real driving conditions.

Steady-state mean-square performance analysis of the block-sparse maximum Versoria criterion

Under Gaussian assumption, online sequential extreme learning machine (OS-ELM) can achieve optimal performance. However, OS-ELM is based on the mean square error (MSE) criterion which is not a good choice for non-Gaussian signals. In this paper, a novel OS-ELM algorithm based on the generalized maximum correntropy criterion (OS-ELM-GMCC) is derived. Since the maximum correntropy criterion (MCC) can be generalized by the generalized maximum correntropy criterion (GMCC), the GMCC-based adaptive filtering algorithms with an appropriate shape parameter achieve better filtering performance than the MCC-based ones in the presence of non-Gaussian noise. As the important parameters, the number of hidden node, the shape parameter and scale parameter are discussed by simulations. Simulations in the context of the two examples including the system identification and Mackey-Glass (MG) chaotic time series demonstrate the superiority of OS-ELM-GMCC over OS-ELM and OS-ELM-MC.