Nonlinear process monitoring using wavelet kernel principal component analysis

Principal component analysis (PCA) has been proven to be an efficient method in dimensionality reduction, feature extraction and pattern recognition. Kernel principal component analysis (KPCA) can be considered as a natural nonlinear generalization of PCA, which performs linear PCA in a high dimensional space implicitly by using kernel trick. However, both conventional PCA and KPCA suffer from the deficiency of being sensitive to outliers. Existing robust KPCA has to eigen-decompose the gram matrix directly in each step and is much more computationally infeasible due to the large size of the matrix when the number of training samples is large. By extending existing robust PCA algorithm using kernel methods, we present a novel robust adaptive algorithm for calculating the kernel principal components. The proposed method not only preserves the characteristic of capturing underlying nonlinear structure of KPCA but also is robust against outliers by restraining the effect of outlying samples. Compared with existing robust KPCA methods, our method is performed without having to store the kernel matrix, which can reduce significantly the storage burden. In addition, our method shows the potential of expansibility to the incremental learning version. Experimental results on synthetic data indicate that our improved algorithm is effective and promising.

Kernel principal component analysis (KPCA) is a method for performing a nonlinear form of principal component analysis (PCA). It involves nonlinear mapping of the input data onto a high dimensional feature space and performing PCA there. Through KPCA, nonlinear relations in the input space are detected as linear relations in the feature space. Furthermore, explicit computation of the mapping is not necessary in KPCA. Only dot products of the mapped data items in the feature space are required which are obtained directly from the input data using a kernel function. Moreover, control limits of traditional statistical process monitoring indices are usually determined under assumption of normal (Gaussian) distribution. Although, it is recognized that this is inappropriate in nonlinear processes, the impact of using kernel density estimation (KDE), which is a nonparametric method, to determine control limits in KPCA-based process monitoring has not been reported. This paper seeks to bridge this gap. The KPCA with KDE approach is applied to the Tennessee Eastman process for the detection of faults. The results confirm that associating KPCA with kernel density estimated control limits provides better monitoring performance than using control limits based on the normal probability density function.

The identification of between-group differences and changes in gait mechanics are useful for injury prevention. Previous studies suggest the differences in gait biomechanical variables may interact in a complex non-linear fashion rather than a simple linear fashion. A non-linear multivariate analysis technique is therefore necessary to unravel the inherent structure in the gait data. Kernel principal component analysis (KPCA) is an extension of principal component analysis (PCA), the most widely used method in this field, and can provide inside into non-linear relationships of the variables of interest. Despite the growing use of PCA in running gait research, no prior studies have evaluated the utility of KPCA in determining the between-group differences in running gait patterns. Therefore, the objective of this study was to compare the performance of KPCA and PCA in identifying differences between male and female runners and as well between young and older runners in running gait kinematics. Running kinematic data were analysed on a gender group (female, n = 50; male, n = 50) and an age group (young, n = 50; older, n = 50) for discrete variables and waveform data. The following key results were obtained: (1) the performance of the KPCA for the identification of gender and age differences in running gait kinematics increased as compared to using the linear PCA; (2) the Gaussian function often performed better than the polynomial function for these two experiments; (3) there was no consistent optimal value of either the width σ of the Gaussian kernel or the degree d of the polynomial kernel for different data types and experiments. These results suggest that non-linear features extracted by KPCA could be potentially useful in identifying and discriminating between-group differences and changes in running gait patterns, and could provide useful information about the intrinsic non-linear dynamics of running movement.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

Automatic ECG beat classification plays an important role in detecting cardiac disease. In this paper, we propose an automatic recognition model for ECG signals based on discrete wavelet transform (DWT), principal component analysis (PCA), kernel principal component analysis (KPCA), and adaptive kernel principal component analysis (AKPCA). We extracted different ECG features using DWT, PCA, KPCA, and AKPCA, respectively. These features were combined and used as support vector machine (SVM) input to classify the ECG. ECG records taken from the MIT-BIH arrhythmia database are selected to test the proposed method. The following five heartbeat types were classified using this method: normal beats (N), premature ventricular beats (V), right bundle branch block beats (R), left bundle branch block beats (L), and premature atrial beats (A). The sensitivity, accuracy, precision, and specificity reached 99.95%, 99.86%, 99.53%, and 99.70%, respectively. These results indicate the proposed method is reliable and efficient for ECG beat classification.

Process monitoring is an integral part of chemical process, required higher product quality and safety operation. Therefore, the objective of this paper is to ensure the suitable functioning and to improve the fault detection performance of conventional kernel Principal Components Analysis (KPCA). Thus, an online Reduced Rank KPCA (OnRR-KPCA) with adaptive model has been developed to monitor a dynamic nonlinear process. The developed method is proposed. Firstly, to extract the useful observations, from large amount of training data registered in normal operating conditions, in order to construct the reduced reference model. Secondly, to monitor the process online and update the reference model if a new useful observation is available and satisfies the condition of independencies between variables in feature space. To demonstrate the effectiveness of the OnRR-KPCA with adaptive model over the conventional KPCA and the RR-KPCA, the fault detection performances are illustrated through two examples: one using synthetic data, the second using a simulated Tennessee Eastman Process (TEP) data.

Fault detection and diagnosis for non-linear processes empowered by dynamic neural networks

Principal Components Analysis (PCA) is used to classify the running condition of a machine by means of projecting the original data to the Principal Components space. However, if the data are concentrated in a nonlinear subspace, PCA will fail to work well. Kernel Principal Components Analysis (KPCA) transforms the input data from the original input space into a higher dimensional feature space with the nonlinear mapping, and then uses the nonlinear principal components to realize the classification. In this paper a case of gear fault diagnosis was studied with KPCA. The characteristic values of frequent domain from vibration signals of the gearbox under the running condition were extracted, and the KPCA method was used to classify gear crack fault. The result shows that KPCA is more effective to distinguish the state of the gear and more suitable to diagnose the gear faults in early stage.

Kernel discriminant analysis (KDA) and the kernel principal component analysis (KPCA), which are the extension of the linear discriminant analysis (LDA) and the principal component analysis (PCA), respectively, from linear domain to nonlinear domain via the kernel trick, are two very popular nonlinear feature extraction methods. In this paper, we present a new feature extraction algorithm by combing KDA and KPCA, and then apply it to the face recognition task. The experimental results on Yale face dataset show that the proposed method can significantly improve the performance both KDA and KPCA.

An SVM-like framework provides a novel way to learn linear principal component analysis (PCA). Actually it is a weighted PCA and leads to a semi-definite optimization problem (SDP). In this paper, we learn linear and nonlinear PCA with linear programming problems, which are easy to be solved and can obtain the unique global solution. Moreover, two algorithms for learning linear and nonlinear PCA are constructed, and all principal components can be obtained. To verify the performance of the proposed method, a series of experiments on artificial datasets and UCI benchmark datasets are accomplished. Simulation results demonstrate that the proposed method can compete with or outperform the standard PCA and kernel PCA (KPCA) in generalization ability but with much less memory and time consuming.

High pressure piston diaphragm pump is the most important power source of the pipeline transportation.To solve the problem of on-line monitoring on the fault of internal piston,the authors put forward a detection method based on acoustic emission signal's wavelet packet frequency and Kernel Principal Component Analysis(KPCA).Firstly,the author adopted wavelet packet to deal with the acoustic emission data to get each frequency band energy value.Secondly,the authors used KPCA to decompose the energy in high dimensional space to find the feature model,and made use of statistics SPE and T2 in feature model to make detection on fault signal.Finally,the authors conducted experiments to verify the statistics of acoustic emission of GEHO diaphragm pump's check valve.In comparison with the PCA method,the proposed method can make on-line monitoring on fault of internal piston fast and accurate,so it has good application prospect on the domain of the high pressure piston diaphragm pump's non-destructive fault detection.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

Subspace methods that utilize principal component analysis (PCA) are widely used for pattern classification or detection problems. Kernel PCA (KPCA) that is an extension of PCA is also applied to subspace methods. However, its computational cost is very high since the computational cost mainly depends on the number of samples in kernel methods. Recently, subset KPCA (SKPCA) has been proposed in order to reduce its computational complexity. In this paper, we apply SKPCA to subspace methods, and compare SKPCA with KPCA using some sample selection methods. Experimental results demonstrate advantages of subspace methods using SKPCA.

The article delves into the development of Statistical Fault Detection and Diagnosis Strategies for an Integrated Steel Plant (ISP) taking into account the nonlinear relationship amongst the monitored Process and Feedstock Characteristics. The strategies being devised is based on Neural Network Fitting model cum Principal Component Analysis based technique (NNF-PCA) and Kernel Principal Component Analysis (KPCA) based technique. For detection of fault(s) Hotelling T2 chart based on (Principal Component Analysis) PCA and KPCA scores were employed and an ensemble strategy amalgamating KPCA and Self Organizing Map neural network has also been proposed for the detection of the out-of-control observations or faults. The article also proposes a 2-phase Fault Diagnosis approach christened as Preliminary Diagnosis phase and Specific Diagnosis phase. The Preliminary Diagnosis phase is based on Pattern Analysis of the control chart monitoring statistic observations and the Specific Diagnosis phase is based on the employment of appropriate Fault Diagnostic Statistic. The Preliminary Diagnosis reveals the broader source of assignable cause for the onset of the fault(s) and the Specific Diagnosis reveals the relative contribution of the individual Process and Feedstock characteristics. An in-depth comparative analysis between the NNF-PCA based strategy and KPCA based strategy w.r.t. three comparative aspects and four comparative parameters were carried out with their findings being duly highlighted which revealed the slight effectiveness of the KPCA based strategy with respect to the NNF-PCA based counterpart.