Nonlinear Process Monitoring: Part 1

Kernel principal component analysis (KPCA) is a method for performing a nonlinear form of principal component analysis (PCA). It involves nonlinear mapping of the input data onto a high dimensional feature space and performing PCA there. Through KPCA, nonlinear relations in the input space are detected as linear relations in the feature space. Furthermore, explicit computation of the mapping is not necessary in KPCA. Only dot products of the mapped data items in the feature space are required which are obtained directly from the input data using a kernel function. Moreover, control limits of traditional statistical process monitoring indices are usually determined under assumption of normal (Gaussian) distribution. Although, it is recognized that this is inappropriate in nonlinear processes, the impact of using kernel density estimation (KDE), which is a nonparametric method, to determine control limits in KPCA-based process monitoring has not been reported. This paper seeks to bridge this gap. The KPCA with KDE approach is applied to the Tennessee Eastman process for the detection of faults. The results confirm that associating KPCA with kernel density estimated control limits provides better monitoring performance than using control limits based on the normal probability density function.

As a valid statistical tool, principal component analysis (PCA) has been widely used in industrial process monitoring. But due to its intrinsic linear character, it performs badly in nonlinear process monitoring. Kernel PCA (KPCA) can extract useful information in nonlinear data. However KPCA-based monitoring is not suitable for on-line monitoring because of large calculation and much memory occupation. The paper introduces an on-line monitoring method based on KPCA and neural network (NN), where KPCA is used to extract nonlinear principal components (PCs) and then NN approximates the relationship between process data and nonlinear PCs. We can obtain nonlinear PCs by NN to compute the monitoring indices and then achieve the on-line monitoring. The case study shows the validity of the method.KeywordsNeural NetworkKernel Principal Component AnalysisMonitoring IndexMemory OccupationComplex Industrial ProcessThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Improved fault detection based on kernel PCA for monitoring industrial applications

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

Kernel principal component analysis (KPCA) has become a popular technique for process monitoring in recent years. However, the performance largely depends on kernel function, yet methods to choose an appropriate kernel function among infinite ones have only been sporadically touched in the research literatures. In this paper, a novel methodology to learn a data-dependent kernel function automatically from specific input data is proposed and the improved kernel principal component analysis is obtained through using the data-dependent kernel function in traditional KPCA. The learning procedure includes two parts: learning a kernel matrix and approximating a kernel function. The kernel matrix is learned via a manifold learning method named maximum variance unfolding (MVU) which considers underlying manifold structure to ensure that principal components are linear in kernel space. Then, a kernel function is approximated via generalized Nystrom formula. The effectiveness of the improved KPCA model is confirmed by a numerical simulation and the Tennessee Eastman (TE) process benchmark.

Principal Components Analysis (PCA) is used to classify the running condition of a machine by means of projecting the original data to the Principal Components space. However, if the data are concentrated in a nonlinear subspace, PCA will fail to work well. Kernel Principal Components Analysis (KPCA) transforms the input data from the original input space into a higher dimensional feature space with the nonlinear mapping, and then uses the nonlinear principal components to realize the classification. In this paper a case of gear fault diagnosis was studied with KPCA. The characteristic values of frequent domain from vibration signals of the gearbox under the running condition were extracted, and the KPCA method was used to classify gear crack fault. The result shows that KPCA is more effective to distinguish the state of the gear and more suitable to diagnose the gear faults in early stage.

Fault discriminant enhanced kernel principal component analysis incorporating prior fault information for monitoring nonlinear processes

Aimed at the non-stationary characteristics of rolling bearing vibration signal,a fault diagnosis method was proposed based on local-wave method and KPCA(kernel principal component analysis)-LSSVM(least squares support vector machine).Firstly,local wave decomposition was used to decompose rolling bearing vibration signal into several intrinsic mode function(IMF),whose feature energy,singular values and AR model parameters were computed as initial feature vectors.Secondly,ini tial feature vectors were mapped into a higher-dimensional space with KPCA,and the kemel principal components were extracted as new feature vectors,which used as the input of LSSVM for fault classification.The experimental results show the KPCA-LSSVM method improves LSSVM's classification performance by KPCA obtaining additional discriminative information,and has better generalization than direct LSSVM method,and can identify rolling bearing fault patterns more accurately.

Data-driven techniques have been receiving considerable attention in the industrial process monitoring field due to their major advantages of easy implementation and less requirement for the prior knowledge and process mechanism. Principal component analysis (PCA) method is known as a popular method for monitoring and fault detection in industrial systems but as it is basically a linear method. However, most practical systems are nonlinear. To make the extension to nonlinear systems, kernel PCA (KPCA) method has been proposed for process modeling and monitoring. We present in this paper an online reduced rank optimized KPCA (RR-KPCA) technique for fault detection in order to extend the advantages of the KPCA models to online processes. Following the fault detection, the identification of the variables correlated to the fault occurred is of great importance. For this purpose, it is proposed to extend the approaches of localization by partial PCA and by elimination in the linear case to the nonlinear case, by exploiting the solution of reduction of the dimension of the kernel matrix in the feature space. The partial RR-KPCA and the elimination sensor identification (ESI-RRKPCA) are generated based on the static RR-KPCA and the online RR-KPCA methods. The idea of these approaches is to generate partial RR-KPCA models with reduced sets of variables. In other words, their goal is to generate indices of fault detection sensitive to certain faults and insensitive to others. The proposed fault isolation methods are applied for monitoring an air quality monitoring network (AIRLOR) data.

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

Support Vector Machines and Kernel methods have become very popular as methods for learning from examples. In this paper, in order to solve the non-linear and the time of recognition problem, recent advanced techniques are presented. For example, Kernel Principal Components and Proximal Support Vector Machines, Kernel Principal Components Analysis can describe multiple correlations between pixels in the image recognition, and can handle the high order statistics of the original data, at the same time the nonlinear features of the image can be preferably extracted. Proximal Support Vector Machines is an improvement for these characters of Support Vector Machines, combines the proximal algorithm with Support Vector Machines, has a better ability of nonlinear mapping and a stronger generalization capability, and the pace of Proximal Support Vector Machines is very enhanced. In this paper, the computer simulation was progressed based on ORL face database, the effectiveness of the Proximal Support Vector Machines algorithm and Kernel Principal Components Analysis algorithm were shown by experimental results.

An optimized feature extraction algorithm for abnormal network traffic detection

In kernel methods, choosing a suitable kernel is in dispensable for favorable results. No well-founded methods, however, have been established in general for unsupervised learning. We focus on kernel Princ ipal Component Analysis (kernel PCA), which is a nonlinear extension of principal component analysis and ha s been used electively for extracting nonlinear featu res and reducing dimensionality. As a kernel method , kernel PCA also suffers from the problem of kernel choice. Although cross-validation is a popular meth od for choosing hyperparameters, it is not applicable straightforwardly to choose a kernel in kernel PCA because of the incomparable norms given by differen t kernels. It is important, thus, to develop a well founded method for choosing a kernel in kernel PCA. This study proposes a method for choosing hyperparameters in kernel PCA (kernel and the number of components) based on cross-validation for the comparable reconstruction errors of pre-images in t he original space. The experimental results on synthesized and real-world datasets demonstrate tha t the proposed method successfully selects an appropriate kernel and the number of components in kernel PCA in terms of visualization and classification errors on the principal components. The results imply that the proposed method enables automatic design of hyperparameters in kernel PCA.

Sensor fault detection and isolation of an industrial gas turbine using partial adaptive KPCA

The identification of between-group differences and changes in gait mechanics are useful for injury prevention. Previous studies suggest the differences in gait biomechanical variables may interact in a complex non-linear fashion rather than a simple linear fashion. A non-linear multivariate analysis technique is therefore necessary to unravel the inherent structure in the gait data. Kernel principal component analysis (KPCA) is an extension of principal component analysis (PCA), the most widely used method in this field, and can provide inside into non-linear relationships of the variables of interest. Despite the growing use of PCA in running gait research, no prior studies have evaluated the utility of KPCA in determining the between-group differences in running gait patterns. Therefore, the objective of this study was to compare the performance of KPCA and PCA in identifying differences between male and female runners and as well between young and older runners in running gait kinematics. Running kinematic data were analysed on a gender group (female, n = 50; male, n = 50) and an age group (young, n = 50; older, n = 50) for discrete variables and waveform data. The following key results were obtained: (1) the performance of the KPCA for the identification of gender and age differences in running gait kinematics increased as compared to using the linear PCA; (2) the Gaussian function often performed better than the polynomial function for these two experiments; (3) there was no consistent optimal value of either the width σ of the Gaussian kernel or the degree d of the polynomial kernel for different data types and experiments. These results suggest that non-linear features extracted by KPCA could be potentially useful in identifying and discriminating between-group differences and changes in running gait patterns, and could provide useful information about the intrinsic non-linear dynamics of running movement.