Kernel PCA in Detecting Moving Vehicle from Its Viewpoint

Kernel principal component analysis (KPCA) is a method for performing a nonlinear form of principal component analysis (PCA). It involves nonlinear mapping of the input data onto a high dimensional feature space and performing PCA there. Through KPCA, nonlinear relations in the input space are detected as linear relations in the feature space. Furthermore, explicit computation of the mapping is not necessary in KPCA. Only dot products of the mapped data items in the feature space are required which are obtained directly from the input data using a kernel function. Moreover, control limits of traditional statistical process monitoring indices are usually determined under assumption of normal (Gaussian) distribution. Although, it is recognized that this is inappropriate in nonlinear processes, the impact of using kernel density estimation (KDE), which is a nonparametric method, to determine control limits in KPCA-based process monitoring has not been reported. This paper seeks to bridge this gap. The KPCA with KDE approach is applied to the Tennessee Eastman process for the detection of faults. The results confirm that associating KPCA with kernel density estimated control limits provides better monitoring performance than using control limits based on the normal probability density function.

Principal Components Analysis (PCA) is used to classify the running condition of a machine by means of projecting the original data to the Principal Components space. However, if the data are concentrated in a nonlinear subspace, PCA will fail to work well. Kernel Principal Components Analysis (KPCA) transforms the input data from the original input space into a higher dimensional feature space with the nonlinear mapping, and then uses the nonlinear principal components to realize the classification. In this paper a case of gear fault diagnosis was studied with KPCA. The characteristic values of frequent domain from vibration signals of the gearbox under the running condition were extracted, and the KPCA method was used to classify gear crack fault. The result shows that KPCA is more effective to distinguish the state of the gear and more suitable to diagnose the gear faults in early stage.

This paper examines the Kernel Principal Component Analysis (KPCA) feature detection and classification for underwater images. In Underwater images the numbers of distortion occurred are blurring of image, illumination of light and rotation of angle, noise etc. Features are normally extracted by the method called SIFT (Scale Invariant Feature Transform for underwater images). It is used for extracting distinctive invariant features from images that can be invariant to image scale and rotation. It is used in image mosaic, recognition, retrieval and etc. where PCA-SIFT (Principal Component Analysis-Scale Invariant Feature Transform) is also used for dimension reduction and feature detection for underwater images. In this paper we propose a method by combining KPCA and SIFT together called KPCA-SIFT feature detection for underwater images. It is well suited for blur, illumination change and rotation of the input image. When apply PCA to the normalized gradient patch it reduces the dimension of the feature extracted. The parameters used for evaluation are Precision Vs Recall curve and the parameter like elapsed time, sigma value and threshold value. PNN classification is used for underwater images. The proposed method gives desirable results with respect to parameters like Roc curve, Precision Vs Recall, Elapsed time and sigma value. It is more robust and distinctive image deformation and more compact. with increased accuracy and faster matching than the PCA SIFT and SIFT algorithms.

Viewing to monitoring the continuous casting process, a nonlinear fault detection method based on kernel principal component analysis (KPCA) was introduced. KPCA can efficiently compute principal components in high-dimensional feature spaces by means of integral operators and nonlinear kernel functions, which is to first map the input space into a feature space via nonlinear mapping and then to compute the principal components in that feature space. Based on T2 and SPE charts in feature space, principal component analysis(PCA)can be used to detect faults.. The simulation results show that the proposed approach effectively captured the nonlinear relationship in the process variables and showed superior process monitoring performance compared to linear PCA.

Summary The principal component analysis (PCA) is exists the problem of lack of probability model and the absence of higher-order statistics information. Therefore，in order to overcome its shortcomings，this paper studies the method that can overcome two disadvantages of the principal component analysis (PCA) — the probability kernel principal component analysis (PKPCA) which is based on Bayesian theory and kernel principal component analysis (KPCA). First，the sample data are mapped to the high dimensional feature space，then define probability model of the data in high-dimensional space，and finally，expectation maximization (EM) estimated is used to get the best results. This method has both the advantage of probability analysis and kernel principal component analysis (KPCA). It is able to effectively adapt to more complex reservoir conditions and can realize the non-linear probability analysis. The probability of kernel principal component analysis (PKPCA) is applied to reservoir prediction of the oilfield. The predicted results show that the method can improve the precision attribute optimization，while improving the accuracy of the forecasts of reservoir.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

There is a trend in recent machine learning community to construct a nonlinear version of a linear algorithm using the 'kernel method', e.g. Support Vector Machines (SVMs), kernel principal component analysis, kernel fisher discriminant analysis and the recent kernel clustering algorithms. In unsupervised clustering algorithms using kernel method, typically, a nonlinear mapping is used first to map the data into a potentially much higher feature space, where clustering is then performed. A drawback of these kernel clustering algorithms is thatthe clustering prototypes lie in high dimensional feature space and hence lack clear and intuitive descriptions unless using additional projection approximation from the feature to the data space as done in the existing literatures. In this paper, a novel clustering algorithm using the 'kernel method' based on the classical fuzzy clustering algorithm (FCM) is proposed and called as kernel fuzzy c-means algorithm (KFCM). KFCM adopts a new kernel-induced metric in the data space to replace the original Euclidean norm metric in FCM and the clustered prototypes still lie in the data space so that the clustering results can be reformulated and interpreted in the original space. Our analysis shows that KFCM is robust to noise and outliers and also tolerates unequal sized clusters. And finally this property is utilized to cluster incomplete data. Experiments on two artificial and one real datasets show that KFCM hasbetter clustering performance and more robust than several modifications of FCM for incomplete data clustering.

In this paper, a novel feature extraction method is proposed; Genetic Programming (GP) is used to discover features, while the Fisher criterion is employed to provide fitness values. This produces nonlinear features for both two-class and multi-class recognition problems by revealing the discriminating information between classes. The proposed approach is experimentally compared to conventional nonlinear feature extraction methods, including kernel generalised discriminant analysis (KGDA), kernel principal component analysis (KPCA). Results demonstrate the capability of the proposed approach to transform information from the high dimensional feature space into a single dimensional space by automatically discovering the relationships among data.

This paper describes a novel approach for creating an efficient, general, and differentiable parameterization of large-scale non-Gaussian, non-stationary random fields (represented by multipoint geostatistics) that is capable of reproducing complex geological structures such as channels. Such parameterizations are appropriate for use with gradient-based algorithms applied to, for example, history-matching or uncertainty propagation. It is known that the standard Karhunen–Loeve (K–L) expansion, also called linear principal component analysis or PCA, can be used as a differentiable parameterization of input random fields defining the geological model. The standard K–L model is, however, limited in two respects. It requires an eigen-decomposition of the covariance matrix of the random field, which is prohibitively expensive for large models. In addition, it preserves only the two-point statistics of a random field, which is insufficient for reproducing complex structures. In this work, kernel PCA is applied to address the limitations associated with the standard K–L expansion. Although widely used in machine learning applications, it does not appear to have found any application for geological model parameterization. With kernel PCA, an eigen-decomposition of a small matrix called the kernel matrix is performed instead of the full covariance matrix. The method is much more efficient than the standard K–L procedure. Through use of higher order polynomial kernels, which implicitly define a high-dimensionality feature space, kernel PCA further enables the preservation of high-order statistics of the random field, instead of just two-point statistics as in the K–L method. The kernel PCA eigen-decomposition proceeds using a set of realizations created by geostatistical simulation (honoring two-point or multipoint statistics) rather than the analytical covariance function. We demonstrate that kernel PCA is capable of generating differentiable parameterizations that reproduce the essential features of complex geological structures represented by multipoint geostatistics. The kernel PCA representation is then applied to history match a water flooding problem. This example demonstrates that kernel PCA can be used with gradient-based history matching to provide models that match production history while maintaining multipoint geostatistics consistent with the underlying training image.

A nonlinearly discriminant feature extraction and fusion scheme is proposed to recognize the different condition of mechanical faults. As most running statuses of machines are nonlinear and non-stationary, it is difficult to extract the effective features for fault diagnosis by linear feature extractor such as principal component analysis (PCA) and Fisher linear discriminant analysis (FLD). Therefore, kernel Fisher discriminant analysis (kernel FDA) is used to extract the nonlinear discriminant feature. In addition, the idea of feature fusion is introduced to improve the robustness of feature extractor, because feature fusion can synthesize complementary information of different feature variables from multi-signal sources. The heavy computational burden induced by the tremendous size of the feature space is also effectively settled by using a kernel function in the input space without explicit computation of the mapping in feature space, and the influence of the parameter in kernel function on recognition performance is also discussed. The results show that the proposed feature fusion scheme can effectively improve the robustness of feature extractor, and nonlinearly discriminant features perform better than kernel PCA features.

This paper reviewed the classical principal components analysis methods for multivariate data analysis and feature extraction in pattern classification. A kernel-based extension to the classical PCA models was discussed to cope with nonlinear data dependencies. Kernel PCA was implicitly performing a linear PCA in some high-dimensional kernel feature space that was nonlinearly related to input space by using a suitable nonlinear kernel function mapping. And then, the conjunction of kernel PCA method and RBF neural networks was proposed in practical and algorithmic considerations. Finally, we illustrate the usefulness of kernel PCA algorithms by discussing kernel PCA RBF neural networks application in handwritten digit classification

Aimed at the non-stationary characteristics of rolling bearing vibration signal,a fault diagnosis method was proposed based on local-wave method and KPCA(kernel principal component analysis)-LSSVM(least squares support vector machine).Firstly,local wave decomposition was used to decompose rolling bearing vibration signal into several intrinsic mode function(IMF),whose feature energy,singular values and AR model parameters were computed as initial feature vectors.Secondly,ini tial feature vectors were mapped into a higher-dimensional space with KPCA,and the kemel principal components were extracted as new feature vectors,which used as the input of LSSVM for fault classification.The experimental results show the KPCA-LSSVM method improves LSSVM's classification performance by KPCA obtaining additional discriminative information,and has better generalization than direct LSSVM method,and can identify rolling bearing fault patterns more accurately.

Recently, we have been investigating the application of kernel methods to improve the performance of eigenvoice-based adaptation methods by exploiting possible nonlinearity in their original working space. We proposed the kernel eigenvoice adaptation (KEV), and the kernel eigenspace-based MLLR adaptation (KEMLLR). In KEMLLR, speaker-dependent MLLR transformation matrices are mapped to a kernel-induced high dimensional feature space, and kernel principal component analysis (KPCA) is used to derive a set of eigenmatrices in the feature space. A new speaker is then represented by a linear combination of the leading eigenmatrices. In this paper, we further improve KEMLLR by the use of multiple regression classes and the quasi-Newton BFGS optimization algorithm.

The identification of between-group differences and changes in gait mechanics are useful for injury prevention. Previous studies suggest the differences in gait biomechanical variables may interact in a complex non-linear fashion rather than a simple linear fashion. A non-linear multivariate analysis technique is therefore necessary to unravel the inherent structure in the gait data. Kernel principal component analysis (KPCA) is an extension of principal component analysis (PCA), the most widely used method in this field, and can provide inside into non-linear relationships of the variables of interest. Despite the growing use of PCA in running gait research, no prior studies have evaluated the utility of KPCA in determining the between-group differences in running gait patterns. Therefore, the objective of this study was to compare the performance of KPCA and PCA in identifying differences between male and female runners and as well between young and older runners in running gait kinematics. Running kinematic data were analysed on a gender group (female, n = 50; male, n = 50) and an age group (young, n = 50; older, n = 50) for discrete variables and waveform data. The following key results were obtained: (1) the performance of the KPCA for the identification of gender and age differences in running gait kinematics increased as compared to using the linear PCA; (2) the Gaussian function often performed better than the polynomial function for these two experiments; (3) there was no consistent optimal value of either the width σ of the Gaussian kernel or the degree d of the polynomial kernel for different data types and experiments. These results suggest that non-linear features extracted by KPCA could be potentially useful in identifying and discriminating between-group differences and changes in running gait patterns, and could provide useful information about the intrinsic non-linear dynamics of running movement.

Feature extraction and denoising using kernel PCA