An Iterative Algorithm for Robust Kernel Principal Component Analysis

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

A novel Gabor-Kernel face recognition method is proposed in this paper. This involves convolving a face image with a series of Gabor wavelets at different scales, locations, and orientations and extracting features from resulting Gabor filtered images. kernel discriminant analysis (KDDA) is then applied to the feature vectors for dimension reduction as well as class separability enhancement. A database of 600 frontal-view face images from the FERET face database is used to test the method. Experimental results demonstrate the advantage of KDDA over other Kernel methods such as kernel principal component analysis (KPCA) and general discriminant analysis (GDA). Significant improvements are also observed when features are extracted from Gabor filtered images instead of the original images. A 94% accuracy has been observed for the novel Gabor + KDDA method on the FERET database using a simple classifier, which could be further improved by employing a more complex classifier and distance measurer.

Subspace methods that utilize principal component analysis (PCA) are widely used for pattern classification or detection problems. Kernel PCA (KPCA) that is an extension of PCA is also applied to subspace methods. However, its computational cost is very high since the computational cost mainly depends on the number of samples in kernel methods. Recently, subset KPCA (SKPCA) has been proposed in order to reduce its computational complexity. In this paper, we apply SKPCA to subspace methods, and compare SKPCA with KPCA using some sample selection methods. Experimental results demonstrate advantages of subspace methods using SKPCA.

Kernel discriminant analysis (KDA) and the kernel principal component analysis (KPCA), which are the extension of the linear discriminant analysis (LDA) and the principal component analysis (PCA), respectively, from linear domain to nonlinear domain via the kernel trick, are two very popular nonlinear feature extraction methods. In this paper, we present a new feature extraction algorithm by combing KDA and KPCA, and then apply it to the face recognition task. The experimental results on Yale face dataset show that the proposed method can significantly improve the performance both KDA and KPCA.

In kernel methods such as kernel principal component analysis (PCA) and support vector machines, the so called kernel trick is used to avoid direct calculations in a high (virtually infinite) dimensional kernel space. In this brief, based on the fact that the effective dimensionality of a kernel space is less than the number of training samples, we propose an alternative to the kernel trick that explicitly maps the input data into a reduced dimensional kernel space. This is easily obtained by the eigenvalue decomposition of the kernel matrix. The proposed method is named as the nonlinear projection trick in contrast to the kernel trick. With this technique, the applicability of the kernel methods is widened to arbitrary algorithms that do not use the dot product. The equivalence between the kernel trick and the nonlinear projection trick is shown for several conventional kernel methods. In addition, we extend PCA-L1, which uses L1-norm instead of L2-norm (or dot product), into a kernel version and show the effectiveness of the proposed approach.

Systematic comparison of unsupervised feature extraction methods for enhancing discrimination accuracy of brown rice flour adulteration in LIBS

The identification of between-group differences and changes in gait mechanics are useful for injury prevention. Previous studies suggest the differences in gait biomechanical variables may interact in a complex non-linear fashion rather than a simple linear fashion. A non-linear multivariate analysis technique is therefore necessary to unravel the inherent structure in the gait data. Kernel principal component analysis (KPCA) is an extension of principal component analysis (PCA), the most widely used method in this field, and can provide inside into non-linear relationships of the variables of interest. Despite the growing use of PCA in running gait research, no prior studies have evaluated the utility of KPCA in determining the between-group differences in running gait patterns. Therefore, the objective of this study was to compare the performance of KPCA and PCA in identifying differences between male and female runners and as well between young and older runners in running gait kinematics. Running kinematic data were analysed on a gender group (female, n = 50; male, n = 50) and an age group (young, n = 50; older, n = 50) for discrete variables and waveform data. The following key results were obtained: (1) the performance of the KPCA for the identification of gender and age differences in running gait kinematics increased as compared to using the linear PCA; (2) the Gaussian function often performed better than the polynomial function for these two experiments; (3) there was no consistent optimal value of either the width σ of the Gaussian kernel or the degree d of the polynomial kernel for different data types and experiments. These results suggest that non-linear features extracted by KPCA could be potentially useful in identifying and discriminating between-group differences and changes in running gait patterns, and could provide useful information about the intrinsic non-linear dynamics of running movement.

A kernel based nonlinear subspace projection (KNSP) method is proposed for reduction of hyperspectral image dimensionality. This method involves three steps: subspace partition of full data space, feature extraction based on kernel principal component analysis (KPCA) in subspace and feature selection based on class separability criterion. The main merit of the proposed method is that it is more suitable for feature extraction than linear principal component analysis (PCA) and segmented principal component: transformation (SPCT), in particular, when hyperspectral data have nonlinear characteristics. In order to testify the effectiveness of the KNSP method for reduction of hyperspectral image dimensionality, hyperspectral image classification is performed on AVIRIS data. Experimental results show that when the hyperspectral dimensionality is reduced to a few features, the average classification accuracy of the new method is higher than those of PCA and SPCT methods.

An approach to Magnetic Resonance (MR) image reconstruction from undersampled data is proposed. Undersampling artifacts are removed using an iterative thresholding algorithm applied to nonlinearly transformed image block arrays. Each block array is transformed using kernel principal component analysis where the contribution of each image block to the transform depends in a nonlinear fashion on the distance to other image blocks. Elimination of undersampling artifacts is achieved by conventional principal component analysis in the nonlinear transform domain, projection onto the main components and back-mapping into the image domain. Iterative image reconstruction is performed by interleaving the proposed undersampling artifact removal step and gradient updates enforcing consistency with acquired k-space data. The algorithm is evaluated using retrospectively undersampled MR cardiac cine data and compared to k-t SPARSE-SENSE, block matching with spatial Fourier filtering and k-t ℓ1-SPIRiT reconstruction. Evaluation of image quality and root-mean-squared-error (RMSE) reveal improved image reconstruction for up to 8-fold undersampled data with the proposed approach relative to k-t SPARSE-SENSE, block matching with spatial Fourier filtering and k-t ℓ1-SPIRiT. In conclusion, block matching and kernel methods can be used for effective removal of undersampling artifacts in MR image reconstruction and outperform methods using standard compressed sensing and ℓ1-regularized parallel imaging methods.

The aim of this paper applies greedy kernel principal component analysis (greedy kernel PCA) to deal with training data reduction and nonlinear feature extraction in classification. Kernel PCA is a nonlinear extension of linear PCA. It shows a powerful nonlinear feature extraction technique via kernel trick. A disadvantage of kernel PCA, however, is that the storage of training data in terms of the dot products is too expensive since the size of kernel matrix increases quadratically with the number of training data. So, a more efficient feature extraction method, greedy kernel PCA, is proposed to reduce training data and nonlinear feature extraction for classification. The reduced set method aims to find a new kernel expansion and well approximates the original training data. Simulation results show both kernel PCA and greedy kernel PCA are more superior to linear PCA in feature extraction. Greedy kernel PCA will tend towards kernel PCA feature extraction as more percentage of training data is included in the reduced set, whilst greedy kernel PCA results in lower evaluation cost due to the reduced training set. The experiments show also that greedy kernel PCA can significantly reduce the complexity while retaining their accuracy in classification.

Principal Components Analysis (PCA) is used to classify the running condition of a machine by means of projecting the original data to the Principal Components space. However, if the data are concentrated in a nonlinear subspace, PCA will fail to work well. Kernel Principal Components Analysis (KPCA) transforms the input data from the original input space into a higher dimensional feature space with the nonlinear mapping, and then uses the nonlinear principal components to realize the classification. In this paper a case of gear fault diagnosis was studied with KPCA. The characteristic values of frequent domain from vibration signals of the gearbox under the running condition were extracted, and the KPCA method was used to classify gear crack fault. The result shows that KPCA is more effective to distinguish the state of the gear and more suitable to diagnose the gear faults in early stage.

Support Vector Machines and Kernel methods have become very popular as methods for learning from examples. In this paper, in order to solve the non-linear and the time of recognition problem, recent advanced techniques are presented. For example, Kernel Principal Components and Proximal Support Vector Machines, Kernel Principal Components Analysis can describe multiple correlations between pixels in the image recognition, and can handle the high order statistics of the original data, at the same time the nonlinear features of the image can be preferably extracted. Proximal Support Vector Machines is an improvement for these characters of Support Vector Machines, combines the proximal algorithm with Support Vector Machines, has a better ability of nonlinear mapping and a stronger generalization capability, and the pace of Proximal Support Vector Machines is very enhanced. In this paper, the computer simulation was progressed based on ORL face database, the effectiveness of the Proximal Support Vector Machines algorithm and Kernel Principal Components Analysis algorithm were shown by experimental results.

Principal component analysis (PCA) is linear method for feature extraction that is known as Karhonen Loove method. PCA was first proposed to recognize face by Turk and Pentland, and was also known as eigenface in 1991 [Turk, 1991]. However, PCA has some weaknesses. The first, it cannot capture the simplest invariance of the face image [Arif et al., 2008b] , when this information is not provided in the training data. The last, the result of feature extraction is global structure [Arif, 2008]. The PCA is very simple, has overcome curse of dimensionality problem, this method have been known and expanded by some researchers to recognize face such as Linear Discriminant Analysis (LDA)[Yambor, 2000; A.M. Martinez, 2003; J.H.P.N. Belhumeur 1998], Linear Preserving Projection that known Lapalacianfaces [Cai, 2005; Cai et al, 2006; Kokiopoulou, 2004; X. He et al., 2005], Independent Component Analysis, Kernel Principal Component Analysis [Scholkopf et al., 1998; Sch olkopf 1999], Kernel Linear Discriminant Analysis (KLDA) [Mika, 1999] and maximum feature value selection of nonlinear function based on Kernel PCA [Arif et al., 2008b]. As we know, PCA is dimensionality reduction method based on object appearance by projecting an original ndimensional (row*column) image into k eigenface where k<<n. Although PCA have been developed into some methods, but in some cases, PCA can outperform LDA, LPP and ICA when it uses small sample size.

Kernel Principal Component Analysis (KPCA) is a widely used technique in the dimension reduction, de-noising and discovering nonlinear intrinsic dimensions of data set. In this paper we describe a reweighing kernel-based classification method for improving recognition problem. Firstly, we map the training samples to the feature space by non-linear transformation, and then perform principal component analysis(PCA) using the selected kernel function in the feature space, and get the linear representation of testing samples in the feature space. Secondly, by using the idea of reweighting, we select the similarity between testing sample and each training sample as the weight of reweighting, then take the final weight as the criteria of classification. The experimental results demonstrate that our method is more accurate than Support Vector Machine (SVM) classification method and Linear Discriminant Analysis (LDA) classification. In addition, the number of training samples that our method need is much smaller than some other methods.

A new method for satellite control system fault pattern recognition is put forward, the most important components of which is feature extraction. Kernel principal component analysis is adopted to work out the most effective feature for fault detection by building a accurate principal component model, and this model makes the feature extraction simple and declines the dimensions of feature space. Furthermore in order to recognize the fault pattern a multi classification support vector machine (SVM) is worked out, and the particle swarm optimization algorithm is used to optimize the parameters of it. The results of simulation indicate that this method can not only detect the fault but also recognize the fault pattern.