Face recognition using kernel principal component analysis and genetic algorithms

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

Classification between individuals with mild cognitive impairment (MCI) and healthy controls (HC) based on electroencephalography (EEG) has been considered a challenging task to be addressed for the purpose of its early detection. In this study, we proposed a novel EEG feature, the kernel eigen-relative-power (KERP) feature, for achieving high classification accuracy of MCI versus HC. First, we introduced the relative powers (RPs) between pairs of electrodes across 21 different subbands of 2-Hz width as the features, which have not yet been used in previous MCI-HC classification studies. Next, the Fisher’s class separability criterion was applied to determine the best electrode pairs (five electrodes) as well as the frequency subbands for extracting the most sensitive RP features. The kernel principal component analysis (kernel PCA) algorithm was further performed to extract a few more discriminating nonlinear principal components from the optimal RPs, and these components form a KERP feature vector. Results carried out on 51 participants (24 MCI and 27 HC) show that the newly introduced subband RP feature showed superior classification performance to commonly used spectral power features, including the band power, single-electrode relative power, and also the RP based on the conventional frequency bands. A high leave-one-participant-out cross-validation (LOPO-CV) classification accuracy 86.27% was achieved by the RP feature, using a simple linear discriminant analysis (LDA) classifier. Moreover, with the same classifier, the proposed KERP further improved the accuracy to 88.24%. Finally, cascading the KERP feature to a nonlinear classifier, the support vector machine (SVM), yields a high MCI-HC classification accuracy of 90.20% (sensitivity = 87.50% and specificity = 92.59%). The proposed method demonstrated a high accuracy and a high usability (only five electrodes are required), and therefore, has great potential to further develop an EEG-based computer-aided diagnosis system that can be applied for the early detection of MCI.

Kernel principal component analysis (KPCA) has been effectively applied as an unsupervised non-linear feature extractor in many machine learning applications and suggested for various data stream classification tasks requiring a nonlinear transformation scheme to reduce dimensions. However, the dimensionality reduction ability is restricted because of KPCA’s high time complexity. So the practicality of KPCA on large datasets is rare. Therefore in this paper, we proposes a novel kind of incremental kernel principal component analysis algorithm: Data characteristic extraction based on IPCA algorithm –DCEIPCA, which allows efficient processing of large datasets and overcome the insufficient of KPCA. On the basis of DCEIPCA, we propose Classification expert system (CES) for intrusion detection system. Extensive experiments on KDDcup99 datasets confirm the superiority of Intrusion detection system based on CES(IDSCES) over other recent Intrusion detection system[4-11].

Wireless sensor network is a kind of brand-new information acquisition platform, which is realized by the introduction of self-organizing and auto-configuration mechanisms. Node localization technology represents a crucial component of wireless sensor network. In this paper, a localization method based on kernel principal component analysis and particle swarm optimization back propagation algorithm is carefully discussed. First of all, taking KPCA as the front-end system to extract the main components of the localization information, and then regarding the nonlinear principal components extracted from distance vectors as the input samples, and meanwhile taking the coordinates of vertices in addition to the region boundary as the output samples, the PSO-BP neural network is trained to achieve the localization model. Finally the localization of unknown nodes can be estimated. The simulation experiment result showed that the method has high ability of stability and precision, and meets the practical need of localization.

Machine learning and pattern recognition recently become a hot topic in computing world. This is due to the fast-growing of resources as well as techniques that make it easier to solve machine learning and pattern recognition problems. Problems that require machine learning solutions may be very simple for humans but actually can be very complex for machines to solve them. Face recognition is amongst those problems. Almost all human can easily recognize others without require specific knowledge to do it, different from machines which require its. This paper discussed face recognition task using machine learning strategies which involved Kernel Principal Component Analysis (KPCA) and Support Vector Machine (SVM) to identify person. KPCA extracted features from 2D image input and produced the important features of an image input. The extracted face features are recognized by SVM by classifying human face according to their stored identity in a database. SVM, which was basically a binary classifier worked by using one-against-one strategy to compare the face feature vector of a single test image to the stored face image in a face image database. Experiment results on grayscale images with size 92x112 pixels gave 96.25% of accuracy rate. Hence, KPCA and SVM for face recognition is a robust machine learning method.

Support Vector Machines and Kernel methods have become very popular as methods for learning from examples. In this paper, in order to solve the non-linear and the time of recognition problem, recent advanced techniques are presented. For example, Kernel Principal Components and Proximal Support Vector Machines, Kernel Principal Components Analysis can describe multiple correlations between pixels in the image recognition, and can handle the high order statistics of the original data, at the same time the nonlinear features of the image can be preferably extracted. Proximal Support Vector Machines is an improvement for these characters of Support Vector Machines, combines the proximal algorithm with Support Vector Machines, has a better ability of nonlinear mapping and a stronger generalization capability, and the pace of Proximal Support Vector Machines is very enhanced. In this paper, the computer simulation was progressed based on ORL face database, the effectiveness of the Proximal Support Vector Machines algorithm and Kernel Principal Components Analysis algorithm were shown by experimental results.

In this paper, a novel HVAC fan machinery fault recognition method combining kernel principal component analysis (KPCA) and support vector machine (SVM) is proposed. KPCA is an improved PCA, which possesses the property of extracting optimal features by adopting a nonlinear kernel function method. Support vector machine (SVM) is a novel approach based on statistical learning theory, which has emerged for feature identification and classification. An integrated method is applied for HVAC fan machinery status monitoring and fault diagnosis, which combines KPCA for fault feature extraction and multiple SVMs (MSVMs) for identification of different fault sources. The experimental results show that KPCA based on LS-SVM has a higher correct recognition rate, and a faster computational speed.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

To address the challenges of difficult feature extraction and suboptimal parameter configuration for cyclotron ion source fault diagnosis in complex environments, this study proposes an intelligent diagnostic framework integrating Kernel Principal Component Analysis (KPCA), an Improved Sparrow Search Algorithm (ISSA), and a Support Vector Machine (SVM). The KPCA algorithm is employed for dimensionality reduction to handle the highly nonlinear nature of fault data. Regarding algorithmic evolution, the basic SSA is enhanced by integrating dynamic weights, opposition-based learning, and Cauchy mutation strategies, which effectively overcome the diagnostic bottlenecks inherent in cyclotron scenarios. Furthermore, the ISSA facilitates the global adaptive optimization of key SVM parameters, eliminating the stochasticity of empirical tuning and fundamentally enhancing the model's robustness. Experimental results across 30 independent tests demonstrate that the KPCA-ISSA-SVM model achieves an average accuracy of 97.6% in multi-class fault detection. Compared with other classic diagnostic models, the proposed framework exhibits superior precision and stability, providing an effective technical approach with significant engineering value for the precise monitoring of ion source statuses.

The inherent physical characteristics of many real-life phenomena, including biological and physiological aspects, require adapted nonlinear tools. Moreover, the additive nature in some situations involve solutions expressed as positive combinations of data. In this paper, we propose a nonlinear feature extraction method, with a non-negativity constraint. To this end, the kernel principal component analysis is considered to define the most relevant features in the reproducing kernel Hilbert space. These features are the nonlinear principal components with high-order correlations between input variables. A pre-image technique is required to get back to the input space. With a non-negative constraint, we show that one can solve the pre-image problem efficiently, using a simple iterative scheme. Furthermore, the constrained solution contributes to the stability of the algorithm. Experimental results on event-related potentials (ERP) illustrate the efficiency of the proposed method.

Condition monitoring of wind turbine based on deep learning networks and kernel principal component analysis

As a valid statistical tool, principal component analysis (PCA) has been widely used in industrial process monitoring. But due to its intrinsic linear character, it performs badly in nonlinear process monitoring. Kernel PCA (KPCA) can extract useful information in nonlinear data. However KPCA-based monitoring is not suitable for on-line monitoring because of large calculation and much memory occupation. The paper introduces an on-line monitoring method based on KPCA and neural network (NN), where KPCA is used to extract nonlinear principal components (PCs) and then NN approximates the relationship between process data and nonlinear PCs. We can obtain nonlinear PCs by NN to compute the monitoring indices and then achieve the on-line monitoring. The case study shows the validity of the method.KeywordsNeural NetworkKernel Principal Component AnalysisMonitoring IndexMemory OccupationComplex Industrial ProcessThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Traditional face recognition methods such as Principal Components Analysis(PCA), Independent Component Analysis(ICA) and Linear Discriminant Analysis(LDA) are linear discriminant methods, but in the real situation, a lot of problems can't be linear discriminated; therefore, researchers proposed face recognition method based on kernel techniques which can transform the nonlinear problem of inputting space into the linear problem of high dimensional space. In this paper, we propose a recognition method based on kernel function which combines kernel Fisher Discriminant Analysis(KFDA) with kernel Principle Components Analysis(KPCA) and use typical ORL(Olivetti Research Laboratory) face database as our experimental database. There are four key steps: constructing feature subspace, image projection, feature extraction and image recognition. We found that the recognition accuracy has been greatly improved by using nonlinear identification method and combined feature extraction methods. We use MATLAB software as the platform, and use the GUI to demonstrate the process of face recognition in order to achieving human-computer interaction and making the process and result more intuitive.

Manufacturing industries collect massive amounts of multivariate measurement through automated inspection processes. Noisy measurements and high-dimensional, irrelevant features make it difficult to identify useful patterns in the data. Principal component analysis provides linear summaries of datasets with fewer latent variables. Kernel Principal Component Analysis (KPCA), however, identifies nonlinear patterns. One challenge in KPCA is to inverse map the denoised signal from a high-dimensional feature space into its preimage in input space to visualize the nonlinear variation sources. However, such an inverse map is not always defined. This article provides a new meta-method applicable to any KPCA algorithm to approximate the preimage. It improves upon previous work where a strong assumption was the availability of noise-free training data. This is problematic for applications such as manufacturing variation analysis. To attenuate noise in kernel subspace estimation the final preimage is estimated as the average from bagged samples drawn from the original dataset. The improvement is most pronounced when the parameters differ from those that minimize the error rate. Consequently, the proposed approach improves the robustness of any base KPCA algorithm. The usefulness of the proposed method is demonstrated by analyzing a classic handwritten digit dataset and a face dataset. Significant improvement over the existing methods is observed.

Automated classification of magnetic resonance brain images (MRIs) is a hot topic in the field of medical and biomedical imaging. Various methods have been suggested recently to improve this technology. In this paper, to reduce the complexity involved in the medical images and to ameliorate the classification of MRIs, a novel 3D magnetic resonance (MR) brain image classifier using kernel principal component analysis (KPCA) and support vector machines (SVMs) is proposed. Experiments are carried out using A deep multiple kernel SVM (DMK-SVM) and a regular SVM. An algorithm entitled SVM–KPCA is put forward. Its main task is to classify a brain MRI as a normal brain image or as a pathological brain image. This algorithm, firstly, adopts the discrete wavelet transform technique to extract features from images. Secondly, KPCA is applied to decrease the dimensionality of features. SVM is then applied to the reduced data. A K-fold cross-validation strategy is used to avoid overfitting and to ameliorate the generalization of the SVM–KPCA algorithm. Three databases are used to validate the suggested SVM–KPCA method. Three conclusions are obtained from this work. First, KPCA is highly efficient in increasing the classifier’s performance compared with similar algorithms working on the proposed database. Second, the SVM–KPCA algorithm performs well in differentiating between two classes of medical images. Third, the approach is robust and might be utilized for other MRIs. This proposes a significant role for computer aided diagnosis analysis systems used for clinical practice.