Face Recognition Technology Based on Kernel Principal Components and Proximal Support Vector Machines

ABSTRACTOne of the major issues in medical field constitutes the correct diagnosis, including the limitation of human expertise in diagnosing the disease in a manual way. Nowadays, the use of machine learning classifiers, such as support vector machines (SVM), in medical diagnosis is increasing gradually. However, traditional classification algorithms can be limited in their performance when they are applied on highly imbalanced data sets, in which negative examples (i.e. negative to a disease) outnumber the positive examples (i.e. positive to a disease). SVM constitutes a significant improvement and its mathematical formulation allows the incorporation of different weights so as to deal with the problem of imbalanced data. In the present work an extensive study of four medical data sets is conducted using a variant of SVM, called proximal support vector machine (PSVM) proposed by Fung and Mangasarian [9]. Additionally, in order to deal with the imbalanced nature of the medical data sets we applied both a variant of SVM, referred as two-cost support vector machine and a modification of PSVM referred as modified PSVM. Both algorithms incorporate different weights one for each class examples.

A novel Gabor-Kernel face recognition method is proposed in this paper. This involves convolving a face image with a series of Gabor wavelets at different scales, locations, and orientations and extracting features from resulting Gabor filtered images. kernel discriminant analysis (KDDA) is then applied to the feature vectors for dimension reduction as well as class separability enhancement. A database of 600 frontal-view face images from the FERET face database is used to test the method. Experimental results demonstrate the advantage of KDDA over other Kernel methods such as kernel principal component analysis (KPCA) and general discriminant analysis (GDA). Significant improvements are also observed when features are extracted from Gabor filtered images instead of the original images. A 94% accuracy has been observed for the novel Gabor + KDDA method on the FERET database using a simple classifier, which could be further improved by employing a more complex classifier and distance measurer.

The focus is made to develop predictor models for wind speed prediction employing the support vector machine neural models. Basically, support vector machines (SVM) is employed as classifiers, but this contribution models variant of SVM to act as predictors. A developed model of linear support vector machine (LSVM) and proximal support vector machine (PSVM) is proposed to carry out the wind speed prediction using the available real time wind farm data. In developed PSVM predictor, it is modeled for both linear PSVM predictor and non-linear PSVM predictor. The difference between the developed linear and non-linear PSVM predictor models lies in their applicability of kernel functions to perform effective wind speed prediction. The prediction application is implemented for the set of wind farm data with a wind mill height of 50 m in a manner to minimize the mean square error. The training process of the neural network algorithmic flow is done with the developed LSVM, L-PSVM (Linear PSVM) and N-PSVM (nonlinear PSVM) for predicting the wind speed in renewable energy systems. Results computed are compared with the other types of predictors to prove the effectiveness of the proposed variants of SVM predictors. The simulated results presents the effectiveness of the proposed predictors for the real time wind farm data and the applicability of the predictors for the considered datasets.

Aimed at the non-stationary characteristics of rolling bearing vibration signal,a fault diagnosis method was proposed based on local-wave method and KPCA(kernel principal component analysis)-LSSVM(least squares support vector machine).Firstly,local wave decomposition was used to decompose rolling bearing vibration signal into several intrinsic mode function(IMF),whose feature energy,singular values and AR model parameters were computed as initial feature vectors.Secondly,ini tial feature vectors were mapped into a higher-dimensional space with KPCA,and the kemel principal components were extracted as new feature vectors,which used as the input of LSSVM for fault classification.The experimental results show the KPCA-LSSVM method improves LSSVM's classification performance by KPCA obtaining additional discriminative information,and has better generalization than direct LSSVM method,and can identify rolling bearing fault patterns more accurately.

Machine learning and pattern recognition recently become a hot topic in computing world. This is due to the fast-growing of resources as well as techniques that make it easier to solve machine learning and pattern recognition problems. Problems that require machine learning solutions may be very simple for humans but actually can be very complex for machines to solve them. Face recognition is amongst those problems. Almost all human can easily recognize others without require specific knowledge to do it, different from machines which require its. This paper discussed face recognition task using machine learning strategies which involved Kernel Principal Component Analysis (KPCA) and Support Vector Machine (SVM) to identify person. KPCA extracted features from 2D image input and produced the important features of an image input. The extracted face features are recognized by SVM by classifying human face according to their stored identity in a database. SVM, which was basically a binary classifier worked by using one-against-one strategy to compare the face feature vector of a single test image to the stored face image in a face image database. Experiment results on grayscale images with size 92x112 pixels gave 96.25% of accuracy rate. Hence, KPCA and SVM for face recognition is a robust machine learning method.

Proximal support vector machines (PSVMs) are support vector machines (SVMs) version which involve equality instead of inequality constraints assigning data points to one of two classes, and have a significant speedup in comparison with traditional SVM-type classifiers. However, PSVMs are very sensitive to outliers or noises because of overfitting problem. In this paper, a robust proximal support vector machines (RPSVMs) is proposed to deal with the problem of overfitting in PSVMs for two-class classification. In experiments, two methods are used to give membership values to training data points according to their relative importance in the training set. Results show that the proposed RPSVMs actually reduce the effect of outliers and yield higher classification rate than PSVMs do

Soft computing approach to fault diagnosis of centrifugal pump

Support vector machine (SVM) is a supervised machine learning approach that was recognized as a statistical learning apotheosis for the small-sample database. SVM has shown its excellent learning and generalization ability and has been extensively employed in many areas. This paper presents a performance analysis of six types of SVMs for the diagnosis of the classical Wisconsin breast cancer problem from a statistical point of view. The classification performance of standard SVM (St-SVM) is analyzed and compared with those of the other modified classifiers such as proximal support vector machine (PSVM) classifiers, Lagrangian support vector machines (LSVM), finite Newton method for Lagrangian support vector machine (NSVM), Linear programming support vector machines (LPSVM), and smooth support vector machine (SSVM). The experimental results reveal that these SVM classifiers achieve very fast, simple, and efficient breast cancer diagnosis. The training results indicated that LSVM has the lowest accuracy of 95.6107 %, while St-SVM performed better than other methods for all performance indices (accuracy = 97.71 %) and is closely followed by LPSVM (accuracy = 97.3282). However, in the validation phase, the overall accuracies of LPSVM achieved 97.1429 %, which was superior to LSVM (95.4286 %), SSVM (96.5714 %), PSVM (96 %), NSVM (96.5714 %), and St-SVM (94.86 %). Value of ROC and MCC for LPSVM achieved 0.9938 and 0.9369, respectively, which outperformed other classifiers. The results strongly suggest that LPSVM can aid in the diagnosis of breast cancer.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

There is a trend in recent machine learning community to construct a nonlinear version of a linear algorithm using the 'kernel method', e.g. Support Vector Machines (SVMs), kernel principal component analysis, kernel fisher discriminant analysis and the recent kernel clustering algorithms. In unsupervised clustering algorithms using kernel method, typically, a nonlinear mapping is used first to map the data into a potentially much higher feature space, where clustering is then performed. A drawback of these kernel clustering algorithms is thatthe clustering prototypes lie in high dimensional feature space and hence lack clear and intuitive descriptions unless using additional projection approximation from the feature to the data space as done in the existing literatures. In this paper, a novel clustering algorithm using the 'kernel method' based on the classical fuzzy clustering algorithm (FCM) is proposed and called as kernel fuzzy c-means algorithm (KFCM). KFCM adopts a new kernel-induced metric in the data space to replace the original Euclidean norm metric in FCM and the clustered prototypes still lie in the data space so that the clustering results can be reformulated and interpreted in the original space. Our analysis shows that KFCM is robust to noise and outliers and also tolerates unequal sized clusters. And finally this property is utilized to cluster incomplete data. Experiments on two artificial and one real datasets show that KFCM hasbetter clustering performance and more robust than several modifications of FCM for incomplete data clustering.

The aim of this letter is to propose a theory of deep restricted kernel machines offering new foundations for deep learning with kernel machines. From the viewpoint of deep learning, it is partially related to restricted Boltzmann machines, which are characterized by visible and hidden units in a bipartite graph without hidden-to-hidden connections and deep learning extensions as deep belief networks and deep Boltzmann machines. From the viewpoint of kernel machines, it includes least squares support vector machines for classification and regression, kernel principal component analysis (PCA), matrix singular value decomposition, and Parzen-type models. A key element is to first characterize these kernel machines in terms of so-called conjugate feature duality, yielding a representation with visible and hidden units. It is shown how this is related to the energy form in restricted Boltzmann machines, with continuous variables in a nonprobabilistic setting. In this new framework of so-called restricted kernel machine (RKM) representations, the dual variables correspond to hidden features. Deep RKM are obtained by coupling the RKMs. The method is illustrated for deep RKM, consisting of three levels with a least squares support vector machine regression level and two kernel PCA levels. In its primal form also deep feedforward neural networks can be trained within this framework.

In kernel methods such as kernel principal component analysis (PCA) and support vector machines, the so called kernel trick is used to avoid direct calculations in a high (virtually infinite) dimensional kernel space. In this brief, based on the fact that the effective dimensionality of a kernel space is less than the number of training samples, we propose an alternative to the kernel trick that explicitly maps the input data into a reduced dimensional kernel space. This is easily obtained by the eigenvalue decomposition of the kernel matrix. The proposed method is named as the nonlinear projection trick in contrast to the kernel trick. With this technique, the applicability of the kernel methods is widened to arbitrary algorithms that do not use the dot product. The equivalence between the kernel trick and the nonlinear projection trick is shown for several conventional kernel methods. In addition, we extend PCA-L1, which uses L1-norm instead of L2-norm (or dot product), into a kernel version and show the effectiveness of the proposed approach.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

Proximal support vector machine is a variation of standard support vector machine and can be trained extremely efficiently for binary classification. However in many application fields, multi-class classification and incremental learning must be supported. Incremental linear proximal support vector classifier for multi-class classification has been developed in recent years, but only its performance in one-against-all manner has been investigated, and the application of proximal support vector machine for nonlinear multi-class classification has not been studied. In order to apply proximal support vector machine to more fields, three multi-class classification policies (one-against-all, one-against-one, DAGSVM) applied to incremental linear proximal support vector classifier are compared and incremental nonlinear proximal support vector classifier for multi-class classification based on Gaussian kernel is investigated in the paper. The experiments indicate that one-against-all policy is best for incremental linear proximal support vector classifier according to the tradeoff between computing complexity and correctness, and the introduced incremental nonlinear proximal support vector classifier is effective in one-against-all manner when the reduce rate is below 0.6.

This thesis presents a critical appraisal of several differing approaches to the design and testing of fault detection (FD) algorithms monitoring the instrumentation used in the alternated aerobic/anoxic cycles (AC) process for nitrogen removal. Several features are selected as the basis of the FD, involving the slope and the timing of the process measurements of the various nitrogen compounds. Two separate FD algorithms have been developed for the anoxic and the aerobic phases, requiring a separate tuning but sharing the same principles: first some low-level checks are performed on the raw signals, discriminating gross malfunctions like missing data and spikes, then more sophisticated methods are used to investigate the presence of more subtle anomalies that were not detected by the previous screening. The FD problem is treated either in terms of classification problem, testing different algorithms such as binary trees, support vector machines (SVM) and principal component analysis (PCA), or as a forecasting one, using the Bayesian theory to predict the faulty or normal state of the process based on the previous records. An operational data set obtained from a municipal plant was used to first train the algorithm. However, due to the fairly limited information which could be extracted, a more comprehensive data set was created building an AC model based on the standard Benchmark simulation model with improved nitrogen kinetics and seasonal temperature variations. Detailed sensor models were also included, so that the occurrence of faults could be totally controlled, both in kind and timing. The performances of the various methods on either the operational and the synthetic datasets have been assessed comparing the anomalies detected by the methods with those actually observed. While the great majority of the gross faults is successfully detected by the preliminary screening, differing performances of the subsequent finer detection are obtained, depending on both the quality of data set and the detection method used: poorer results are observed using the plant data, in part due to an insufficient characterization of the fault events and in part due to the limited number of signals monitored. The higher availability of measurements provided by the numerical model, instead, enhances the discrimination capabilities of the tested methods, especially the nonlinear SVM, while the PCA-based approach and the Bayesian predictor results less affected by a change in the combination of diagnostic parameters used.