Dimensionality reduction of large datasets with explicit feature maps

While explicit mapping is generally unknown for kernel data analysis, its inner product should be known. Although we proposed a kernel fuzzy c-means algorithm for data with tolerance, cluster centers and tolerance in higher dimensional space have not been seen. Contrary to this common assumption, explicit mapping has been introduced and the situation of kernel fuzzy c-means in higher dimensional space has been described via kernel principal component analysis using explicit mapping. In this paper, cluster centers and the tolerance of kernel fuzzy c-means for data with tolerance are described via kernel principal component analysis using explicit mapping.

Second-order Online Kernel Learning (OKL) has attracted considerable research interest due to its promising predictive performance in streaming environments. However, existing second-order OKL approaches suffer from at least quadratic time complexity with respect to the pre-set budget, rendering them unsuitable for large-scale datasets. Moreover, the singular value decomposition required to obtain explicit feature mapping is computationally expensive due to the complete decomposition process. To address these issues, we propose FORKS, a fast incremental matrix sketching and decomposition approach tailored for second-order OKL. FORKS constructs an incremental maintenance paradigm for second-order kernelized gradient descent, which includes incremental matrix sketching for kernel approximation and incremental matrix decomposition for explicit feature mapping construction. Theoretical analysis demonstrates that FORKS achieves a logarithmic regret guarantee on par with other second-order approaches while maintaining a linear time complexity w.r.t. the budget, significantly enhancing efficiency over existing methods. We validate the performance of our method through extensive experiments conducted on real-world datasets, demonstrating its superior scalability and robustness against adversarial attacks.

To improve smoke detection accuracy, we combine local binary pattern (LBP) like features, kernel principal component analysis (KPCA), and Gaussian process regression (GPR) to propose a novel data processing pipeline for smoke detection. The data processing pipeline consists of three steps including original feature extraction, dimensionality reduction, and classification. We use LBP-like methods to extract original features. To obtain a more discriminant feature, KPCA is used to non-linearly map the original features into a discriminant feature space, where manifold structures are embedded. Finally, in order to improve generalization performance, we apply GPR to model classification as a Gaussian process by imposing Gaussian priors on both data and hyper-parameters. In addition, we can replace any steps of the pipeline by similar methods for further improvement or exploration, so the pipeline is flexible and extensible. Experimental results show that KPCA and GPR are truly able to improve the performance of smoke detection and texture classification, and our method obviously outperforms the same features with Support Vector Machine (SVM).

Principal component analysis (PCA) has been proven to be an efficient method in dimensionality reduction, feature extraction and pattern recognition. Kernel principal component analysis (KPCA) can be considered as a natural nonlinear generalization of PCA, which performs linear PCA in a high dimensional space implicitly by using kernel trick. However, both conventional PCA and KPCA suffer from the deficiency of being sensitive to outliers. Existing robust KPCA has to eigen-decompose the gram matrix directly in each step and is much more computationally infeasible due to the large size of the matrix when the number of training samples is large. By extending existing robust PCA algorithm using kernel methods, we present a novel robust adaptive algorithm for calculating the kernel principal components. The proposed method not only preserves the characteristic of capturing underlying nonlinear structure of KPCA but also is robust against outliers by restraining the effect of outlying samples. Compared with existing robust KPCA methods, our method is performed without having to store the kernel matrix, which can reduce significantly the storage burden. In addition, our method shows the potential of expansibility to the incremental learning version. Experimental results on synthetic data indicate that our improved algorithm is effective and promising.

We propose a new kernel entropy component analysis approach to object recognition with image set by fusing the information entropy. Since the geometry of Symmetric Positive Definite (SPD) matrices, we model the image set with its covariance matrix (nonsingular). Thus the object recognition with image set can be model as classifying problem on the riemannian manifold space. Given the proper kernel function derived from efficient metric for SPD matrices, points lie on manifold space represented by covariance matrices can be cast into high dimensional euclidean space. In this euclidean space, traditional methods used for dimensional reduction can be applied directly. Accounting for the measurability to information of the Renyi entropy, we fuse the information entropy to the dimensional reduction progress of our method. Similar to Kernel Principal Component Analysis (KPCA), the method accomplish data dimensionality reduction by projection onto a subset of entropy preserving KPCA axes. But this subset does not need to correspond to the top eigenvalues of the kernel matrix. The positive results of experiment on datasets demonstrated effectiveness of our method.

Muscle fatigue is an inevitable phenomenon in athletic training, and its accurate assessment is crucial for preventing injuries. To address the issues of high redundancy in multi-channel surface electromyography (sEMG) features and the limitations of linear dimensionality reduction methods, this study proposes a muscle fatigue recognition model based on Kernel Principal Component Analysis (KPCA) and Support Vector Machine (SVM). sEMG signals were collected from 12 subjects performing sustained contractions. A comprehensive set of 48 time-domain, frequency-domain, and nonlinear features was extracted. KPCA was employed for nonlinear dimensionality reduction before classification. The resulting features were fed into an SVM classifier to distinguish between three fatigue states: relaxed, transitional fatigue, and fatigued. The model was evaluated using 10-fold cross-validation. Results demonstrated that the KPCA-SVM combination achieved the highest performance, with an average recognition accuracy of 91.5%, precision of 0.91, recall of 0.93, and F1-score of 0.92, outperforming other combinations of dimensionality reduction methods (MI, PCA) and classifiers (FLDA, KNN). This method provides an effective tool for the objective assessment of muscle fatigue in athletes.

Kernel principal component analysis (KPCA) has been effectively applied as an unsupervised non-linear feature extractor in many machine learning applications and suggested for various data stream classification tasks requiring a nonlinear transformation scheme to reduce dimensions. However, the dimensionality reduction ability is restricted because of KPCA’s high time complexity. So the practicality of KPCA on large datasets is rare. Therefore in this paper, we proposes a novel kind of incremental kernel principal component analysis algorithm: Data characteristic extraction based on IPCA algorithm –DCEIPCA, which allows efficient processing of large datasets and overcome the insufficient of KPCA. On the basis of DCEIPCA, we propose Classification expert system (CES) for intrusion detection system. Extensive experiments on KDDcup99 datasets confirm the superiority of Intrusion detection system based on CES(IDSCES) over other recent Intrusion detection system[4-11].

An indoor positioning optimization method based on dimensionality reduction and clustering fusion

Feature matching is a fundamental process in a variety of computer vision tasks. Beyond the standard L2 metric, various methods to measure similarity between features have been proposed mainly on the assumption that the features are defined in a histogram form. On the other hand, in a field of image quality assessment, SSIM [27] produces effective similarity between images, taking the place of L2 metric. In this paper, we propose a feature similarity measurement method based on the SSIM. Unlike the previous methods, the proposed method is built on not a histogram form but a tensor structure of a feature array extracted such as on spatial grids, in order to construct effective SSIMbased similarity measure of high robustness which is a key requirement in feature matching. In addition, we provide the explicit feature map such that the proposed similarity metric is embedded as a dot product. It contributes to significant speedup in similarity measurement as well as to feature transformation toward an effective vector form to which linear classifiers are directly applicable. In the experiments on various tasks, the proposed method exhibits favorable performance in both feature matching and classification.

Underground metal target detection technology has been widely applied in industrial production, resource exploration, and engineering construction, etc. However, due to the influence of non-negligible noise and high dimensionality in collected data, achieving efficient and accurate underground target classification remains a grand challenge for further applications of underground metal target detection on portable devices with limited computing capability and energy supply. This study aimed to seek out robust and efficient data-based strategies to classify the underground metal targets of different shapes and materials based on electromagnetic induction detection. We investigated thirty-three classification strategies based on eleven dimensionality reduction methods, namely, the least absolute shrinkage and selection operator (LASSO), genetic algorithm-support vector machine (GA-SVM), Pearson correlation coefficient (PCC), mutual information (MI), maximal relevance minimal redundancy Pearson correlation (mRMRP), maximal relevance minimal redundancy mutual information (mRMRMI), statistical features (SF), principal component analysis (PCA), kernel principal component analysis (KPCA), locally linear embedding (LLE), and stacked denoising autoencoder (SDAE), and three machine learning models, namely, artificial neural network (ANN), linear support vector machine (L-SVM), and Gaussian Naïve Bayes (GNB). Several parameters, including classification accuracy, the number of features after dimensionality, the feature type importance, and the time consumption were considered to evaluate the data-based classification strategies. Among the classification strategies investigated and considering the above evaluation parameters, the artificial neural network (ANN) classifier assisted with the kernel principal component analysis (KPCA) feature extraction method yielded the best performance in the material-based classification (accuracy:0.99) and the shape-based classification (accuracy:0.99). The locally linear embedding (LLE) improved the robustness of machine learning classifiers and efficiency of the artificial neural network in the material-based classification (improvement of average accuracy:0.17, reduction of classification time cost:14%) and shape-based classification (improvement of average accuracy:0.16, reduction of classification time cost:22%). Our comparative investigation provides a robust and efficient data-based strategy for underground metal target classification, which is significant for applications of underground metal target detection on portable devices with limited computing capability and energy supply. The cross-combination strategy of dimensionality reduction methods and machine learning models provides a way to find the optimal machine learning model for underground target detection.

A novel Gabor-Kernel face recognition method is proposed in this paper. This involves convolving a face image with a series of Gabor wavelets at different scales, locations, and orientations and extracting features from resulting Gabor filtered images. kernel discriminant analysis (KDDA) is then applied to the feature vectors for dimension reduction as well as class separability enhancement. A database of 600 frontal-view face images from the FERET face database is used to test the method. Experimental results demonstrate the advantage of KDDA over other Kernel methods such as kernel principal component analysis (KPCA) and general discriminant analysis (GDA). Significant improvements are also observed when features are extracted from Gabor filtered images instead of the original images. A 94% accuracy has been observed for the novel Gabor + KDDA method on the FERET database using a simple classifier, which could be further improved by employing a more complex classifier and distance measurer.

Web document ranking via active learning and kernel principal component analysis

Kernel Entropy Component Analysis (KECA) is a newer method than Kernel Principle Component Analysis (KPCA) for data transformation and dimensionality reduction in case of face recognition. Although in almost all previous researches using KECA are shown to be more superior and more appropriate method compared to KPCA, here in this paper the significance of Kernel PCA in handling face pose in surveillance images is compared to KECA. Comparative analysis is made to signify the importance of Kernel Principle Component Analysis in terms of pose invariant face recognition in surveillance.

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

In this study, a subspace detection technique is developed using a hybrid approach which combines both feature extraction and feature selection for the task of hyperspectral image classification. The proposed approach applies Kernel Principal Component Analysis (KPCA) at the first step, then feature selection from the KPCA images is accomplished by combining the KPCA score with a Jeffries-Matusita (JM) distance based ranking score. Experimental analysis is carried out on a hyperspectral image acquired by the AVIRIS sensor and the results show the advantage of the proposed approach in terms of classification accuracy in the tested cases.