Kernel principal subspace Mahalanobis distances for outlier detection

Outlier detection is the task of classifying test data that differ in some respect from the data that are available during training. This may be seen as one-class classification, in which a model is constructed to describe normal training data. In wireless sensor networks (WSNs), the outlier detection process is a necessary step in building sensor network systems to assure data quality for perfect decision making. For this case, the task amounts to create a useful model based on kernel principal component analysis (KPCA) to recognize data as normal or outliers. Recently, KPCA has used for nonlinear case which can extract higher order statistics. KPCA mapping the data onto another feature space and using nonlinear function. On account of the attractive capability, KPCA-based methods have been extensively investigated, and it have showed excellent performance. Within this setting, we propose KPCA-based Mahalanobis kernel as a new outlier detection method using Mahalanobis distance to implicitly calculate the mapping of the data points in the feature space so that we can separate outlier points from normal pattern of data distribution. The use of KPCA-based Mahalanobis kernel on real-word data obtained from Intel Berkeley are reported showing that the proposed method performs better in finding outliers in WSNs when compared with the original reconstruction error-based variant and the one-class support vector machine detection approach. All computation are done in the original space, thus saving computing time using Mahalanobis kernel.

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

In kernel methods, choosing a suitable kernel is in dispensable for favorable results. No well-founded methods, however, have been established in general for unsupervised learning. We focus on kernel Princ ipal Component Analysis (kernel PCA), which is a nonlinear extension of principal component analysis and ha s been used electively for extracting nonlinear featu res and reducing dimensionality. As a kernel method , kernel PCA also suffers from the problem of kernel choice. Although cross-validation is a popular meth od for choosing hyperparameters, it is not applicable straightforwardly to choose a kernel in kernel PCA because of the incomparable norms given by differen t kernels. It is important, thus, to develop a well founded method for choosing a kernel in kernel PCA. This study proposes a method for choosing hyperparameters in kernel PCA (kernel and the number of components) based on cross-validation for the comparable reconstruction errors of pre-images in t he original space. The experimental results on synthesized and real-world datasets demonstrate tha t the proposed method successfully selects an appropriate kernel and the number of components in kernel PCA in terms of visualization and classification errors on the principal components. The results imply that the proposed method enables automatic design of hyperparameters in kernel PCA.

Anomaly detection for satellite power subsystem with associated rules based on Kernel Principal Component Analysis

Underground metal target detection technology has been widely applied in industrial production, resource exploration, and engineering construction, etc. However, due to the influence of non-negligible noise and high dimensionality in collected data, achieving efficient and accurate underground target classification remains a grand challenge for further applications of underground metal target detection on portable devices with limited computing capability and energy supply. This study aimed to seek out robust and efficient data-based strategies to classify the underground metal targets of different shapes and materials based on electromagnetic induction detection. We investigated thirty-three classification strategies based on eleven dimensionality reduction methods, namely, the least absolute shrinkage and selection operator (LASSO), genetic algorithm-support vector machine (GA-SVM), Pearson correlation coefficient (PCC), mutual information (MI), maximal relevance minimal redundancy Pearson correlation (mRMRP), maximal relevance minimal redundancy mutual information (mRMRMI), statistical features (SF), principal component analysis (PCA), kernel principal component analysis (KPCA), locally linear embedding (LLE), and stacked denoising autoencoder (SDAE), and three machine learning models, namely, artificial neural network (ANN), linear support vector machine (L-SVM), and Gaussian Naïve Bayes (GNB). Several parameters, including classification accuracy, the number of features after dimensionality, the feature type importance, and the time consumption were considered to evaluate the data-based classification strategies. Among the classification strategies investigated and considering the above evaluation parameters, the artificial neural network (ANN) classifier assisted with the kernel principal component analysis (KPCA) feature extraction method yielded the best performance in the material-based classification (accuracy:0.99) and the shape-based classification (accuracy:0.99). The locally linear embedding (LLE) improved the robustness of machine learning classifiers and efficiency of the artificial neural network in the material-based classification (improvement of average accuracy:0.17, reduction of classification time cost:14%) and shape-based classification (improvement of average accuracy:0.16, reduction of classification time cost:22%). Our comparative investigation provides a robust and efficient data-based strategy for underground metal target classification, which is significant for applications of underground metal target detection on portable devices with limited computing capability and energy supply. The cross-combination strategy of dimensionality reduction methods and machine learning models provides a way to find the optimal machine learning model for underground target detection.

Credit card fraud is a major problem in today’s financial world. It induces severe damage to financial institutions and individuals. There has been an exponential increase in the losses due to fraud in recent years. Hence, effectively detecting fraudulent behavior is of vital importance for either financial institutions or individuals. Since credit fraud events account for a small proportion of all transaction events in real life, the datasets about credit fraud are usually imbalanced. Some common classifiers, such as decision tree and naïve Bayes, are unable to detect fraud. Furthermore, in some cases, traditional strategies for dealing with an imbalanced problem, such as the synthetic minority oversampling technique (SMOTE), are not effective for the fraud detection datasets. To accurately detect fraud behavior, this study uses anomaly detection on imbalanced data, as well as Isolation Forest (IForest) with kernel principal component analysis. A one-class support vector machine (OCSVM) with AdaBoost is used as two models to detect outliers which significantly improves detection accuracy and efficiency. The model achieved 96% accuracy, 100% precision, 96% recall, and 98% F 1 score, respectively. The proposed model is expected to become a helpful tool for finding credit card fraud detection, and the analysis presented in this study will provides useful insights into credit card fraud detection mechanisms.

The dimension reduction methods have been proved powerful and practical to extract latent features hidden in the signal for process monitoring. A novel nonlinear dimension reduction method called kernel orthogonal global-local preserving projections (KOGLPP) is proposed and applied for condition monitoring and fault detection. To overcome the shortcomings of kernel locality preserving projections (KLPP) and kernel principal component analysis (KPCA), the KOGLPP model aims at preserving the global and local data structures simultaneously by constructing a dual-objective optimization function, and a tuning parameter is introduced to adjust the trade-off between the global and local data structures. For the purpose of condition monitoring and fault detection, monitoring statistics are constructed in low dimensional feature space. As KOGLPP combines the advantages of both KPCA and KLPP, KOGLPP is also more powerful in extracting potential useful data characteristics. Finally, the effectiveness of the proposed nonlinear dimension reduction method is evaluated experimentally on a numerical example and a bearing test-rig. The results indicate its potential applications as an effective and reliable tool for condition monitoring and fault detection.

On using prototype reduction schemes to optimize kernel-based nonlinear subspace methods

Unsupervised anomaly detection-which aims to identify outliers in data sets without the use of labeled training data-is critically important across a variety of domains including medicine, security, defense, finance, and imaging. In particular, detection of anomalous pixels within hyperspectral images is used for purposes ranging from the detection of military targets to the location of invasive plant species. Kernel methods have frequently been employed for this unsupervised learning task but are limited by their sensitivity to parameter choices and the absence of a validation step. Here, we use reconstruction error in the kernel Principal Component Analysis (kPCA) feature space as a metric for anomaly detection and propose, via batch gradient descent minimization of a novel loss function, to automate the selection of the Gaussian RBF kernel parameter, σ. In addition, we leverage an ensemble of learned models to reduce computational cost and improve detection performance. We describe how to select the model ensemble and show that our method yields better detection accuracy relative to competing algorithms on a pair of data sets.

We consider the detection of submerged targets in a hyperspectral image comprising a difficult maritime scene with littoral, open water, and land regions, characterized by the presence of false alarms arising from highly variable depths and sandbars. We employ a baseline detection scheme that uses kernel principal component analysis (kPCA) to learn a background model from a random small sub-sample of the scene. Detection statistics for test pixels are formed from the reconstruction error between their Nystrom projection into the kPCA feature space and their synthesis in the feature space using the learned background principal components. We show that false alarms associated with mixed littoral pixels are reduced by augmenting the background model with spectral samples drawn from either the linear geodesic constructed between sand and water end-members or an alternative nonlinear geodesic constructed using an optimal transport technique. We show that background augmentation using the the optimal transport geodesic yields significant improvements in detection performance.

In the last decade, many approaches have been developed to solve one-class classification (OCC) problems for anomaly detection. Many of them rely on estimating the statistical distribution of the data, find hidden patterns, or remap the data in advantageous feature spaces. This kind of techniques usually needs some a priori knowledge of the data distribution (i.e., Gaussian) or the setting of some parameters to achieve good classification performance, making their use less effective when the data distribution is unknown. In this paper, we propose a novel blind anomaly detection for low dimensional feature spaces, that exploits the flexibility of the neural network (NN) structure to find the class boundaries without any information about the shape of the data distribution. To prove the generality of the solution, we tested many different class shapes, and we applied it to a structural health monitoring (SHM) case study. Without requiring the tuning of hyperparameters, the performance of the proposed algorithm overcomes that of some known approaches like principal component analysis (PCA), kernel principal component analysis (KPCA), Gaussian mixture model (GMM), and autoassociative neural network (ANN) in many cases, and performs well in the specific SHM setting.

Quantum algorithms for anomaly detection using amplitude estimation

The escalating diversity of Internet of Things (IoT) devices has intensified security challenges, as signature-based and heavyweight machine learning (ML) defenses often fall short in resource-constrained edge environments, unable to meet real-time demands. To overcome these limitations, we introduce Patch Gate Time-series Anomaly Detection (PGTAD), a novel, lightweight, unsupervised multivariate time-series (MVTS) anomaly detection algorithm specifically tailored for IoT network traffic. PGTAD is a Gated recurrent units (GRU) autoencoder-based framework that intelligently integrates features from three perspectives—forward sequences, backward sequences, and local patch segments—using a Gate Mechanism (GM). It employs sinusoidal positional encoding for temporal awareness and a patch operation to capture localized patterns. Anomaly detection is achieved via Mahalanobis distance on reconstruction errors. Extensive experiments on publicly available IoT network traffic datasets: Edge-IIoTset, CICIoT2023, CICIoMT2024, and CICIoV2024, utilizing both packet-based and flow-based feature perspectives, demonstrated PGTAD's superior performance. Evaluated using various F-score metrics (including point-wise, composite, and affiliation) and inference throughput, PGTAD achieved an average F-score of 0.92 and sustained over 480K packets per second on an NVIDIA Jetson Orin Nano 8GB development kit. This confirms its effectiveness and generalizability for real-time detection in IoT edge environments.

摘要: 核主成分分析(Kernel principal component analysis, KPCA)是一种非线性降维工具, 在降低数据流分类处理量方面发挥着积极作用. 然而, 由于复杂性太高, 导致KPCA的降维能力有限. 为此, 本文给出了一种增量核主成分分析算法(Incremental KPCA for dimensionality-reduction, IKDR), 该算法在每步迭代估计中只需线性内存开销, 大大降低了复杂性. 在IKDR的基础上, 结合BP (Back propagation)神经网络提出了数据流在线分类框架: IKOCFrame (Online classification frame based on IKDR). 通过一系列真实和人工数据集上的实验, 检验了IKDR算法的收敛性, 并且验证了IKOCFrame相对于同类基于成分分析的分类算法的优越性. 关键词: 降维技术 / 数据流分类 / 增量核主成分分析 / 独立成分分析

Unsupervised learning-based anomaly detection using autoencoders has gained importance since anomalies behave differently than normal data when reconstructed from a well-regularized latent space. Existing research shows that retaining valuable properties of input data in latent space helps in the better reconstruction of unseen data. Moreover, real-world sensor data is skewed and non-Gaussian in nature rendering mean-based estimators unreliable for such cases. Reconstructionbased anomaly detection methods rely on Euclidean distance as the reconstruction error which does not consider useful correlation information in the latent space. In this work, we address some of the limitations of the Euclidean distance when used as a reconstruction error to detect anomalies (especially near anomalies) that have a similar distribution as the normal data in the feature space. We propose a latent dimension regularized autoencoder that leverages a robust form of the Mahalanobis distance (MD) to measure the latent space correlation to effectively detect near as well as far anomalies. We showcase that incorporating the correlation information in the form of robust MD in the latent space is quite helpful in separating both near and far anomalies in the reconstructed space.