One Class Classifier Neural Network for Anomaly Detection in Low Dimensional Feature Spaces

Outlier detection is the task of classifying test data that differ in some respect from the data that are available during training. This may be seen as one-class classification, in which a model is constructed to describe normal training data. In wireless sensor networks (WSNs), the outlier detection process is a necessary step in building sensor network systems to assure data quality for perfect decision making. For this case, the task amounts to create a useful model based on kernel principal component analysis (KPCA) to recognize data as normal or outliers. Recently, KPCA has used for nonlinear case which can extract higher order statistics. KPCA mapping the data onto another feature space and using nonlinear function. On account of the attractive capability, KPCA-based methods have been extensively investigated, and it have showed excellent performance. Within this setting, we propose KPCA-based Mahalanobis kernel as a new outlier detection method using Mahalanobis distance to implicitly calculate the mapping of the data points in the feature space so that we can separate outlier points from normal pattern of data distribution. The use of KPCA-based Mahalanobis kernel on real-word data obtained from Intel Berkeley are reported showing that the proposed method performs better in finding outliers in WSNs when compared with the original reconstruction error-based variant and the one-class support vector machine detection approach. All computation are done in the original space, thus saving computing time using Mahalanobis kernel.

The dimension reduction methods have been proved powerful and practical to extract latent features hidden in the signal for process monitoring. A novel nonlinear dimension reduction method called kernel orthogonal global-local preserving projections (KOGLPP) is proposed and applied for condition monitoring and fault detection. To overcome the shortcomings of kernel locality preserving projections (KLPP) and kernel principal component analysis (KPCA), the KOGLPP model aims at preserving the global and local data structures simultaneously by constructing a dual-objective optimization function, and a tuning parameter is introduced to adjust the trade-off between the global and local data structures. For the purpose of condition monitoring and fault detection, monitoring statistics are constructed in low dimensional feature space. As KOGLPP combines the advantages of both KPCA and KLPP, KOGLPP is also more powerful in extracting potential useful data characteristics. Finally, the effectiveness of the proposed nonlinear dimension reduction method is evaluated experimentally on a numerical example and a bearing test-rig. The results indicate its potential applications as an effective and reliable tool for condition monitoring and fault detection.

Kernel principal component analysis (KPCA) is a method for performing a nonlinear form of principal component analysis (PCA). It involves nonlinear mapping of the input data onto a high dimensional feature space and performing PCA there. Through KPCA, nonlinear relations in the input space are detected as linear relations in the feature space. Furthermore, explicit computation of the mapping is not necessary in KPCA. Only dot products of the mapped data items in the feature space are required which are obtained directly from the input data using a kernel function. Moreover, control limits of traditional statistical process monitoring indices are usually determined under assumption of normal (Gaussian) distribution. Although, it is recognized that this is inappropriate in nonlinear processes, the impact of using kernel density estimation (KDE), which is a nonparametric method, to determine control limits in KPCA-based process monitoring has not been reported. This paper seeks to bridge this gap. The KPCA with KDE approach is applied to the Tennessee Eastman process for the detection of faults. The results confirm that associating KPCA with kernel density estimated control limits provides better monitoring performance than using control limits based on the normal probability density function.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.

A novel method for machine performance degradation assessment based on fixed cycle features test

In multi-label learning, each object belongs to multiple class labels simultaneously. In the data explosion age, the size of data is often huge, i.e., large number of instances, features and class labels. The high dimension of both the feature and label spaces has posed great challenges to multi-label learning problems, e.g., high time and memory costs. In this paper, we propose a new framework for multi-label learning with a large number of class labels and features, i.e., M ulti- L abel L earning via F eature and L abel S pace D imension R eduction, namely MLL-FLSDR. Specifically, both the feature space and label space are reduced to low dimensional spaces respectively, in which the local structure of data points is utilized to constrain the geometrical structure on both the learned low dimensional spaces and guarantee the qualities of them. Then, an effective multi-label classifier is constructed from the low dimensional feature space to the latent label space. Last, the final prediction for new test data examples can be obtained by recovering from their prediction results in the latent label space with an encoding matrix learned in the previous stage. Extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method MLL-FLSDR.

Autoassociative neural networks (ANNs) have been proposed as a nonlinear extension of principal component analysis (PCA), which is commonly used to identify linear variation patterns in high-dimensional data. While principal component scores represent uncorrelated features, standard backpropagation methods for training ANNs provide no guarantee of producing distinct features, which is important for interpretability and for discovering the nature of the variation patterns in the data. Here, we present an alternating nonlinear PCA method, which encourages learning of distinct features in ANNs. A new measure motivated by the condition of orthogonal loadings in PCA is proposed for measuring the extent to which the nonlinear principal components represent distinct variation patterns. We demonstrate the effectiveness of our method using a simulated point cloud data set as well as a subset of the MNIST handwritten digits data. The results show that standard ANNs consistently mix the true variation sources in the low-dimensional representation learned by the model, whereas our alternating method produces solutions where the patterns are better separated in the low-dimensional space.

An observation-constrained generative approach for probabilistic classification of image regions

Principal Components Analysis (PCA) is used to classify the running condition of a machine by means of projecting the original data to the Principal Components space. However, if the data are concentrated in a nonlinear subspace, PCA will fail to work well. Kernel Principal Components Analysis (KPCA) transforms the input data from the original input space into a higher dimensional feature space with the nonlinear mapping, and then uses the nonlinear principal components to realize the classification. In this paper a case of gear fault diagnosis was studied with KPCA. The characteristic values of frequent domain from vibration signals of the gearbox under the running condition were extracted, and the KPCA method was used to classify gear crack fault. The result shows that KPCA is more effective to distinguish the state of the gear and more suitable to diagnose the gear faults in early stage.

Multivariate outlier detection and remediation in geochemical databases

Kernel principal component analysis (KPCA) has gained much attention for capturing nonlinear image features which is particularly important for clustering high-dimensional multi-class features. We introduce KPCA in this paper for detecting and classifying moving vehicles from its viewpoint images. The KPCA extracts non-linear features of multi-class moving vehicles by mapping input space to a higher dimensional feature space through a nonlinear map. The results provide us a clustered feature space of the car and non-car for classifying them by separating the dimensional space or eigenvector. The experimental results show the robustness of the KPCA's feature separation in our car database that lead to the cars' classification. Extended experiments on various vehicles detection have also shown the remarkable performance of the proposed method

Viewing to monitoring the continuous casting process, a nonlinear fault detection method based on kernel principal component analysis (KPCA) was introduced. KPCA can efficiently compute principal components in high-dimensional feature spaces by means of integral operators and nonlinear kernel functions, which is to first map the input space into a feature space via nonlinear mapping and then to compute the principal components in that feature space. Based on T2 and SPE charts in feature space, principal component analysis(PCA)can be used to detect faults.. The simulation results show that the proposed approach effectively captured the nonlinear relationship in the process variables and showed superior process monitoring performance compared to linear PCA.

Summary The principal component analysis (PCA) is exists the problem of lack of probability model and the absence of higher-order statistics information. Therefore，in order to overcome its shortcomings，this paper studies the method that can overcome two disadvantages of the principal component analysis (PCA) — the probability kernel principal component analysis (PKPCA) which is based on Bayesian theory and kernel principal component analysis (KPCA). First，the sample data are mapped to the high dimensional feature space，then define probability model of the data in high-dimensional space，and finally，expectation maximization (EM) estimated is used to get the best results. This method has both the advantage of probability analysis and kernel principal component analysis (KPCA). It is able to effectively adapt to more complex reservoir conditions and can realize the non-linear probability analysis. The probability of kernel principal component analysis (PKPCA) is applied to reservoir prediction of the oilfield. The predicted results show that the method can improve the precision attribute optimization，while improving the accuracy of the forecasts of reservoir.

The article delves into the development of Statistical Fault Detection and Diagnosis Strategies for an Integrated Steel Plant (ISP) taking into account the nonlinear relationship amongst the monitored Process and Feedstock Characteristics. The strategies being devised is based on Neural Network Fitting model cum Principal Component Analysis based technique (NNF-PCA) and Kernel Principal Component Analysis (KPCA) based technique. For detection of fault(s) Hotelling T2 chart based on (Principal Component Analysis) PCA and KPCA scores were employed and an ensemble strategy amalgamating KPCA and Self Organizing Map neural network has also been proposed for the detection of the out-of-control observations or faults. The article also proposes a 2-phase Fault Diagnosis approach christened as Preliminary Diagnosis phase and Specific Diagnosis phase. The Preliminary Diagnosis phase is based on Pattern Analysis of the control chart monitoring statistic observations and the Specific Diagnosis phase is based on the employment of appropriate Fault Diagnostic Statistic. The Preliminary Diagnosis reveals the broader source of assignable cause for the onset of the fault(s) and the Specific Diagnosis reveals the relative contribution of the individual Process and Feedstock characteristics. An in-depth comparative analysis between the NNF-PCA based strategy and KPCA based strategy w.r.t. three comparative aspects and four comparative parameters were carried out with their findings being duly highlighted which revealed the slight effectiveness of the KPCA based strategy with respect to the NNF-PCA based counterpart.

As a valid statistical tool, principal component analysis (PCA) has been widely used in industrial process monitoring. But due to its intrinsic linear character, it performs badly in nonlinear process monitoring. Kernel PCA (KPCA) can extract useful information in nonlinear data. However KPCA-based monitoring is not suitable for on-line monitoring because of large calculation and much memory occupation. The paper introduces an on-line monitoring method based on KPCA and neural network (NN), where KPCA is used to extract nonlinear principal components (PCs) and then NN approximates the relationship between process data and nonlinear PCs. We can obtain nonlinear PCs by NN to compute the monitoring indices and then achieve the on-line monitoring. The case study shows the validity of the method.KeywordsNeural NetworkKernel Principal Component AnalysisMonitoring IndexMemory OccupationComplex Industrial ProcessThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.