Sparsity divergence index based on locally linear embedding for hyperspectral anomaly detection

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

Modern manufacturing processes are often characterized by high dimensionality and nonlinearity; dimension reduction is usually required to identify the critical features and help improve the process yield. Dimension reduction and analysis of data with high nonlinearity has been a challenging task. Many methods have been proposed based on manifold learning, which aims to learn a lower-dimensional manifold from the original high-dimensional space. However, since the mappings are usually implicit, it is difficult to build a connection between the classification results and the original features. Apart from the prediction results, industry data analysis also focus on the features themselves. Therefore, methods that are able to combine feature selection with dimension reduction is often needed. This paper proposed a hybrid method for nonlinear dimension reduction and feature selection based on Locally Linear Embedding (LLE). LLE and many filter feature selection shares a common procedure of neighbor search. By integrating nearest neighbor search for both LLE and ReliefF, and adjusting the distance measure in supervised LLE with results from feature selection, the proposed method could connect the feature selection process with the nonlinear model, extract the critical features for further analysis, and improve the performance of process modelling. Two dataset from real industry processes are also illustrated as case studies.

Dimensionality reduction plays a significant role for the performance of large-scale image retrieval. In this paper, various dimensionality reduction methods are compared to validate their own performance in image retrieval. For this purpose, first, the Scale Invariant Feature Transform (SIFT) features and HSV (Hue, Saturation, Value) histogram are extracted as image features. Second, the Principal Component Analysis (PCA), Fisher Linear Discriminant Analysis (FLDA), Local Fisher Discriminant Analysis (LFDA), Isometric Mapping (ISOMAP), Locally Linear Embedding (LLE), and Locality Preserving Projections (LPP) are respectively applied to reduce the dimensions of SIFT feature descriptors and color information, which can be used to generate vocabulary trees. Finally, through setting the match weights of vocabulary trees, large-scale image retrieval scheme is implemented. By comparing multiple sets of experimental data from several platforms, it can be concluded that dimensionality reduction method of LLE and LPP can effectively reduce the computational cost of image features, and maintain the high retrieval performance as well.

Locally linear embedding (LLE) depends on the Euclidean distance (ED) to select the k-nearest neighbors. However, the ED may not reflect the actual geometry structure of data, which may lead to the selection of ineffective neighbors. The aim of our work is to make full use of the local spectral angle (LSA) to find proper neighbors for dimensionality reduction (DR) and classification of hyperspectral remote sensing data. At first, we propose an improved LLE method, called local spectral angle LLE (LSA-LLE), for DR. It uses the ED of data to obtain large-scale neighbors, then utilizes the spectral angle to get the exact neighbors in the large-scale neighbors. Furthermore, a local spectral angle-based nearest neighbor classifier (LSANN) has been proposed for classification. Experiments on two hyperspectral image data sets demonstrate the effectiveness of the presented methods.

Chapter 19 - Dimensionality Reduction and Latent Variable Modeling

In this paper, we study the Locally Linear Embedding (LLE) for nonlinear dimen- sionality reduction of hyperspectral data. We improve the existing LLE in terms of both compu- tational complexity and memory consumption by introducing a spatial neighbourhood window for calculating the k nearest neighbours. The improved LLE can process larger hyperspectral images than the existing LLE and it is also faster. We conducted experiments of endmember extraction to assess the effectiveness of the dimensionality reduction methods. Experimental results show that the improved LLE is better than PCA and the existing LLE in identifying endmembers. It finds more endmembers than PCA and the existing LLE when the Pixel Purity Index (PPI) based endmember extraction method is used. Also, better results are obtained for detection.

Underground metal target detection technology has been widely applied in industrial production, resource exploration, and engineering construction, etc. However, due to the influence of non-negligible noise and high dimensionality in collected data, achieving efficient and accurate underground target classification remains a grand challenge for further applications of underground metal target detection on portable devices with limited computing capability and energy supply. This study aimed to seek out robust and efficient data-based strategies to classify the underground metal targets of different shapes and materials based on electromagnetic induction detection. We investigated thirty-three classification strategies based on eleven dimensionality reduction methods, namely, the least absolute shrinkage and selection operator (LASSO), genetic algorithm-support vector machine (GA-SVM), Pearson correlation coefficient (PCC), mutual information (MI), maximal relevance minimal redundancy Pearson correlation (mRMRP), maximal relevance minimal redundancy mutual information (mRMRMI), statistical features (SF), principal component analysis (PCA), kernel principal component analysis (KPCA), locally linear embedding (LLE), and stacked denoising autoencoder (SDAE), and three machine learning models, namely, artificial neural network (ANN), linear support vector machine (L-SVM), and Gaussian Naïve Bayes (GNB). Several parameters, including classification accuracy, the number of features after dimensionality, the feature type importance, and the time consumption were considered to evaluate the data-based classification strategies. Among the classification strategies investigated and considering the above evaluation parameters, the artificial neural network (ANN) classifier assisted with the kernel principal component analysis (KPCA) feature extraction method yielded the best performance in the material-based classification (accuracy:0.99) and the shape-based classification (accuracy:0.99). The locally linear embedding (LLE) improved the robustness of machine learning classifiers and efficiency of the artificial neural network in the material-based classification (improvement of average accuracy:0.17, reduction of classification time cost:14%) and shape-based classification (improvement of average accuracy:0.16, reduction of classification time cost:22%). Our comparative investigation provides a robust and efficient data-based strategy for underground metal target classification, which is significant for applications of underground metal target detection on portable devices with limited computing capability and energy supply. The cross-combination strategy of dimensionality reduction methods and machine learning models provides a way to find the optimal machine learning model for underground target detection.

Detection of maize seed germination rate based on improved locally linear embedding

Dimensionality reduction plays a fundamental role in data processing, for which principal component analysis (PCA) is widely used. In this paper, we develop the Laplacian PCA (LPCA) algorithm which is the extension of PCA to a more general form by locally optimizing the weighted scatter. In addition to the simplicity of PCA, the benefits brought by LPCA are twofold: the strong robustness against noise and the weak metric-dependence on sample spaces. The LPCA algorithm is based on the global alignment of locally Gaussian or linear subspaces via an alignment technique borrowed from manifold learning. Based on the coding length of local samples, the weights can be determined to capture the local principal structure of data. We also give the exemplary application of LPCA to manifold learning. Manifold unfolding (non-linear dimensionality reduction) can be performed by the alignment of tangential maps which are linear transformations of tangent coordinates approximated by LPCA. The superiority of LPCA to PCA and kernel PCA is verified by the experiments on face recognition (FRGC version 2 face database) and manifold (Scherk surface) unfolding.

Divergent Projection Analysis for Unsupervised Dimensionality Reduction

Broad Absorption Line Quasars (BALQSOs) represent a significant phenomenon in the realm of quasar astronomy, displaying distinct blueshifted broad absorption lines. These enigmatic objects serve as invaluable probes for unraveling the intricate structure and evolution of quasars, shedding light on the profound influence exerted by supermassive black holes on galaxy formation. The proliferation of large-scale spectroscopic surveys such as LAMOST (the Large Sky Area Multi-Object Fiber Spectroscopic Telescope), SDSS (the Sloan Digital Sky Survey), and DESI (the Dark Energy Spectroscopic Instrument) has exponentially expanded the repository of quasar spectra at our disposal. In this study, we present an innovative approach to streamline the identification of BALQSOs, leveraging the power of dimensionality reduction and machine-learning algorithms. Our dataset is meticulously curated from the SDSS Data Release 16 (DR16), amalgamating quasar spectra with classification labels sourced from the DR16Q quasar catalog. We employ a diverse array of dimensionality-reduction techniques, including principal component analysis (PCA), t-Distributed stochastic neighbor embedding (t-SNE), locally linear embedding (LLE), and isometric mapping (ISOMAP), to distill the essence of the original spectral data. The resultant low-dimensional representations serve as inputs for a suite of machine-learning classifiers, including the robust XGBoost and Random Forest models. Through rigorous experimentation, we unveil PCA as the most effective dimensionality-reduction methodology, adeptly navigating the intricate balance between dimensionality reduction and preservation of vital spectral information. Notably, the synergistic fusion of PCA with the XGBoost classifier emerges as the pinnacle of efficacy in the BALQSO classification endeavor, boasting impressive accuracy rates of $97.60\%$ by 10-cross validation and $96.92\%$ on the outer test sample. This study not only introduces a novel machine-learning-based paradigm for quasar classification but also offers invaluable insights transferrable to a myriad of spectral classification challenges pervasive in the realm of astronomy.

The nonlinear characteristics in hyperspectral data is considered as an influential factor curtailing the classification accuracy. To deal with the problem, a new method for classification is developed, especially for hyperspectral imagery (HSI). It is a supervised method based on Locally Linear Embedding (LLE) and k-Nearest Neighbor (KNN), named with KNN based supervised LLE (S-LLE KNN). We use two real HIS dataset of AVIRIS in experiment section and compare overall classification accuracy and accuracy of each class in different methods, the results shows that the supervised nonlinear feature extraction method contributes more to classification accuracies methods.

Dimensionality reduction is a key step in hyperspectral image processing. Recent investigations are looking into nonlinear manifold learning for dimensionality reduction in hyperspectral imagery. Nonlinear manifold learning methods such as Isomap, and Local Linear Embedding are use to recover the low dimensional representation of an unknown nonlinear manifold for high dimensional data where it is important to retain the neighborhood structure of the manifold. Although these algorithms use different philosophies for recovering the nonlinear manifold, they all incorporate neighborhood information from each data point to construct a weighted graph having the data points as vertices. Thus the performance of these methods highly depends on how well these neighborhoods are selected, since all subsequent steps rely on it. The k -NN algorithm is the most widely used technique for neighbor selection in manifold learning. However, it can result in a disconnected graph and it does not fully exploit spatial neighborhood information from the image in selecting points to form the neighborhoods. In this paper, recently proposed methods for constructing the weighted graph in manifold learning are studied: k -VC, k -EC and k -MST, which have the advantage of creating connected graphs and have performed well in artificial data sets. Spatial information of the hyperspectral images is included in the manifold learning process by using spatial coherence. Experiments are conducted with artificial data and hyperspectral images. For hyperspectral images, classification accuracy was used as an indirect measure to understand how the low dimensional embedding of the data reduces dimensionality while still maintaining good discrimination performance.