Locally linear embedding with additive noise

The locally linear embedding (LLE) is one of the most promising algorithm among nonlinear dimensionality reduction technique. When the number of the nearest neighbor k is larger than the input dimension D, LLE will make the local covariance matrix Ci singular in the process of calculating local reconstruction weights vectors Wi. In this case, regularization is needed, but at the same time, which leads Wi can not reflect optimally intrinsic structure of neighbours. To solve the problem, we propose NWLLE (New Weighted LLE) to improve LLE algorithm and test the new algorithm from embedded performance and quantitative error. The numerical experiment results show that the NWLLE yields less error than LLE and can produce a good embedding effect.

Chapter 19 - Dimensionality Reduction and Latent Variable Modeling

Classification of human hand movements based on EMG signals using nonlinear dimensionality reduction and data fusion techniques

Unlike classical linear dimensionality reduction techniques, nonlinear ones are capable of discovering the nonlinear degrees of freedom that are present in natural observations, by assuming that the data lie on an embedded nonlinear manifold within an observed high dimensional feature space. Nevertheless, when measured objects are actually functional data, nonlinear dimensionality reduction techniques do not produce suitable unfolded results, particularly in the locally linear embedding (LLE) algorithm. In this case, the Euclidean distance employed in cost function does not correctly represent the similarity between objects, because this distance does not take into account the intrinsical relations presented in functional data, besides, it is easily distorted by non-gaussian noise, such as an artifacts, impulsive noise, etc. The main contribution in this paper is to use (inside the LLE algorithm) a localized similarity measure called correntropy, which has a particular metric that allows it to conform to suitable neighborhoods, and indeed convenient representations of the objects, avoiding distortions.

Perception by autonomous systems, in unstructured dynamic worlds, is one of the significant research challenges in the development of effective intelligent systems. Nonlinear dimensionality reduction techniques have been extensively utilized within the artificial intelligence community to devise compact representations of high dimensional data. These techniques display great promise in yielding low dimensional, meaningful representations of an unstructured environment in real time from raw sensory information. Two such techniques, the kernel principal component analysis method and locally linear embedding (LLE) are evaluated herein, with respect to their ability to generate compact and physically reasonable embeddings of an unstructured environment. The LLE technique shows great potential in the computation of low dimensional and perceptually meaningful embeddings of natural environments.

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

In principle, the differential dynamics of a protein perturbed by various ligands should be able to reflect ligands’ different functions. However, in the field of G protein-coupled receptor (GPCR), the phenomenon of conformational heterogeneity, i.e., the sharing of conformations traversed by differently liganded receptors, poses a challenge for delineating ligand’s action on perturbing protein dynamics. In a previous work, we have conduct multiple molecular dynamics (MD) simulations of the agonists- and antagonists-bound human A2A adenosine receptor (A2AAR) starting from an intermediate state conformation to maximize the sensitivity of ligand-perturbed dynamics. Conformational heterogeneity can be visualized directly by the Markov state model (MSM) analysis, which is a two-stage procedure first by performing clustering based on conformational similarity to form microstates and then kinetic lumping based on state inter-convertibility to aggregate microstates into macrostates. To delineate the geometric properties of these macrostates, we embedded them onto the low dimensional space constructed with a non-linear dimensionality reduction scheme. While the crystal structures of the G-protein coupled receptor in different states (fully active, intermediate, inactive) can be projected onto divisible regions in the first two dimensions of the isomap embedding, conformations from three “purer” states (agonist-enriched, apo-enriched, antagonist-enriched) cannot be very clearly separated with this two-dimensional embedding. Dimensionality higher than two may still be needed to specify dynamically discriminable states even with nonlinear dimensionality reduction techniques.

Modern manufacturing processes are often characterized by high dimensionality and nonlinearity; dimension reduction is usually required to identify the critical features and help improve the process yield. Dimension reduction and analysis of data with high nonlinearity has been a challenging task. Many methods have been proposed based on manifold learning, which aims to learn a lower-dimensional manifold from the original high-dimensional space. However, since the mappings are usually implicit, it is difficult to build a connection between the classification results and the original features. Apart from the prediction results, industry data analysis also focus on the features themselves. Therefore, methods that are able to combine feature selection with dimension reduction is often needed. This paper proposed a hybrid method for nonlinear dimension reduction and feature selection based on Locally Linear Embedding (LLE). LLE and many filter feature selection shares a common procedure of neighbor search. By integrating nearest neighbor search for both LLE and ReliefF, and adjusting the distance measure in supervised LLE with results from feature selection, the proposed method could connect the feature selection process with the nonlinear model, extract the critical features for further analysis, and improve the performance of process modelling. Two dataset from real industry processes are also illustrated as case studies.

Current practice in Quantitative Structure Activity Relationship (QSAR) methods usually involves generating a great number of chemical descriptors and then cutting them back with variable selection techniques. Variable selection is an effective method to reduce the dimensionality but may discard some valuable information. This paper introduces Locally Linear Embedding (LLE), a local non-linear dimensionality reduction technique, that can statistically discover a low-dimensional representation of the chemical data. LLE is shown to create more stable representations than other non-linear dimensionality reduction algorithms, and to be capable of capturing non-linearity in chemical data.

Hyperspectral imagery (HSI) has high spectral and spatial resolutions, which are essential for anomaly detection (AD). Many anomaly detectors assume that the spectrum signature of HSI pixels can be modeled with a Gaussian distribution, which is actually not accurate and often leads to many false alarms. Therefore, a sparsity model without any distribution hypothesis is usually employed. Dimensionality reduction (DR) as a preprocessing step for HSI is important. Principal component analysis as a conventional DR method is a linear projection and cannot exploit the nonlinear properties in hyperspectral data, whereas locally linear embedding (LLE) as a local, nonlinear manifold learning algorithm works well for DR of HSI. A modified algorithm of sparsity divergence index based on locally linear embedding (SDI-LLE) is thus proposed. First, kernel collaborative representation detection is adopted to calculate the sparse dictionary matrix of local reconstruction weights in LLE. Then, SDI is obtained both in the spectral and spatial domains, where spatial SDI is computed after DR by LLE. Finally, joint SDI, combining spectral SDI and spatial SDI, is computed, and the optimal SDI is performed for AD. Experimental results demonstrate that the proposed algorithm significantly improves the performance, when compared with its counterparts.

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.

Detection of maize seed germination rate based on improved locally linear embedding

In this paper, a novel technique for the recognition of weed seeds, known as Locally Linear Embedding (LLE), is presented. This experiment contains two steps: dimensionality reduction with LLE and classification. In dimensionality reduction part, LLE, which is an unsupervised non-linear technique, is used to map high dimensional data into low dimensional, neighborhood-preserving embeddings and extract features. In classification part, minimum Euclidean distance classifier is used to recognize different weed seeds. This paper presents a comparison between LLE and PCA. Experimental results demonstrate that the representation of LLE is superior significantly as expected and gains much higher recognition rates, amounting to 96%, which is 8% higher than those of PCA.

Analysis of population genetic variation and structure is a common practice for genome-wide studies, including association mapping, ecology, and evolution studies in several crop species. In this study, machine learning (ML) clustering methods, K-means (KM), and hierarchical clustering (HC), in combination with non-linear and linear dimensionality reduction techniques, deep autoencoder (DeepAE) and principal component analysis (PCA), were used to infer population structure and individual assignment of maize inbred lines, i.e., dent field corn (n = 97) and popcorn (n = 86). The results revealed that the HC method in combination with DeepAE-based data preprocessing (DeepAE-HC) was the most effective method to assign individuals to clusters (with 96% of correct individual assignments), whereas DeepAE-KM, PCA-HC, and PCA-KM were assigned correctly 92, 89, and 81% of the lines, respectively. These findings were consistent with both Silhouette Coefficient (SC) and Davies–Bouldin validation indexes. Notably, DeepAE-HC also had better accuracy than the Bayesian clustering method implemented in InStruct. The results of this study showed that deep learning (DL)-based dimensional reduction combined with ML clustering methods is a useful tool to determine genetically differentiated groups and to assign individuals into subpopulations in genome-wide studies without having to consider previous genetic assumptions.

Manifold learning and its various algorithms have attracted more and more attention in recent years. Locally linear embedding (LLE) algorithm, as a classical nonlinear dimensionality reduction algorithm, has been widely used in many technical fields, such as image classification and object recognition. This paper proposes an improved algorithm based on LLE algorithm–Sparse LLE, which applies a new method of classification in the process of finding K-nearest neighbours with original LLE algorithm. According to the experiment results, the new algorithm can not only guarantee the effect of dimensionality reduction, but also greatly reduce the processing time of the dimensionality reduction of datasets.