Weighted Locally Linear Embedding for Plant Leaf Visualization

Locally Linear Embedding (LLE) is a recently proposed algorithm for non-linear dimensionality reduction and manifold learning. However, it may not be optimal for classification problem. In this paper, an improved version of LLE, namely KFDA-LLE, is proposed using kernel Fisher discriminant analysis (KFDA) method, combined with SVM classifier for face recognition task. Firstly, the input training samples are projected into the low-dimensional space by LLE. Then KFDA is introduced for finding the optimal projection direction. Finally, SVM classifier is used for face recognition. Experimental results on face database demonstrate that the extended LLE method is more efficient and robust.

We propose an unsupervised approach to select representative face samples (models) from raw videos and build an appearance-based face recognition system. The approach is based on representing the face manifold in a low-dimensional space using the locally linear embedding (LLE) algorithm and then performing K-means clustering. We define the face models as the cluster centers. Our strategy is motivated by the efficiency of LLE to recover meaningful low-dimensional structures hidden in complex and high dimensional data such as face images. Two other well-known unsupervised learning algorithms (Isomap and SOM) are also considered. We compare and assess the efficiency of these different schemes on the CMU MoBo database which contains 96 face sequences of 24 subjects. The results clearly show significant performance enhancements over traditional methods such as the PCA-based one.

Visualizing Alzheimer's disease progression in low dimensional manifolds

Multi-level adaptive neuro-fuzzy inference system-based reconstruction of 1D ISOMAP representations

Modern manufacturing processes are often characterized by high dimensionality and nonlinearity; dimension reduction is usually required to identify the critical features and help improve the process yield. Dimension reduction and analysis of data with high nonlinearity has been a challenging task. Many methods have been proposed based on manifold learning, which aims to learn a lower-dimensional manifold from the original high-dimensional space. However, since the mappings are usually implicit, it is difficult to build a connection between the classification results and the original features. Apart from the prediction results, industry data analysis also focus on the features themselves. Therefore, methods that are able to combine feature selection with dimension reduction is often needed. This paper proposed a hybrid method for nonlinear dimension reduction and feature selection based on Locally Linear Embedding (LLE). LLE and many filter feature selection shares a common procedure of neighbor search. By integrating nearest neighbor search for both LLE and ReliefF, and adjusting the distance measure in supervised LLE with results from feature selection, the proposed method could connect the feature selection process with the nonlinear model, extract the critical features for further analysis, and improve the performance of process modelling. Two dataset from real industry processes are also illustrated as case studies.

The present paper put forward a non-destructive detection method which combines semi-transmission hyperspectral imaging technology with manifold learning dimension reduction algorithm and least squares support vector machine (LSSVM) to recognize internal and external defects in potatoes simultaneously. Three hundred fifteen potatoes were bought in farmers market as research object, and semi-transmission hyperspectral image acquisition system was constructed to acquire the hyperspectral images of normal external defects (bud and green rind) and internal defect (hollow heart) potatoes. In order to conform to the actual production, defect part is randomly put right, side and back to the acquisition probe when the hyperspectral images of external defects potatoes are acquired. The average spectrums (390-1,040 nm) were extracted from the region of interests for spectral preprocessing. Then three kinds of manifold learning algorithm were respectively utilized to reduce the dimension of spectrum data, including supervised locally linear embedding (SLLE), locally linear embedding (LLE) and isometric mapping (ISOMAP), the low-dimensional data gotten by manifold learning algorithms is used as model input, Error Correcting Output Code (ECOC) and LSSVM were combined to develop the multi-target classification model. By comparing and analyzing results of the three models, we concluded that SLLE is the optimal manifold learning dimension reduction algorithm, and the SLLE-LSSVM model is determined to get the best recognition rate for recognizing internal and external defects potatoes. For test set data, the single recognition rate of normal, bud, green rind and hollow heart potato reached 96.83%, 86.96%, 86.96% and 95% respectively, and he hybrid recognition rate was 93.02%. The results indicate that combining the semi-transmission hyperspectral imaging technology with SLLE-LSSVM is a feasible qualitative analytical method which can simultaneously recognize the internal and external defects potatoes and also provide technical reference for rapid on-line non-destructive detecting of the internal and external defects potatoes.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Locally linear embedding (LLE) is a method for nonlinear dimensionality reduction, which calculates a low dimensional embedding with the property that nearby points in the high dimensional space remain nearby and similarly co-located with respect to one another in the low dimensional space [1]. LLE algorithm needs to set up a free parameter, the number of nearest neighbors k . This parameter has a strong influence in the transformation. In this paper is proposed a cost function that quantifies the quality of the embedding results and computes an appropriate k . Quality measure is tested on artificial and real-world data sets, which allow us to visually confirm whether the embedding was correctly calculated.

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

Detection of maize seed germination rate based on improved locally linear embedding

Locally linear embedding (LLE) is an unsupervised nonlinear manifold learning algorithm. It performs well in visualizing data yet has a very poor recognition rate in facial expression recognition. In this paper, to improve the performance of LLE in facial expression recognition, we first employ the independent component analysis (ICA) technique to preprocess the face images such that they are represented by some independent components and some noise is filtered from them. We then propose a supervised LLE (SLLE) algorithm to learn the hidden manifold. SLLE constructs the neighborhood graphs for the data according to the Euclidean distances between them and the cluster information of them. Its embedding step is the same as that of LLE. Finally, we use a generalized regression neural network (GRNN) to learn the implicit nonlinear mapping from the ICA space to the embedded manifold. Experiments on the JAFFE database show promising results

Locally linear embedding (LLE) is an elegant nonlinear method for feature extraction and manifold learning, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. However, LLE algorithm fails when it is directly applied to video with multi-shot. In this paper, video manifold feature is defined firstly, and then using LLE we extract video manifold feature through adding virtual frames on an enriched set. The proposed manifold feature extraction method is applied to shot transition and video trajectories. The video manifold feature description gives a new tool for video analysis.

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.

Manifold learning and its various algorithms have attracted more and more attention in recent years. Locally linear embedding (LLE) algorithm, as a classical nonlinear dimensionality reduction algorithm, has been widely used in many technical fields, such as image classification and object recognition. This paper proposes an improved algorithm based on LLE algorithm–Sparse LLE, which applies a new method of classification in the process of finding K-nearest neighbours with original LLE algorithm. According to the experiment results, the new algorithm can not only guarantee the effect of dimensionality reduction, but also greatly reduce the processing time of the dimensionality reduction of datasets.

Hyperspectral images comprise, by design, high dimensional image data. However, research has shown that for a d-dimensional hyperspectral image, it is typical for the data to inherently occupy an m-dimensional space, with m << d. In the remote sensing community, this has led to a recent increase in the use of non-linear manifold learning, which aims to characterize the embedded lower-dimensional, non-linear manifold upon which the hyperspectral data inherently lie. Classic hyperspectral data models include statistical, linear subspace, and linear mixture models, but these can place restrictive assumptions on the distribution of the data. With graph theory and manifold learning based models, the only assumption is that the data reside on an underlying manifold. In previous publications, we have shown that manifold coordinate approximation using locally linear embedding (LLE) is a viable pre-processing step for target detection with the Adaptive Cosine/Coherence Estimator (ACE) algorithm. Here, we improve upon that methodology using a more rigorous, data-driven implementation of LLE that incorporates the injection of a cloud" of target pixels and the Spectral Angle Mapper (SAM) detector. The LLE algorithm, which holds that the data is locally linear, is typically governed by a user defined parameter k, indicating the number of nearest neighbors to use in the initial graph model. We use an adaptive approach to building the graph that is governed by the data itself and does not rely upon user input. This implementation of LLE can yield greater separation between the target pixels and the background pixels in the manifold space. We present an analysis of target detection performance in the manifold coordinates using scene-derived target spectra and laboratory-measured target spectra across two different data sets.