3
In recent years, a variety of nonlinear dimensionality reduction techniques (NLDR) have been proposed in the literature. They aim to address the limitations of traditional techniques such as PCA and classical scaling. Most of these techniques assume that the data of interest lie on an embedded non-linear manifold within the higher-dimensional space. They provide a mapping from the high-dimensional space to the low-dimensional embedding and may be viewed, in the context of machine learning, as a preliminary feature extraction step, after which pattern recognition algorithms are applied. Laplacian Eigenmaps (LE) is a nonlinear graph-based dimensionality reduction method. It has been successfully applied in many practical problems such as face recognition. However the construction of LE graph suffers, similarly to other graph-based DR techniques from the following issues: (1) the neighborhood graph is artificially defined in advance, and thus does not necessary benefit the desired DR task; (2) the graph is built using the nearest neighbor criterion which tends to work poorly due to the high-dimensionality of original space; and (3) its computation depends on two parameters whose values are generally uneasy to assign, the neighborhood size and the heat kernel parameter. To address the above-mentioned problems, for the particular case of the LPP method (a linear version of LE), L. Zhang et al.1 have developed a novel DR algorithm whose idea is to integrate graph construction with specific DR process into a unified framework. This algorithm results in an optimized graph rather than a predefined one.
8Visible-NIR spectral characterization and grade inversion modelling study of the Derni copper deposit
Visible-NIR spectral characterization and grade inversion modelling study of the Derni copper deposit
1Locally Linear Embedding based on Rank-order Distance
Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighborsâ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.
Read more47Manifold Learning in MR spectroscopy using nonlinear dimensionality reduction and unsupervised clustering.
To investigate whether nonlinear dimensionality reduction improves unsupervised classification of (1) H MRS brain tumor data compared with a linear method. In vivo single-voxel (1) H magnetic resonance spectroscopy (55 patients) and (1) H magnetic resonance spectroscopy imaging (MRSI) (29 patients) data were acquired from histopathologically diagnosed gliomas. Data reduction using Laplacian eigenmaps (LE) or independent component analysis (ICA) was followed by k-means clustering or agglomerative hierarchical clustering (AHC) for unsupervised learning to assess tumor grade and for tissue type segmentation of MRSI data. An accuracy of 93% in classification of glioma grade II and grade IV, with 100% accuracy in distinguishing tumor and normal spectra, was obtained by LE with unsupervised clustering, but not with the combination of k-means and ICA. With (1) H MRSI data, LE provided a more linear distribution of data for cluster analysis and better cluster stability than ICA. LE combined with k-means or AHC provided 91% accuracy for classifying tumor grade and 100% accuracy for identifying normal tissue voxels. Color-coded visualization of normal brain, tumor core, and infiltration regions was achieved with LE combined with AHC. The LE method is promising for unsupervised clustering to separate brain and tumor tissue with automated color-coding for visualization of (1) H MRSI data after cluster analysis.
Read more5HSIC regularized manifold learning
At present, manifold learning is mainly applied to dimensionality reduction. However, from viewpoint of dimensionality reduction, manifold learning algorithms are only local feature preserving algorithms. For example, Local Linear Embedding is local linear preserving, Local Tangent Space Alignment is local homeomorphic preserving and Laplacian Eigenmap is local similarity preserving. The community of dimensionality reduction is now pursuing the algorithms which can preserve both local and global features of data during dimensionality reduction. In this paper, a new algorithm of dimensionality reduction, called Hilbert-Schmidt Independence Criterion Regularized Manifold Learning (HSIC-ML for short), is proposed, in which HSIC between the high dimensional data and the dimension-reduced data is added as a regularization term to the objective functions of manifold learning. The addition of HSIC regularization term makes HSIC-ML capable of preserving both local and global features during dimensionality reduction. HSIC is a criterion measuring the statistical dependence between two data sets and has been widely applied to machine learning in recent years. However, since HSIC was first proposed around 2005, there seems to have not been applied directly to dimensionality reduction, not applied as a regularization term either. The proposed HSIC-ML may be the first try in this respect. The experimental results presented in this paper show that the manifold learning with HSIC regularization performs better than that without HSIC regularization.
Read moreEnhancing Generalized Autoencoder for Improved Supervised Dimensionality Reduction in Image Recognition
Objectives: The article proposes enhancements to the Generalized Autoencoder (GAE) framework for Dimensionality Reduction (DR).The first objective is to improve the reconstruction process so that recognition accuracy increases. The second objective is to embed class label information into the framework. This turns GAE into a supervised method with better interclass discrimination. Method: Two improvements to the GAE are proposed: (i) an adjustment to reconstruction weights to capture neighbourhood relations in high-dimensional spaces effectively, and (ii) a strategy that incorporates class label information directly into the reconstruction set. The approach is evaluated using standard benchmark datasets for face recognition and handwritten digit classification. The approach is evaluated on eight benchmark datasets: four standard face databases (ORL, AR, UMIST, YaleB) and four handwritten digit databases (MNIST, Gujarati, Devnagari, Bangla). For a fair comparison, recognition performance is studied under varying dimensionality of the reduced feature space, different neighborhood sizes. Findings: Experimental results show that the modified GAE significantly improves recognition performance compared to the original GAE by 1-5%. The supervised version demonstrates stronger discriminative power across all datasets with better recognition rates by 3-7%. Novelty: The proposed approach adds class label information into the reconstruction set, turning GAE into a supervised framework. At the same time, it refines the weight adjustment process. Together, these two changes improve both accuracy and discrimination in dimensionality reduction tasks. Keywords: Dimensionality Reduction, Auto Encoder, Laplacian EigenMaps, Locally Linear Embeddings, Locality Preserving Projection, Neighbourhood Preserving Projection
Read more11D representation of Laplacian eigenmaps and dual k ‐nearest neighbours for unified video coding
This study proposes a framework of video coding based on Laplacian eigenmaps (LEM) and its related embedding and reconstruction algorithm (ERA). Firstly, a one-dimensional (1D) representation of LEM is adopted to achieve an extremely low bit per pixel (BPP). Secondly, dual k-nearest neighbours, which keeps neighbour relationships both in high-dimensional data space and low-dimensional representation space and overcomes the disadvantage of classical non-linear dimensionality reduction methods which cannot preserve the neighbour properties in both of the spaces, based ERA of LEM is employed to gain extraordinarily high peak-signal-to-noise ratio (PSNR). Thirdly, a unified framework of video coding is fit for intra-frame, inter-frame and multi-view video coding. Finally, it is evaluated by simulation experiments that, in the situation of low bitrate transmission, the proposed method can attain better performance of BPP and PSNR than that of the state-of-the-art methods, such as highly efficient video coding.
Read more32Detection of maize seed germination rate based on improved locally linear embedding
Detection of maize seed germination rate based on improved locally linear embedding
10Systematic comparison of unsupervised feature extraction methods for enhancing discrimination accuracy of brown rice flour adulteration in LIBS
Systematic comparison of unsupervised feature extraction methods for enhancing discrimination accuracy of brown rice flour adulteration in LIBS
Read more1Chapter 19 - Dimensionality Reduction and Latent Variable Modeling
Chapter 19 - Dimensionality Reduction and Latent Variable Modeling
36Variable selection based on locally linear embedding mapping for near-infrared spectral analysis
Variable selection based on locally linear embedding mapping for near-infrared spectral analysis
96Trace Ratio Optimization-Based Semi-Supervised Nonlinear Dimensionality Reduction for Marginal Manifold Visualization
Visualizing similarity data of different objects by exhibiting more separate organizations with local and multimodal characteristics preserved is important in multivariate data analysis. Laplacian Eigenmaps (LAE) and Locally Linear Embedding (LLE) aim at preserving the embeddings of all similarity pairs in the close vicinity of the reduced output space, but they are unable to identify and separate interclass neighbors. This paper considers the semi-supervised manifold learning problems. We apply the pairwise Cannot-Link and Must-Link constraints induced by the neighborhood graph to specify the types of neighboring pairs. More flexible regulation on supervised information is provided. Two novel multimodal nonlinear techniques, which we call trace ratio (TR) criterion-based semi-supervised LAE (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LAE) and LLE (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LLE), are then proposed for marginal manifold visualization. We also present the kernelized S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LAE and S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LLE. We verify the feasibility of S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LAE and S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LLE through extensive simulations over benchmark real-world MIT CBCL, CMU PIE, MNIST, and USPS data sets. Manifold visualizations show that S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LAE and S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> LLE are able to deliver large margins between different clusters or classes with multimodal distributions preserved. Clustering evaluations show they can achieve comparable to or even better results than some widely used methods.
Read more128A multitask multiview clustering algorithm in heterogeneous situations based on LLE and LE
A multitask multiview clustering algorithm in heterogeneous situations based on LLE and LE
2Measuring and optimising performance of an offline text writer identification system in terms of dimensionality reduction techniques
Usually, most of the data generated in real-world such as images, speech signals, or fMRI scans has a high dimensionality. Therefore, dimensionality reduction techniques can be used to reduce the number of variables in that data and then the system performance can be improved. Because the processing of the high dimensional data leads the increase of complexity both in execution time and memory usage. In the previous work, we developed an offline writer identification system using a combination of Oriented Basic Image features (OBI) and the concept of graphemes codebook. In order to measure and optimise the system performance, a variety of nonlinear dimensionality reduction algorithms such as Kernel Principal Component Analysis (KPCA), Isomap, Locally linear embedding (LLE), Hessian LLE and Laplacian Eigenmaps have been used. The performance has been evaluated based on IAM dataset for English handwriting and ICFHR 2012 dataset for Arabic handwriting. The results obtained indicated the system performance based KPCA was better than the other reduction techniques that have been used and investigated in this work.
Read more23Curvature analysis of frequency modulated manifolds in dimensionality reduction
Recent advances in the analysis of high-dimensional signal data have triggered an increasing interest in geometry-based methods for nonlinear dimensionality reduction (NDR). In many applications, high-dimensional datasets typically contain redundant information, and NDR methods are important for an efficient analysis of their properties. During the last few years, concepts from differential geometry were used to create a new range of NDR methods. In the construction of such geometry-based strategies, a natural question is to understand their interaction with classical and modern signal processing tools (convolution transforms, Fourier analysis, wavelet functions). In particular, an important task is the analysis of the incurred geometrical deformation when applying signal transforms to the elements of a dataset. In this paper, we propose the concepts of modulation maps and modulation manifolds for the construction of particular datasets relevant in signal processing and NDR. We consider numerical methods for analyzing geometrical properties of the modulation manifolds, with a particular focus on their scalar and mean curvature. In our numerical examples, we apply the resulting geometry-based analysis to simple test cases, where we present geometrical and topological effects of relevance in manifold learning.
Read more6Locally Minimizing Embedding and Globally Maximizing Variance: Unsupervised Linear Difference Projection for Dimensionality Reduction
Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.
Read more
Copyright 2026 Cactus Communications. All rights reserved.
