Dimensionality reduction of hyperspectral imagery using improved locally linear embedding

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

Modern manufacturing processes are often characterized by high dimensionality and nonlinearity; dimension reduction is usually required to identify the critical features and help improve the process yield. Dimension reduction and analysis of data with high nonlinearity has been a challenging task. Many methods have been proposed based on manifold learning, which aims to learn a lower-dimensional manifold from the original high-dimensional space. However, since the mappings are usually implicit, it is difficult to build a connection between the classification results and the original features. Apart from the prediction results, industry data analysis also focus on the features themselves. Therefore, methods that are able to combine feature selection with dimension reduction is often needed. This paper proposed a hybrid method for nonlinear dimension reduction and feature selection based on Locally Linear Embedding (LLE). LLE and many filter feature selection shares a common procedure of neighbor search. By integrating nearest neighbor search for both LLE and ReliefF, and adjusting the distance measure in supervised LLE with results from feature selection, the proposed method could connect the feature selection process with the nonlinear model, extract the critical features for further analysis, and improve the performance of process modelling. Two dataset from real industry processes are also illustrated as case studies.

Dimensionality reduction is a key step in hyperspectral image processing. Recent investigations are looking into nonlinear manifold learning for dimensionality reduction in hyperspectral imagery. Nonlinear manifold learning methods such as Isomap, and Local Linear Embedding are use to recover the low dimensional representation of an unknown nonlinear manifold for high dimensional data where it is important to retain the neighborhood structure of the manifold. Although these algorithms use different philosophies for recovering the nonlinear manifold, they all incorporate neighborhood information from each data point to construct a weighted graph having the data points as vertices. Thus the performance of these methods highly depends on how well these neighborhoods are selected, since all subsequent steps rely on it. The k -NN algorithm is the most widely used technique for neighbor selection in manifold learning. However, it can result in a disconnected graph and it does not fully exploit spatial neighborhood information from the image in selecting points to form the neighborhoods. In this paper, recently proposed methods for constructing the weighted graph in manifold learning are studied: k -VC, k -EC and k -MST, which have the advantage of creating connected graphs and have performed well in artificial data sets. Spatial information of the hyperspectral images is included in the manifold learning process by using spatial coherence. Experiments are conducted with artificial data and hyperspectral images. For hyperspectral images, classification accuracy was used as an indirect measure to understand how the low dimensional embedding of the data reduces dimensionality while still maintaining good discrimination performance.

Recently, many dimensionality reduction algorithms, including local methods and global methods, have been presented. The representative local linear methods are locally linear embedding (LLE) and linear preserving projections (LPP), which seek to find an embedding space that preserves local information to explore the intrinsic characteristics of high dimensional data. However, both of them still fail to nicely deal with the sparsely sampled or noise contaminated datasets, where the local neighborhood structure is critically distorted. On the contrary, principal component analysis (PCA), the most frequently used global method, preserves the total variance by maximizing the trace of feature variance matrix. But PCA cannot preserve local information due to pursuing maximal variance. In order to integrate the locality and globality together and avoid the drawback in LLE and PCA, in this paper, inspired by the dimensionality reduction methods of LLE and PCA, we propose a new dimensionality reduction method for face recognition, namely, unsupervised linear difference projection (ULDP). This approach can be regarded as the integration of a local approach (LLE) and a global approach (PCA), so that it has better performance and robustness in applications. Experimental results on the ORL, YALE and AR face databases show the effectiveness of the proposed method on face recognition.

Detection of maize seed germination rate based on improved locally linear embedding

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Endmember extraction is the process of selecting a collection of pure signature spectra of the materials present in a hyperspectral scene. Most of the spectral-based endmember extraction methods relay on the ability to discriminate between pixels based on their spectral characteristics and the assumption that pure pixels exist in the image. In some cases, though pure pixels are available inside image, spectral complexity of the image (e.g. low spectral contrast) makes it difficult to extract the best endmember candidates from hyperspectral imagery. This paper investigates the use of statistical convex partitioning (SCP) as a preprocessing tool for endmember extraction. The SCP method comprises three main steps: 1) partitioning input hyperspectral data set into partitions or so called convex regions using K-mean clustering algorithm; 2) finding the best candidate endmembers for each convex region; and, 3) comparing and listing of candidate endmembers extracted from each partition in order of spectral similarity. In order to demonstrate the performance of the proposed method, the sequential maximum angle convex cone (SMACC) algorithm was used to extract endmembers of each partition and the results were compared to pixel purity index (PPI). Optimum number of convex regions as well as the impact of different dimensionality reduction transforms, principal component analysis (PCA), minimum noise fraction (MNF), and independent component analysis (ICA) were also investigated. Experimental results on both simulated and real AVIRIS hyperspectral image indicate that SCP is an effective method to preprocess hyperspectral data spectrally and extract low contrast and similar endmembers effectively.

Abstract. An essential step in the characterization of surface materials using hyperspectral image analysis is image classification using endmembers. Spectral unmixing is the best method for hyperspectral image classification. This method assumes that the pixel-topixel variability in scene results from varying proportions of spectral endmembers. Spectral endmembers can be derived from the imagery or measurements in the laboratory or field. The primary objective of this paper was to assess the ability of extracted endmembers against some different solutions for extraction of endmembers for hyperspectral image classification. In this paper, we compared the Pixel Purity Index (PPI) and the Sequential Maximum Angle Convex Cone (SMACC) as two popular methods of endmember extraction with library and field spectral. We used spectral information divergence for detection desirable endmembers from field spectral. For accuracy assessment of spectral mixture analysis and production of endmember abundance images for each of methods, the linear spectral unmixing algorithm is used. After a comparison between the results of these methods, it has been verified that field spectral have a better classification result in comparing with other endmember extraction methods. Also, the PPI has reliable results as an automatic endmember extraction method in comparing.

The endmember extraction is a challenging problem in spectral unmixing (SU) of a mixed pixel in hyperspectral imagery. There are plenty of attempts to solve the endmember extraction problem. Still, the pure pixel assumption-based algorithms have probably been used most in solving the endmember extraction of SU due to the light computational burden. These pure pixel assumption-based algorithms usually follow one of the criteria: (1) Maximum simplex volume or (2) Extreme projection on a subspace. We propose a novel integrated framework that uses both the criteria mentioned above and the proposed one is referred to as the Convexity-based Pure Index (CPI) algorithm. The CPI generates a fixed number of convex sets based on the number of available bands in the hyperspectral image. The algorithm defines the purity score based on the availability of pixels in the convex sets for the two-band data. The CPI has been compared with contemporary algorithms such as Automatic Target Generation Process (ATGP), Vertex Component Analysis (VCA), Pixel Purity Index (PPI), Successive Volume MAXimization (SVMAX), Alternating Volume MAXimization (AVMAX), TRIple-P: P-norm based Pure Pixel identification (TRIP), Successive Decoupled Volume Max–Min (SDVMM), Negative ABundance-Oriented (NABO), and Entropy-based Convex Set Optimization (ECSO). The metrics, Spectral Angle Distance (SAD) and Spectral Information Divergence (SID) used in the comparison were improved up to 5.9% and 9%, respectively. The CPI outperforms prevailing algorithms on real benchmark data and new AVIRIS-NG data. The robustness of the CPI is also tested for various noisy synthetic data. The efficacy of the proposed algorithm is also tested by using qualitative analysis by visualizing the spectra comparison, and abundance maps for all real data.

Hyperspectral imagery is a class of image data which is used in many scientific areas, most notably, medical imaging and remote sensing. It is characterized by a wealth of spatial and spectral information. Over the last years, many algorithms have been developed with the purpose of finding "spectral endmembers," which are assumed to be pure signatures in remotely sensed hyperspectral data sets. Such pure signatures can then be used to estimate the abundance or concentration of materials in mixed pixels, thus allowing sub-pixel analysis which is crucial in many remote sensing applications due to current sensor optics and configuration. One of the most popular endmember extraction algorithms has been the pixel purity index (PPI), available from Kodak's Research Systems ENVI software package. This algorithm is very time consuming, a fact that has generally prevented its exploitation in valid response times in a wide range of applications, including environmental monitoring, military applications or hazard and threat assessment/tracking (including wildland fire detection, oil spill mapping and chemical and biological standoff detection). Field programmable gate arrays (FPGAs) are hardware components with millions of gates. Their reprogrammability and high computational power makes them particularly attractive in remote sensing applications which require a response in near real-time. In this paper, we present an FPGA design for implementation of PPI algorithm which takes advantage of a recently developed fast PPI (FPPI) algorithm that relies on software-based optimization. The proposed FPGA design represents our first step toward the development of a new reconfigurable system for fast, onboard analysis of remotely sensed hyperspectral imagery.

Hyperspectral imagery (HSI) has high spectral and spatial resolutions, which are essential for anomaly detection (AD). Many anomaly detectors assume that the spectrum signature of HSI pixels can be modeled with a Gaussian distribution, which is actually not accurate and often leads to many false alarms. Therefore, a sparsity model without any distribution hypothesis is usually employed. Dimensionality reduction (DR) as a preprocessing step for HSI is important. Principal component analysis as a conventional DR method is a linear projection and cannot exploit the nonlinear properties in hyperspectral data, whereas locally linear embedding (LLE) as a local, nonlinear manifold learning algorithm works well for DR of HSI. A modified algorithm of sparsity divergence index based on locally linear embedding (SDI-LLE) is thus proposed. First, kernel collaborative representation detection is adopted to calculate the sparse dictionary matrix of local reconstruction weights in LLE. Then, SDI is obtained both in the spectral and spatial domains, where spatial SDI is computed after DR by LLE. Finally, joint SDI, combining spectral SDI and spatial SDI, is computed, and the optimal SDI is performed for AD. Experimental results demonstrate that the proposed algorithm significantly improves the performance, when compared with its counterparts.

Local Linear Embedding with Adaptive Neighbors

Nonnegative matrix factorization (NMF), which can lead to nonsubtractive parts-based representation, has been demonstrated to be effective for dimensionality reduction of hyperspectral imagery (HSI). However, existing NMF methods applied to HSI use only a single spectral feature and do not take into consideration spatial information, such as texture or morphological features, while it has been widely acknowledged that exploiting multiple features can improve performance. Consequently, a variant of orthogonal NMF, which can not only achieve a nonnegative factorization but also exploit the complementary information that arises among heterogeneous features, is proposed for hyperspectral dimensionality reduction. The proposed method, which couples orthogonal NMF with a previous multiple-features-combining algorithm, yields a discriminative low-dimensional feature representation that matches the intuition that parts should sum to produce a whole. An efficient multiplicative updating procedure is derived, and its local convergence is guaranteed theoretically. Experimental results on two hyperspectral data sets demonstrate the effectiveness of the proposed method.

Locally linear embedding (LLE) depends on the Euclidean distance (ED) to select the k-nearest neighbors. However, the ED may not reflect the actual geometry structure of data, which may lead to the selection of ineffective neighbors. The aim of our work is to make full use of the local spectral angle (LSA) to find proper neighbors for dimensionality reduction (DR) and classification of hyperspectral remote sensing data. At first, we propose an improved LLE method, called local spectral angle LLE (LSA-LLE), for DR. It uses the ED of data to obtain large-scale neighbors, then utilizes the spectral angle to get the exact neighbors in the large-scale neighbors. Furthermore, a local spectral angle-based nearest neighbor classifier (LSANN) has been proposed for classification. Experiments on two hyperspectral image data sets demonstrate the effectiveness of the presented methods.

Manifold learning and its various algorithms have attracted more and more attention in recent years. Locally linear embedding (LLE) algorithm, as a classical nonlinear dimensionality reduction algorithm, has been widely used in many technical fields, such as image classification and object recognition. This paper proposes an improved algorithm based on LLE algorithm–Sparse LLE, which applies a new method of classification in the process of finding K-nearest neighbours with original LLE algorithm. According to the experiment results, the new algorithm can not only guarantee the effect of dimensionality reduction, but also greatly reduce the processing time of the dimensionality reduction of datasets.