Comparative Study and Analysis of Dimensionality Reduction Techniques for Hyperspectral Data

Abstract. The existence of various natural objects such as grass, trees, and rivers along with artificial manmade features such as buildings and roads, make it difficult to classify ground objects. Consequently using single data or simple classification approach cannot improve classification results in object identification. Also, using of a variety of data from different sensors; increase the accuracy of spatial and spectral information. In this paper, we proposed a classification algorithm on joint use of hyperspectral and Lidar (Light Detection and Ranging) data based on dimension reduction. First, some feature extraction techniques are applied to achieve more information from Lidar and hyperspectral data. Also Principal component analysis (PCA) and Minimum Noise Fraction (MNF) have been utilized to reduce the dimension of spectral features. The number of 30 features containing the most information of the hyperspectral images is considered for both PCA and MNF. In addition, Normalized Difference Vegetation Index (NDVI) has been measured to highlight the vegetation. Furthermore, the extracted features from Lidar data calculated based on relation between every pixel of data and surrounding pixels in local neighbourhood windows. The extracted features are based on the Grey Level Co-occurrence Matrix (GLCM) matrix. In second step, classification is operated in all features which obtained by MNF, PCA, NDVI and GLCM and trained by class samples. After this step, two classification maps are obtained by SVM classifier with MNF+NDVI+GLCM features and PCA+NDVI+GLCM features, respectively. Finally, the classified images are fused together to create final classification map by decision fusion based majority voting strategy.

Hyperspectral imaging is concerned with the measurement, analysis, and interpretation of spectra acquired from a given scene (or specific object) at a short, medium or long distance by an airbone or satellite sensor. Over the last few years, hyperspectral image data sets have been collected for a great amount of locations over the world, using a variety of instruments for Earth observation. Despite the increasing importance of hyperspectral images in remote sensing applications, there is no common repository of hyperspectral data intended to distribute and share hyperspectral data sets in the community. Quite opposite, the hyperspectral data sets which are available for public use are spread among different storage locations and present significant heterogeneity regarding the storage format, associated meta-data (if any), or ground-truth availability. As a result, the development of a standardized hyperspectral data repository is a highly desired goal in the remote sensing community. In this paper, we take a necessary first step towards the development of a digital repository for remotely sensed hyperspectral data. The proposed system allows uploading new hyperspectral data sets along with meta-data, ground-truth and analysis results, with the ultimate goal of sharing publicly available hyperspectral images within the remote sensing community. The database has been designed in order to allow storing relevant information for the hyperspectral data available through the system, including basic image characteristics (width, height, number of bands, format) and more advanced meta-data (ground-truth information, publications in which the data has been used). The current implementation consists of a front-end to ease the management of images through a web interface, thus containing both synthetic and real hyperspectral images from two highly representative instruments, such as NASAs Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite Mining District in Nevada. Most importantly, the developed system includes a spectral unmixing-based content based image retrieval (CBIR) functionality which allows searching for images on the spectral unmixing information (spectrally pure components or endmembers and their associated abundances in the scene). This information is stored as meta-data associated to each hyperspectral image instance, and then used to search and retrieve images based on information content. This paper presents the design of the system and a preliminary validation of the unmixing-based retrieval functionality using both synthetic and real hyperspectral images stored in the database.

Hyperspectral remote sensing data contains a large number of spectral bands along with a large a number of features. So for classification purposes, it becomes imperative to reduce the number of spectral bands as well as to reduce the number of features to achieve higher performance in terms of accuracy and computational complexity. Principal component analysis (PCA) and minimum noise fraction (MNF) were used for dimension reduction of spectral bands, on the other hand correlation based feature selection (CFS) technique was employed for selection of features. Parametric classifier like maximum likelihood classifier (MLC), advanced non parametric classifiers like support vector machine (SVM), Multilayer Perceptron (MLP) and ensemble random forest (RF) classifier were used for investigation in two test datasets of different land cover characteristics of Hyperion sensor. It was observed that all of the classifiers have behaved differently with the dimension reduced dataset produced either in spectral bands or feature level. Most of the classifiers have achieved higher performance with dimension reduced dataset produced by MNF as compared to PCA. On the other hand, dataset with selected features could not give noticeably better classification performance than the datasets with entire features. However, computational complexity of the classifiers has been reduced greatly while they were classified with selected features.

Extracting proper features is crucial to the performance of a pattern recognition system. Since the goal of a pattern recognition system is to recognize a pattern correctly, a natural measure of goodness of extracted features is the probability of classification error. However, popular feature extraction techniques like principal component analysis (PCA), Fisher linear discriminant analysis (FLD), and independent component analysis (ICA) extract features that are not directly related to the classification accuracy. In this paper, we present two linear discriminant analysis algorithms (LDA) whose criterion functions are directly based on minimum probability of classification error, or the Bayes error. We term these two linear discriminants as recursive Bayesian linear discriminant I (RBLD-I) and recursive Bayesian linear discriminant II (RBLD-II). Experiments on databases from UCI Machine Learning Repository show that the two novel linear discriminants achieve superior classification performance over recursive FLD (RFLD).

Dimensionality reduction techniques are widely used in machine learning to reduce the computational complexity of the model and improve its performance by identifying the most relevant features. In this research paper, we compare various dimensionality reduction techniques, including Principal Component Analysis(PCA), Independent Component Analysis(ICA), Local Linear Embedding(LLE), Local Binary Patterns(LBP), and Simple Autoencoder, on the Olivetti dataset, which is a popular benchmark dataset in the field of face recognition. We evaluate the performance of these dimensionality reduction techniques using various classification algorithms, including Support Vector Classifier (SVC), Linear Discriminant Analysis (LDA), Logistic Regression (LR), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). The goal of this research is to determine which combination of dimensionality reduction technique and classification algorithm is the most effective for the Olivetti dataset. Our research provides insights into the performance of various dimensionality reduction techniques and classification algorithms on the Olivetti dataset. These results can be useful in improving the performance of face recognition systems and other applications that deal with high-dimensional data.

In this paper, we investigate the application of Fisher’s linear discriminant analysis (FLDA) to hyperspectral remote sensing image classification. The basic idea of FLDA is to design an optimal transform so that the classes can be well separated in the low-dimensional space. The practical difficulty of applying FLDA to hyperspectral images includes the unavailability of enough training samples and unknown information for all the classes present. So the original FLDA is modified to avoid the requirements of complete class knowledge, such as the number of actual classes present. We also investigate the performance of the class of principal component analysis (PCA) techniques prior to FLDA and find that the interference and noise adjusted PCA (INAPCA) can provide the improvement in the final classification.KeywordsFisher’s Linear Discriminant AnalysisDimensionality ReductionClassificationHyperspectral Imagery

Endmember extraction is the process of selecting a collection of pure signature spectra of the materials present in a hyperspectral scene. Most of the spectral-based endmember extraction methods relay on the ability to discriminate between pixels based on their spectral characteristics and the assumption that pure pixels exist in the image. In some cases, though pure pixels are available inside image, spectral complexity of the image (e.g. low spectral contrast) makes it difficult to extract the best endmember candidates from hyperspectral imagery. This paper investigates the use of statistical convex partitioning (SCP) as a preprocessing tool for endmember extraction. The SCP method comprises three main steps: 1) partitioning input hyperspectral data set into partitions or so called convex regions using K-mean clustering algorithm; 2) finding the best candidate endmembers for each convex region; and, 3) comparing and listing of candidate endmembers extracted from each partition in order of spectral similarity. In order to demonstrate the performance of the proposed method, the sequential maximum angle convex cone (SMACC) algorithm was used to extract endmembers of each partition and the results were compared to pixel purity index (PPI). Optimum number of convex regions as well as the impact of different dimensionality reduction transforms, principal component analysis (PCA), minimum noise fraction (MNF), and independent component analysis (ICA) were also investigated. Experimental results on both simulated and real AVIRIS hyperspectral image indicate that SCP is an effective method to preprocess hyperspectral data spectrally and extract low contrast and similar endmembers effectively.

Different classification techniques are being designed and under usage for classification of hyper spectral images. The usage of these classifiers differ for different type of hyper spectral data and application domain. The performance of these classifiers are influenced by feature preprocessing stage. In this research work we have investigated the impact of feature preprocessing using Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) on classification stage. The experiment is carried out using three sets of hyperspectral data. Classifications were carried out using three different classification techniques namely Maximum Likelihood Estimation (MLE), Constraint Energy Minimization (CEM) and Spectral Angle Mapper (SAM) on the preprocessed hyperspectral data. It is observed that the impact of PCA and LDA on the classification techniques are in two aspects (a) The preprocessing techniques facilitates to achieve high classification accuracy even with less number of training features and (b) Preprocessing expedites the classification process for large data sets. Also it can be concluded that PCA outperforms LDA in case of noisy data.

A derivative of Fisher's Linear Discriminant Analysis (FLDA), named Fisherapples for the recognition of apple lesions which is not sensitive to large variations in illumination is proposed in this paper. We make use of the linear projection that is orthogonal to the within-class scatter of the apple images from a high-dimensional image space to a considerably low-dimensional image space. It separates the data-cases well, projecting away variations in lighting. Our approach maximizes the ratio of between-class scatter to that of within-class scatter of apple lesions, i.e., we can get maximal between-class distances and minimal within-class distances after projection. This implies that the gap between the classes becomes bigger and ensures optimal separability in the new space. Besides, we take advantage of Principal Component Analysis (PCA) to project the set of apple images to a lower dimensional space in order to overcome the complication of the singular within-class scatter matrix. After that, the resulting within-class scatter becomes nonsingular and subsequently we can use standard FLDA to reduce the dimension further. Consequently, it is effortless for the computer to calculate the result. Experimental results demonstrate that Fisherapples performs better in apple lesion recognition than PCA.

Hyperspectral image based automatic target recognition (ATR) systems often project the high dimensional hyperspectral reflectance signatures onto a lower dimensional subspace using techniques such as principal components analysis (PCA), Fisher's linear discriminant analysis (LDA) and stepwise LDA. In a general classification framework, these projections are sub-optimal, and in the absence of sufficient training data, are likely to be ill conditioned. In recent work, the authors proposed a divide and conquer approach that partitions the hyperspectral space into contiguous subspaces followed by multi-classifiers and decision fusion. Although this technique alleviated the small sample size problem and provided a good recognition performance in light and moderate pixel mixing, the performance significantly decreased under severe mixing conditions, as does with conventional ATR techniques. In this work, the authors propose a kernel discriminant analysis based projection in each subspace of the partition, followed by the multi-classifier, decision fusion framework to ensure robust recognition even in severe pixel mixing conditions. The performance of the proposed system (as measured by overall target recognition accuracies) is greatly superior to conventional dimensionality reduction techniques, as well as the more recently proposed divide-and-conquer technique.

The nondestructive characterization of citrus varieties (Egyptian sweet orange, Lane Late navel orange, Australian orange, and Blood orange) was developed based on near-infrared diffuse reflectance spectroscopy (NIRDRS) together with principal component analysis (PCA) and Fisher linear discriminant analysis (FLDA). An experiment for the penetration of NIRDRS into the peel was designed and the effects of different spectral acquisition points were investigated. Pretreatments were used to eliminate the spectral interferences. As an unsupervised pattern recognition method, PCA was used to establish the characterization models. Furthermore, supervised pattern recognition based on PCA and FLDA was employed to enhance the accuracy. The results of the penetration experiments show that near-infrared light enters the citrus peel and is able to characterize the internal composition. Even with the optimized spectral pretreatment, accurate characterization of citrus varieties was not achieved by PCA. However, the accurate characterization of citrus varieties was provided by PCA-FLDA. The accuracies of four spectral acquisition points are 95%, while the characterization accuracies of six spectral acquisition points are 100% combined with optimized spectral pretreatment. Therefore, NIRDRS with PCA-FLDA is suitable for the rapid and nondestructive characterization of citrus varieties.

Assessment of spectral reduction techniques for endmember extraction in unmixing of hyperspectral images

A computer vision system for automatic recognition and classification of five varieties of plant leaves under controlled laboratory imaging conditions, comprising: 1–Cydonia oblonga (quince), 2–Eucalyptus camaldulensis dehn (river red gum), 3–Malus pumila (apple), 4–Pistacia atlantica (mt. Atlas mastic tree) and 5–Prunus armeniaca (apricot), is proposed. 516 tree leaves images were taken and 285 features computed from each object including shape features, color features, texture features based on the gray level co-occurrence matrix, texture descriptors based on histogram and moment invariants. Seven discriminant features were selected and input for classification purposes using three classifiers: hybrid artificial neural network–ant bee colony (ANN–ABC), hybrid artificial neural network–biogeography based optimization (ANN–BBO) and Fisher linear discriminant analysis (LDA). Mean correct classification rates (CCR), resulted in 94.04%, 89.23%, and 93.99%, for hybrid ANN–ABC; hybrid ANN–BBO; and LDA classifiers, respectively. Best classifier mean area under curve (AUC), mean sensitivity, and mean specificity, were computed for the five tree varieties under study, resulting in: 1–Cydonia oblonga (quince) 0.991 (ANN–ABC), 95.89% (ANN–ABC), 95.91% (ANN–ABC); 2–Eucalyptus camaldulensis dehn (river red gum) 1.00 (LDA), 100% (LDA), 100% (LDA); 3–Malus pumila (apple) 0.996 (LDA), 96.63% (LDA), 94.99% (LDA); 4–Pistacia atlantica (mt. Atlas mastic tree) 0.979 (LDA), 91.71% (LDA), 82.57% (LDA); and 5–Prunus armeniaca (apricot) 0.994 (LDA), 88.67% (LDA), 94.65% (LDA), respectively.

Principal component analysis (PCA) is linear method for feature extraction that is known as Karhonen Loove method. PCA was first proposed to recognize face by Turk and Pentland, and was also known as eigenface in 1991 [Turk, 1991]. However, PCA has some weaknesses. The first, it cannot capture the simplest invariance of the face image [Arif et al., 2008b] , when this information is not provided in the training data. The last, the result of feature extraction is global structure [Arif, 2008]. The PCA is very simple, has overcome curse of dimensionality problem, this method have been known and expanded by some researchers to recognize face such as Linear Discriminant Analysis (LDA)[Yambor, 2000; A.M. Martinez, 2003; J.H.P.N. Belhumeur 1998], Linear Preserving Projection that known Lapalacianfaces [Cai, 2005; Cai et al, 2006; Kokiopoulou, 2004; X. He et al., 2005], Independent Component Analysis, Kernel Principal Component Analysis [Scholkopf et al., 1998; Sch olkopf 1999], Kernel Linear Discriminant Analysis (KLDA) [Mika, 1999] and maximum feature value selection of nonlinear function based on Kernel PCA [Arif et al., 2008b]. As we know, PCA is dimensionality reduction method based on object appearance by projecting an original ndimensional (row*column) image into k eigenface where k<<n. Although PCA have been developed into some methods, but in some cases, PCA can outperform LDA, LPP and ICA when it uses small sample size.

Three sets of coast live oak (Quercus agrifolia) leaf samples were collected on three dates: 20 April 2002, 23 July 2002 and 11 September 2001, respectively, for Sudden Oak Death (SOD) monitoring. A total of 330 reflectance spectra (covering 350–2500 nm) were measured in the laboratory with a spectrometer FieldSpec®Pro FR. In this study, the spectroscopic determination of two health levels of the coast live oak leaves was conducted with three sets of spectra. We used two classification algorithms, penalized discriminant analysis (PDA) and cross correlogram spectral matching (CCSM), to discriminate between healthy and infected leaves. PDA is a penalized version of Fisher's linear discriminant analysis (LDA) and can considerably improve upon LDA when it is used for the classification of hyperspectral data. CCSM is practised by calculating the cross correlation at different match positions between a test spectrum and a reference spectrum and is also suitable for processing hyperspectral data. Experimental results indicate that the PDA algorithm has produced approximately 7% higher classification accuracy than that produced by CCSM, although both are very low. When considering the subtle spectral differentiation between the two health levels, the PDA method demonstrates its promise as a classification algorithm. Among the 10 spectral ranges, some higher accuracies are produced by both PDA and CCSM algorithms from those spectral range wavelengths shorter than 1400 nm. Based on our experimental results and previous work, existing remote sensing techniques, including airborne or satellite remote sensing and multispectral or hyperspectral remote sensing, may be insufficient for monitoring and mapping disease-induced moisture stress in trees that have recently been infected. However, this does not preclude the analysis of trees at very advanced stages of disease, and the practicality of finding trees within weeks of dying is considerable.