Supervised Locally Linear Embedding based dimension reduction for hyperspectral image classification

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Hyperspectral imagery (HSI) has high spectral and spatial resolutions, which are essential for anomaly detection (AD). Many anomaly detectors assume that the spectrum signature of HSI pixels can be modeled with a Gaussian distribution, which is actually not accurate and often leads to many false alarms. Therefore, a sparsity model without any distribution hypothesis is usually employed. Dimensionality reduction (DR) as a preprocessing step for HSI is important. Principal component analysis as a conventional DR method is a linear projection and cannot exploit the nonlinear properties in hyperspectral data, whereas locally linear embedding (LLE) as a local, nonlinear manifold learning algorithm works well for DR of HSI. A modified algorithm of sparsity divergence index based on locally linear embedding (SDI-LLE) is thus proposed. First, kernel collaborative representation detection is adopted to calculate the sparse dictionary matrix of local reconstruction weights in LLE. Then, SDI is obtained both in the spectral and spatial domains, where spatial SDI is computed after DR by LLE. Finally, joint SDI, combining spectral SDI and spatial SDI, is computed, and the optimal SDI is performed for AD. Experimental results demonstrate that the proposed algorithm significantly improves the performance, when compared with its counterparts.

Unmanned aerial vehicles (UAVs) provide a new research tool to obtain high spatial and temporal resolution imagery at a reduced cost. Rapid advances in miniature sensor technology are leading to greater potentials for ecological research. We demonstrate one of the first applications of UAV lidar and hyperspectral imagery and a fusion method for individual plant species identification and 3D characterization at submeter scales in south‐eastern Arizona, USA. The UAV lidar scanner characterized the individual vegetation canopy structure and bare ground elevation, whereas the hyperspectral sensor provided species‐specific spectral signatures for the dominant and target species at our study area in leaf‐on condition. We hypothesized that the fusion of the two different data sources would perform better than either data type alone in the arid and semi‐arid ecosystems with sparse vegetation. The fusion approach provides 84–89% overall accuracy (kappa values of 0.80–0.86) in target species classification at the canopy scale, leveraging a wide range of target spectral responses in the hyperspectral data and a high point density (50 points/m2) in the lidar data. In comparison, the hyperspectral image classification alone produced 72–76% overall accuracies (kappa values of 0.70 and 0.71). The UAV lidar‐derived digital elevation model (DEM) is also strongly correlated with manned airborne lidar‐derived DEM (R2 = 0.98 and 0.96), but was obtained at a lower cost. The lidar and hyperspectral data as well as the fusion method demonstrated here can be widely applied across a gradient of vegetation and topography to monitor and detect ecological changes at a local scale.

Dimensionality reduction is a key step in hyperspectral image processing. Recent investigations are looking into nonlinear manifold learning for dimensionality reduction in hyperspectral imagery. Nonlinear manifold learning methods such as Isomap, and Local Linear Embedding are use to recover the low dimensional representation of an unknown nonlinear manifold for high dimensional data where it is important to retain the neighborhood structure of the manifold. Although these algorithms use different philosophies for recovering the nonlinear manifold, they all incorporate neighborhood information from each data point to construct a weighted graph having the data points as vertices. Thus the performance of these methods highly depends on how well these neighborhoods are selected, since all subsequent steps rely on it. The k -NN algorithm is the most widely used technique for neighbor selection in manifold learning. However, it can result in a disconnected graph and it does not fully exploit spatial neighborhood information from the image in selecting points to form the neighborhoods. In this paper, recently proposed methods for constructing the weighted graph in manifold learning are studied: k -VC, k -EC and k -MST, which have the advantage of creating connected graphs and have performed well in artificial data sets. Spatial information of the hyperspectral images is included in the manifold learning process by using spatial coherence. Experiments are conducted with artificial data and hyperspectral images. For hyperspectral images, classification accuracy was used as an indirect measure to understand how the low dimensional embedding of the data reduces dimensionality while still maintaining good discrimination performance.

Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features

This paper examines the performance of different machine and deep learning algorithms in classifying colon histological images using different feature extraction methods. The relationship between the feature extraction methods and the selected machine learning methods to improve the classification accuracy is analyzed. Widely used methods like local binary patterns, histograms of oriented gradients, Gabor filter and Dobeshi wavelets are investigated for feature extraction from colon histological images. The features extracted by histogram of oriented gradients and Gabor filter methods were used as a single joint feature vector. And popular machine learning methods such as Support vector machine, Decision trees, Random forest, k-nearest neighbors and Naive Bayesian method were used to classify the selected images. The paper also investigates ensemble methods using gradient bousting and voting classifier as examples. The authors also focus on the study of convolutional neural networks as they are one of the main deep learning methods at the moment. The classification methods selected for analysis are compared in terms of classification accuracy and time taken for training and recognition. All pre-defined and adjustable parameters of both feature extraction methods and classification methods were personally selected by the authors as a result of experimental studies, which were conducted using a software tool created in the Python programming language on a set of LC25000 histological images. The software created is easily customizable and can be used in the future to investigate classification methods on other datasets.

Spinal cord injuries (SCI) occur when the spinal cord is damaged due to any trauma. Treatment of this condition typically involves a long and challenging rehabilitation process. Brain-computer interface (BCI) controlled rehabilitation systems show promise for people with SCI, as they offer repetitive and controlled treatment in the home environment without requiring a specialist. There is a growing demand for electroencephalography (EEG)–based BCI rehabilitation systems, particularly for patients with SCI. In this study, EEG signals from ten SCI patients were analyzed. At the same time, they imagined performing rehabilitation movements for five different hand and arm actions (pronation, supination, palmar grasp, lateral grasp, and hand opening). The study tested manifold learning algorithms, feature extraction, and classification methods in EEG analysis to improve usability in real-time applications. Manifold learning algorithms were used to represent complex and high-dimensional data in a lower-dimensional space, allowing for better representation and separation of the temporal and spatial characteristics of brain activity. The spectral embedding algorithm was used in this study, which, to the best of our knowledge, is the first time this algorithm has been applied to the data of SCI patients. Additionally, we conducted a comparative analysis of commonly encountered methods in the literature, including multi-dimensional scaling (MDS), isometric feature mapping (ISOMAP), local linear embedding (LLE), and t-distributed stochastic neighbor embedding (t-SNE). Machine learning algorithms, such as k-nearest neighbor (k-NN), support vector machine (SVM), and Naive Bayes methods, were used to obtain classification results for both multi-class and binary classes. The combination of pronation and hand open was found to yield the best performance for binary classification of the movements. The study determined that the ISOMAP manifold learning method with the k-NN algorithm is the optimal method for processing times of both the train and the test. The method also demonstrated a high accuracy value of 0.967 and a short time of 0.088 units in multiple classifications, which is promising. The study utilized the spectral embedding method for the first time and achieved an accuracy rate of 0.649 in multi-class classification and 0.933 ± 0.049 in binary-class classification.

A new nonlinear feature extraction method for face recognition

Manifold learning and its various algorithms have attracted more and more attention in recent years. Locally linear embedding (LLE) algorithm, as a classical nonlinear dimensionality reduction algorithm, has been widely used in many technical fields, such as image classification and object recognition. This paper proposes an improved algorithm based on LLE algorithm–Sparse LLE, which applies a new method of classification in the process of finding K-nearest neighbours with original LLE algorithm. According to the experiment results, the new algorithm can not only guarantee the effect of dimensionality reduction, but also greatly reduce the processing time of the dimensionality reduction of datasets.

Hyperspectral imagery (HSI) and light detection and ranging (LiDAR) remote sensing technologies are important ways to obtain surface information. Combining the characteristics and advantages of hyperspectral images and LiDAR DSM data, we propose a classification method for co-classification of hyperspectral images and LiDAR data in this paper. The model uses a dual-branch HSI image and LiDAR data classification method. First, the features of shallow convolution and deep convolution are merged in the spatial feature extraction process of HSI, which combines to focus on more global information. Then the space and spectrum are combined, considering the characteristics of different layers, the network has the characteristics of multiscale. Finally, the LiDAR branch introduces dilated convolution to obtain a larger receptive field and reduce information loss. The classification experimental results on Houston and Trento dataset show the superiority of our proposed method in the collaborative classification of hyperspectral images and LiDAR data.

Traditional joint sparse representation based hyperspectral classification methods define a local region for each pixel. Through representing the pixels within the local region simultaneously, the class of the central pixel is able to be decided. A common limitation of this kind of methods is that only local pixels are considered in such methods, and thus, non-local information will be ignored. In order to improve the classification accuracy with the non-local information of hyperspectral images, a novel hyperspectral image classification based on K nearest neighbors (KNN) sparse representation is proposed in this paper. First, a feature space is defined based on the first principal components of the hyperspectral image and the spatial coordinates of different pixels. Then, in the defined feature space, K non-local neighborhoods of each pixel are found by using the KNN searching scheme. At last, through jointly representing the K pixels with the joint sparse model and comparing the representation residuals, the label of each pixel can be determined. Experiments performed on a widely used real HSI data set show that the proposed method obtain better classification performances when compared with the traditional joint sparse representation method and other recently proposed hyperspectral image classification methods.

Hyperspectral sensors record the reflectance from the Earth's surface over the full range of solar wavelengths with high spectral resolution. The resulting high-dimensional data contain rich information for a wide range of applications. However, for a specific application, not all the measurements are important and useful. The original feature space may not be the most effective space for representing the data. Feature mining, which includes feature generation, feature selection (FS), and feature extraction (FE), is a critical task for hyperspectral data classification. Significant research effort has focused on this issue since hyperspectral data became available in the late 1980s. The feature mining techniques which have been developed include supervised and unsupervised, parametric and nonparametric, linear and nonlinear methods, which all seek to identify the informative subspace. This paper provides an overview of both conventional and advanced feature reduction methods, with details on a few techniques that are commonly used for analysis of hyperspectral data. A general form that represents several linear and nonlinear FE methods is also presented. Experiments using two widely available hyperspectral data sets are included to illustrate selected FS and FE methods.

In this paper, an attempt is made to apply few conventional methods of EEG feature extraction and classification methods and compare their performance for a specific task. Two different feature extraction and classification methods are implemented to classify the mental tasks of EEG signals from a known dataset. For this purpose, the auto regression model and the wavelet transform is used as feature extraction. A combined EEG feature vector is also evaluated on the classification accuracy. The features extracted from these methods are applied to the k-nearest neighbour and support vector machine classifiers separately. Each subject has ten trials of each mental task, in which five trials of each task is used for training the system. The remaining five tasks are used for testing the system. Four different trial combinations of each task are made. The results are evaluated using the confusion matrix. Experimental results specify that each method has specific advantages and disadvantages and is suitable for EEG signal analysis for a specific application.

Underground metal target detection technology has been widely applied in industrial production, resource exploration, and engineering construction, etc. However, due to the influence of non-negligible noise and high dimensionality in collected data, achieving efficient and accurate underground target classification remains a grand challenge for further applications of underground metal target detection on portable devices with limited computing capability and energy supply. This study aimed to seek out robust and efficient data-based strategies to classify the underground metal targets of different shapes and materials based on electromagnetic induction detection. We investigated thirty-three classification strategies based on eleven dimensionality reduction methods, namely, the least absolute shrinkage and selection operator (LASSO), genetic algorithm-support vector machine (GA-SVM), Pearson correlation coefficient (PCC), mutual information (MI), maximal relevance minimal redundancy Pearson correlation (mRMRP), maximal relevance minimal redundancy mutual information (mRMRMI), statistical features (SF), principal component analysis (PCA), kernel principal component analysis (KPCA), locally linear embedding (LLE), and stacked denoising autoencoder (SDAE), and three machine learning models, namely, artificial neural network (ANN), linear support vector machine (L-SVM), and Gaussian Naïve Bayes (GNB). Several parameters, including classification accuracy, the number of features after dimensionality, the feature type importance, and the time consumption were considered to evaluate the data-based classification strategies. Among the classification strategies investigated and considering the above evaluation parameters, the artificial neural network (ANN) classifier assisted with the kernel principal component analysis (KPCA) feature extraction method yielded the best performance in the material-based classification (accuracy:0.99) and the shape-based classification (accuracy:0.99). The locally linear embedding (LLE) improved the robustness of machine learning classifiers and efficiency of the artificial neural network in the material-based classification (improvement of average accuracy:0.17, reduction of classification time cost:14%) and shape-based classification (improvement of average accuracy:0.16, reduction of classification time cost:22%). Our comparative investigation provides a robust and efficient data-based strategy for underground metal target classification, which is significant for applications of underground metal target detection on portable devices with limited computing capability and energy supply. The cross-combination strategy of dimensionality reduction methods and machine learning models provides a way to find the optimal machine learning model for underground target detection.

Based on non-supervised classification methods, multi-temporal images, such as NOAA-AVHRR, SPOT-VEGETATION and MODIS series data, have been used to map regional, continental or global land cover patterns. Because of limitations of the classification method and spatial resolution of images, traditional non-supervised classification often results in many errors in the transition zones in which the spatial distribution of vegetation show segmental patterns. This problem can decrease the integral accuracy of the classification to a certain extent.  In this paper, a new method, the topographical restrictive features, is presented to classify remotely sensed images using the Jinhe watershed in the Loess plateau as a case study. First, 25 relatively independent classes were obtained using the non-supervised classification method with multi-temporal NDVI data derived from the first two bands of MODIS data. Every cell with 500 m spatial resolution of the classification determined by the non-supervised method was divided into 25 homogenous cells with 100 m spatial resolution. Then, topographical features were defined. Information about aspect, slope, elevation, river net structure, and vegetation regional characteristics, derived from 1:250 000 geographical spatial data, ETM+ image and yield data, were used to construct topographical restrictive features. Finally, the classification of every sub-cell was tested using the restrictive features and some cells were reclassified while maintaining their original classification. After the secondary classification, all cells were labeled by land cover type according to the land cover classification system, which was defined previously on the base of the IGBP land cover classification system and UMD system.  Using this new method, the accuracy of the land cover classification increased from 71.88%, in traditional non-supervised classification method, to 84.09%. Inparticular, the accuracy of cropland and urban type classification improved. The method of cell sub-division avoids the shortcomings of traditional classification methods owing to the coarse resolution of the image processing, and makes it more highly probable that land cover types are homogenous within cells. The introduction of topographical restrictive features decreases uncertainty of traditional fuzzy classification and provides more precise distinctive features to classify the fuzzy zone, and thus improves the accuracy of classification.