Performance evaluation of rotation forest for svm-based recursive feature elimination using hyperspectral imagery

Landsat-like moderate resolution remote sensing images are widely used in land use and land cover (LULC) classification. Limited by coarser resolutions, most of the traditional LULC classifications that are based on moderate resolution remote sensing images focus on the spectral features of a single pixel. Inspired by the spatial evaluation methods in landscape ecology, this study proposed a new method to extract neighborhood characteristics around a pixel for moderate resolution images. 3 landscape-metric-like indexes, i.e., mean index, standard deviation index, and distance weighted value index, were defined as adjacent region features to include the surrounding environmental characteristics. The effects of the adjacent region features and the different feature set configurations on improving the LULC classification were evaluated by a series of well-controlled LULC classification experiments using K nearest neighbor (KNN) and support vector machine (SVM) classifiers on a Landsat 8 Operational Land Imager (OLI) image. When the adjacent region features were added, the overall accuracies of both the classifiers were higher than when only spectral features were used. For the KNN and SVM classifiers that used only spectral features, the overall accuracies of the LULC classification were 85.45% and 88.87%, respectively, and the accuracies were improved to 94.52% and 96.97%. The classification accuracies of all the LULC types improved. Highly heterogeneous LULC types that are easily misclassified achieved greater improvements. As comparisons, the grey-level co-occurrence matrix (GLCM) and convolutional neural network (CNN) approaches were also implemented on the same dataset. The results revealed that the new method outperformed GLCM and CNN approaches and can significantly improve the classification performance that is based on moderate resolution data.

Feature selection and analysis on correlated gas sensor data with recursive feature elimination

We discriminated different successional forest stages, forest degradation, and land use classes in the Tapajós National Forest (TNF), located in the Central Brazilian Amazon. We used full polarimetric images from ALOS/PALSAR-2 that have not yet been tested for land use and land cover (LULC) classification, neither for forest degradation classification in the TNF. Our specific objectives were: (1) to test the potential of ALOS/PALSAR-2 full polarimetric images to discriminate LULC classes and forest degradation; (2) to determine the optimum subset of attributes to be used in LULC classification and forest degradation studies; and (3) to evaluate the performance of Random Forest (RF) and Support Vector Machine (SVM) supervised classifications to discriminate LULC classes and forest degradation. PALSAR-2 images from 2015 and 2016 were processed to generate Radar Vegetation Index, Canopy Structure Index, Volume Scattering Index, Biomass Index, and Cloude–Pottier, van Zyl, Freeman–Durden, and Yamaguchi polarimetric decompositions. To determine the optimum subset, we used principal component analysis in order to select the best attributes to discriminate the LULC classes and forest degradation, which were classified by RF. Based on the variable importance score, we selected the four first attributes for 2015, alpha, anisotropy, volumetric scattering, and double-bounce, and for 2016, entropy, anisotropy, surface scattering, and biomass index, subsequently classified by SVM. Individual backscattering indexes and polarimetric decompositions were also considered in both RF and SVM classifiers. Yamaguchi decomposition performed by RF presented the best results, with an overall accuracy (OA) of 76.9% and 83.3%, and Kappa index of 0.70 and 0.80 for 2015 and 2016, respectively. The optimum subset classified by RF showed an OA of 75.4% and 79.9%, and Kappa index of 0.68 and 0.76 for 2015 and 2016, respectively. RF exhibited superior performance in relation to SVM in both years. Polarimetric attributes exhibited an adequate capability to discriminate forest degradation and classes of different ecological succession from the ones with less vegetation cover.

With the vigorous development of computer and pattern recognition technologies, using magnetic resonance imaging (MRI) and machine learning methods to assist diagnosis of Alzheimer's disease (AD) has became a research hotspot. A new method based on Support Vector Machine-Recursive Feature Elimination (SVM-RFE) and Linear Discriminant Analysis (LDA) for aided diagnosis of AD is proposed in this study. Firstly, Structural MRI images including 34 patients with AD, 26 subjects with subjective memory complaints (SMC) and 50 normal controls (NC) from the ADNI database are preprocessed to obtain the gray matter volumes of 90 brain regions. Then the fusion idea on SVM-RFE and LDA is used to select the characteristics on the above gray matter volumes. Finally, the selected features are classified by support vector machine (SVM), and the average classification accuracies of AD/NC, AD/SMC and NC/SMC reach respectively 94%, 100% and 93.6%. Compared with SVM-RFE or LDA alone, the average classification results on the fusion idea have obvious advantages. The above experimental results show that the proposed method can effectively extract features and assist doctors for exact diagnosis of AD and SMC.

In a DNA microarray dataset, gene expression data often has a huge number of features(which are referred to as genes) versus a small size of samples. With the development of DNA microarray technology, the number of dimensions increases even faster than before, which could lead to the problem of the curse of dimensionality. To get good classification performance, it is necessary to preprocess the gene expression data. Support vector machine recursive feature elimination (SVM-RFE) is a classical method for gene selection. However, SVM-RFE suffers from high computational complexity. To remedy it, this paper enhances SVM-RFE for gene selection by incorporating feature clustering, called feature clustering SVM-RFE (FCSVM-RFE). The proposed method first performs gene selection roughly and then ranks the selected genes. First, a clustering algorithm is used to cluster genes into gene groups, in each which genes have similar expression profile. Then, a representative gene is found to represent a gene group. By doing so, we can obtain a representative gene set. Then, SVM-RFE is applied to rank these representative genes. FCSVM-RFE can reduce the computational complexity and the redundancy among genes. Experiments on seven public gene expression datasets show that FCSVM-RFE can achieve a better classification performance and lower computational complexity when compared with the state-the-art-of methods, such as SVM-RFE.

In the past two decades, several screening instruments were developed to detect toddlers who may be autistic both in clinical and unselected samples. Among others, the Quantitative CHecklist for Autism in Toddlers (Q-CHAT) is a quantitative and normally distributed measure of autistic traits that demonstrates good psychometric properties in different settings and cultures. Recently, machine learning (ML) has been applied to behavioral science to improve the classification performance of autism screening and diagnostic tools, but mainly in children, adolescents, and adults. In this study, we used ML to investigate the accuracy and reliability of the Q-CHAT in discriminating young autistic children from those without. Five different ML algorithms (random forest (RF), naïve Bayes (NB), support vector machine (SVM), logistic regression (LR), and K-nearest neighbors (KNN)) were applied to investigate the complete set of Q-CHAT items. Our results showed that ML achieved an overall accuracy of 90%, and the SVM was the most effective, being able to classify autism with 95% accuracy. Furthermore, using the SVM–recursive feature elimination (RFE) approach, we selected a subset of 14 items ensuring 91% accuracy, while 83% accuracy was obtained from the 3 best discriminating items in common to ours and the previously reported Q-CHAT-10. This evidence confirms the high performance and cross-cultural validity of the Q-CHAT, and supports the application of ML to create shorter and faster versions of the instrument, maintaining high classification accuracy, to be used as a quick, easy, and high-performance tool in primary-care settings.

In the classification of hyperspectral images with supervised methods, acquisition of ground-truth information for a hyperspectral image is a challenging process in terms of time and cost. Besides, amount of the labeled data also affects the performance of classifiers. In this study, as a solution to this problem, a hyperspectral image classifier is proposed with semi-supervised learning, support vector machine classifier and deep learning. In the first phase to improve the classification performance, limited number of training data is increased by semisupervised learning methodology. Then, the classification process is performed with support vector machines and convolutional neural networks. According to the acquired classification results, a close classification performance is obtained by the system with small number of training data to the supervised classification. Furthermore, deep neural network has reached more successful results than support vector machines.

Accurate glioma grading plays an important role in patient treatment. To investigate the influence of varied texture retrieving models on the efficacy of grading glioma with support vector machine (SVM). Retrospective. In all, 117 glioma patients including 25, 29, and 63 grade II, III, and IV gliomas, respectively, based on WHO 2007. 3.0T MRI/ T1 WI, T2 fluid-attenuated inversion recovery, contrast enhanced T1 , arterial spinal labeling, diffusion-weighted imaging (0, 30, 50, 100, 200, 300, 500, 800, 1000, 1500, 2000, 3000, and 3500 sec/mm2 ), and dynamic contrast-enhanced. Texture attributes from 30 parametric maps were retrieved using four models, including Global, gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), and gray-level size-zone matrix (GLSZM). Attributes derived from varied models were input into radial basis function SVM (RBF-SVM) combined with attribute selection using SVM-recursive feature elimination (SVM-RFE). The SVM model was trained and established with 80% randomly selected data of each category using 10-fold crossvalidation. The model performance was further tested using the remaining 20% data. Ten-fold crossvalidation was used to validate the model performance. Based on 30 parametric maps, 90, 240, 390, or 390 texture attributes were retrieved using the Global, GLCM, GLRLM, or GLSZM model, respectively. SVM-RFE was able to reduce attribute redundancy as well as improve RBF-SVM performance. Training data were oversampled by applying the Synthetic Minority Oversampling Technique (SMOTE) method to overcome the data imbalance problem; test results were able to further demonstrate the classifying performance of the final models. GLSZM using gray-level 64 was the optimal model to retrieve powerful image texture attributes to produce enough classifying power with an accuracy / area under the curve of 0.760/0.867 for the training and 0.875/0.971 for the independent test. Fifteen attributes were selected with SVM-RFE to provide comparable classifying efficacy. When using image textures-based SVM classification of gliomas, the GLSZM model in combination with gray-level 64 and attribute selection may be an optimized solution. 2 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;49:1263-1274.

This study proposes a workflow for land use and land cover (LULC) classification of Advanced Land Observing Satellite-2 (ALOS-2) Phased Array type L-band Synthetic Aperture Radar-2 (PALSAR-2) images of the Brazilian tropical savanna (Cerrado) biome. The following LULC classes were considered: forestlands; shrublands; grasslands; reforestations; croplands; pasturelands; bare soils/straws; urban areas; and water reservoirs. The proposed approach combines polarimetric attributes, image segmentation, and machine-learning procedures. A set of 125 attributes was generated using polarimetric ALOS-2/PALSAR-2 images, including the van Zyl, Freeman–Durden, Yamaguchi, and Cloude–Pottier target decomposition components, incoherent polarimetric parameters (biomass indices and polarization ratios), and HH-, HV-, VH-, and VV-polarized amplitude images. These attributes were classified using the Naive Bayes (NB), DT J48 (DT = decision tree), Random Forest (RF), Multilayer Perceptron (MLP), and Support Vector Machine (SVM) algorithms. The RF, MLP, and SVM classifiers presented the most accurate performances. NB and DT J48 classifiers showed a lower performance in relation to the RF, MLP, and SVM. The DT J48 classifier was the most suitable algorithm for discriminating urban areas and natural vegetation cover. The proposed workflow can be replicated for other SAR images with different acquisition modes or for other types of vegetation domains.

The paradigm shift brought by deep learning in land cover object classification in hyperspectral images (HSIs) is undeniable, particularly in addressing the intricate 3D cube structure inherent in HSI data. Leveraging convolutional neural networks (CNNs), despite their architectural constraints, offers a promising solution for precise spectral data classification. However, challenges persist in object classification in hyperspectral imagery or hyperspectral image classification, including the curse of dimensionality, data redundancy, overfitting, and computational costs. To tackle these hurdles, we introduce the spectrally segmented-enhanced neural network (SENN), a novel model integrating segmentation-based, multi-layer CNNs, SVM classification, and spectrally segmented dimensionality reduction. SENN adeptly integrates spectral–spatial data and is particularly crucial for agricultural land classification. By strategically fusing CNNs and support vector machines (SVMs), SENN enhances class differentiation while mitigating overfitting through dropout and early stopping techniques. Our contributions extend to effective dimensionality reduction, precise CNN-based classification, and enhanced performance via CNN-SVM fusion. SENN harnesses spectral information to surmount challenges in “hyperspectral image classification in hyperspectral imagery”, marking a significant advancement in accuracy and efficiency within this domain.

Considering the form diversity of tree species composition in the Bagong Mountain National Forest Park of China, we mapped tree species utilizing Machine Learning Algorithms (support vector machines (SVM) and random forest (RF) classifiers) based on the OHS-2 hyperspectral satellite image by different datasets which combined spectral information and hyperspectral-derived vegetation indices (VIs) for improving tree species classification and explored the best performance of them. To verify the improvement, the results of physically-based spectral classifiers (spectral angle mapper (SAM) and maximum likelihood (ML) classifiers) were applied to compare with the results of machine learning algorithms. The results indicated an overall accuracy of 94.01%, 96.08%, 82.9% and 79.3% for SVM, RF, SAM and ML classifiers of the best performance using different datasets. Highest accuracies resulted from two machine learning algorithms classifiers; SVM and RF compared to SAM and ML classifiers. Although SVM outperformed RF when using all hyperspectral bands and VIs, the overall accuracy of the RF classifier is higher when compared to the SVM classifier using VIs combined selected features. Meanwhile, the RF classifier performed better than SVM after removing the redundancy of spectral data in training samples. Moreover, the machine learning algorithms successfully classified a small number of tree species (Cedrus deodara and Pterocarya stenoptera C. DC.) in the study area, but the physical spectroscopy-based method failed to classify these species. Such integration strategy improved the effectiveness of enhancing the accuracy of tree species classification and mapping their distribution on broad spatial and temporal scales using machine learning algorithms and hyperspectral imagery.

Hyperspectral image (HSI) classification has many applications in different diverse research fields. We propose a method for HSI classification using principal component analysis (PCA), 2D spatial convolution, and support vector machine (SVM). Our method takes advantage of correlation in both spatial and spectral domains in an HSI data cube at the same time. We use PCA to reduce the dimensionality of an HSI data cube. We then perform spatial convolution to the dimension-reduced data cube once and then to the convolved data cube for the second time. As a result, we have generated two convolved PCA output data cubes in a multiresolution way. We feed the two convolved data cubes to SVM to classify each pixel to one of the known classes. Experiments on three widely used hyperspectral data cubes (i.e., Indian Pines, Pavia University, and Salinas) demonstrate that our method can improve the classification accuracy significantly when compared to a few existing methods. Our method is relatively fast in terms of central processing unit computational time as well.

Hyperspectral image classification is a challenging problem for machine learning methods due to the small number of labelled samples and high spectral variability. In this paper, to solve this problem, a novel superpixel and multi-classifier fusion (SMCF)-based classification method for hyperspectral images is proposed. This method takes full advantage of the spectral information of superpixels and the spatial information of hyperspectral images and includes the following three steps. First, superpixels are used to increase the number of training samples and their spectral diversity. Second, label propagation (LP) is used to classify the hyperspectral images. Although LP is an efficient semi-supervised classification method, the corresponding performance is poor for certain land cover types with dispersed spatial distributions. Thus, a support vector machine (SVM) classifier is introduced to classify the hyperspectral images. Finally, the results of the SVM and LP classifiers are combined using our new class-specific weighted fusion algorithm. In the experiments, we selected three widely used and real hyperspectral data sets for evaluation. The final classification performance was evaluated based on two common metrics: the overall accuracy (OA) and the Kappa coefficient. The experimental results show that the proposed SMCF method is superior to six well-known classification methods, even when only 1% or less of the labelled samples are used.

Selecting highly predictive features from the high dimensional dataset is a formidable task. The existing feature selection algorithms available today are not dealing with multiple equally predictive subsets of features. We strongly believe that there is other subsets of features as well, which can give equivalent predictive accuracy as that of state-of-the-art algorithms. Statistically equivalent signature (SES) is one such feature selection algorithm, which is centred on constraint-based learning of Bayesian networks. The proposed model selects equivalent subsets of features from oral squamous cell carcinoma (OSCC) dataset with the help of SES. To inspect the validity of SES algorithms output, we are using K-nearest neighbour (KNN), support vector machine (SVM) and neural networks (NN) on each subset of predictive features. Finally, the results of proposed technique is compared with support vector machine - recursive feature elimination (SVM-RFE). SES produces more stable accuracy as compared to SVM-RFE.

The main objective of this PhD research has been to utilize machine learning techniques on near infrared spectroscopy (NIRS) signals, for the development of highly accurate and clinically practical biomarkers for the objective assessment of pain perception. While advances in medical imaging technology have significantly improved the scientific knowledge in regards to the brain's response to noxious stimuli, there remains an unmet clinical need for a practical, inexpensive tool for the reliable and objective assessment of pain perception. Even though functional imaging modalities such as fMRI and PET scans deliver superior spatial information, they are not readily accessible for routine clinical use. On the other hand NIRS is non-invasive, safe, portable and affordable with a short setup time. These features make NIRS ideal for clinical applications. In this thesis we used the cold pressor test to induce different levels of pain in healthy subjects while the NIRS signal was recorded from the frontal regions of the brain. We extracted 54 features from each dataset and used machine learning techniques, logistic regression and support vector machine, to classify the signals based on the self-reported pain scores. To select the model for machine learning, we developed our feature selection algorithm based on a RFE-SVM (recursive feature elimination - support vector machine) method to find subsets of feature space with the highest classification capability. Through this process we identified a subset of 10 features which could distinguish high-pain from low-pain stimuli with an accuracy of 85% (Leave-one-out cross validation). Moreover we applied functional data analysis on the collected NIRS data and converted discrete samples to continuous curves. This time we used the same RFE-SVM method on the coefficients of fDA bases (as opposed to extracted features) and we achieved 94% of accuracy to classify low-pain high-pain signals. Then using machine learning techniques (k-means and hierarchical clustering) we found clusters in the data which covered low pain and high pain groups with an accuracy of 91.2%. The center of these clusters can represent the prototype NIRS response of that pain level. Our approaches provided trial-by-trial predictions of pain level from NIRS measurement for each individual (as opposed to methods based on responses averaged across many trials and subjects), and thus, represent a step towards the goal of establishing an objective clinical bio marker of pain perception. Further refinement of proposed methods, including incorporating more datasets and employing other noxious stimuli, is required to make the NIRS technique a powerful clinical tool for pain assessment.