Nonlinear Multiple Kernel Learning With Multiple-Structure-Element Extended Morphological Profiles for Hyperspectral Image Classification

In this paper, a novel multiple kernel learning (MKL) algorithm is proposed for the classification of hyperspectral images. The proposed MKL algorithm adopts a two-step strategy to learn a multiple kernel machine. In the first step, unsupervised learning is carried out to learn a combined kernel from the predefined base kernels. In our algorithms, low-rank nonnegative matrix factorization (NMF) is used to carry out the unsupervised learning and learn an optimal combined kernel. Furthermore, the kernel NMF (KNMF) is introduced to substitute NMF for enhancing the ability of the unsupervised learning with the predefined base kernels. In the second step, the optimal kernel is embedded into the standard optimization routine of support vector machine (SVM). In addition, we address a major challenge in hyperspectral data classification, i.e., using very few labeled samples in a high-dimensional space. Experiments are conducted on three real hyperspectral datasets, and the experimental results show that the proposed algorithms, especially for KNMF-based MKL, achieve the outstanding performance for hyperspectral image classification with few labeled samples when compared with several state-of-the-art algorithms.

Multiple Kernel Learning (MKL) algorithms have recently demonstrated their effectiveness for classifying the data with numerous features. These algorithms aim at learning an optimal composite kernel through combining the basis kernels constructed from different features. Despite their satisfactory results, MKL algorithms assume that the basis kernels are a priori computed. Moreover, they adopt complex optimization methods to train the combination of the basis kernels, which are usually hard to solve and can only handle the binary classification problems. In this paper, a novel MKL framework was introduced in order to address all these issues. This framework optimizes a data-dependent kernel evaluation measure in order to learn both the basis kernels and their combination. The kernel evaluation measure should be able to estimate the goodness of the composite kernel for a multiclass classification problem. In this paper, we defined such a measure based on the similarity between the composite kernel and an ideal kernel. To this end, three different kernel-based similarity measures, namely kernel alignment (KA), centered kernel alignment (CKA), and Hilbert-Schmidt independence criterion (HSIC), were presented. For solving the optimization problem of the proposed MKL framework, we used the metaheuristic optimization algorithms, which in addition to being accurate algorithms can be easily implemented. The performance of the proposed framework was evaluated by classifying the features extracted from two multispectral and hyperspectral datasets. The results showed that this framework outperformed the other state-of-the-art MKL algorithms in terms of both classification accuracy and the computational time.

Kernel Methods have been successfully applied in different tasks and used on a variety of data sample sizes. Multiple Kernel Learning (MKL) and Multilayer Multiple Kernel Learning (MLMKL), as new families of kernel methods, consist of learning the optimal kernel from a set of predefined kernels by using an optimization algorithm. However, learning this optimal combination is considered to be an arduous task. Furthermore, existing algorithms often do not converge to the optimal solution (i.e., weight distribution). They achieve worse results than the simplest method, which is based on the average combination of base kernels, for some real-world applications. In this paper, we present a hybrid model that integrates two methods: Support Vector Machine (SVM) and Multiple Classifier (MC) methods. More precisely, we propose a multiple classifier framework of deep SVMs for classification tasks. We adopt the MC approach to train multiple SVMs based on multiple kernel in a multi-layer structure in order to avoid solving the complicated optimization tasks. Since the average combination of kernels gives high performance, we train multiple models with a predefined combination of kernels. Indeed, we apply a specific distribution of weights for each model. To evaluate the performance of the proposed method, we conducted an extensive set of classification experiments on a number of benchmark data sets. Experimental results show the effectiveness and efficiency of the proposed method as compared to various state-of-the-art MKL and MLMKL algorithms.

Kernel methods have been successfully used in many practical machine learning problems. Choosing a suitable kernel is left to the practitioner. A common way to an automatic selection of optimal kernels is to learn a linear combination of element kernels. In this paper, a novel framework of multiple kernel learning is proposed based on conditional entropy minimization criterion. For the proposed framework, three multiple kernel learning algorithms are derived. The algorithms are experimentally shown to be comparable to or outperform kernel Fisher discriminant analysis and other multiple kernel learning algorithms on benchmark data sets.

Elastic Multiple Kernel Learning

In order to improve the performance of hand multi-movement pattern recognition in myoelectric control system, a multi-feature fusion method is proposed. The effect of identification method based on single surface electromyography (sEMG) feature is limited. To obtain more comprehensive information about multi-movement pattern, we investigate a multi-feature fusion method based on multiple kernel learning framework. General multi-kernel learning (Generalized-MKL) algorithm is adopted to integrate different kinds of features, which takes full advantage of valuable features for movement classification. Each kind of sEMG feature is endowed with a base kernel respectively. The proposed method adaptively combines multiple features together with optimal combination weights and finds a mixed kernel to classify. In the experiment, we classify multiple different hand movements for fifteen participants, such as wrist, gross hand and finger movements. The experimental results indicate that the feature set formed by the proposed fusion method has the lowest classification error value across all the multi-feature sets. The classification error value of the proposed feature set is 2.86%-4.24% lower than the other commonly used feature combinations. It proves the effectiveness of the proposed multi-feature fusion method. The proposed method is quite promising for hand multi-movement classification with low classification error value.

The availability of graphics processing units (GPUs) provides a low-cost solution to real-time processing, which may benefit many remote sensing applications. In this paper, a spectral–spatial classification scheme for hyperspectral images is specifically adapted for computing on GPUs. It is based on wavelets, extended morphological profiles (EMPs), and support vector machine (SVM). Additionally, a preprocessing stage is used to remove noise in the original hyperspectral image. The local computation of the techniques used in the proposed scheme makes them particularly suitable for parallel processing by blocks of threads in the GPU. Moreover, a block-asynchronous updating process is applied to the EMP to speedup the morphological reconstruction. The results over different hyperspectral images show that the execution can be speeded up to $8.2\times$ compared to an efficient OpenMP parallel implementation, achieving real-time hyperspectral image classification while maintaining the high classification accuracy values of the original classification scheme.

With the increasing acquisition ability of hyperspectral remote sensing images, unsupervised cross-temporal classification (UCTC) of hyperspectral images (HSIs) has attracted more and more attention. In this paper, we focus on cross-temporal HSI classification, i.e., using one labeled HSI to classify the other unlabeled HSI. A multiple geodesic flow kernel learning (MGFKL) framework is proposed to exploit both spatial and spectral features for UCTC with bitemporal HSIs and called S2-MGFKL. The proposed S2-MGFKL method first extracts extended multi-attribute profiles (EMAPs) from the original bitemporal HSIs. The spatial features of the bitemporal HSIs obtained by the same attribute filter are paired up, so are the original spectral features. Second, each pair of features from both source and target domains are used to construct multiple geodesic flows. According to the original definition of GFK, we can obtain the construction of Gaussian base GFKs. The base kernels consist of two parts, the spectral part is obtained base on the same geodesic flow (which is constructed on the bitemporal spectral features) by tuning the kernel scale, while the spatial part is obtained under the same kernel scale but different geodesic flows constructed on different spatial feature pairs. After that, the mean rule is adopted to acquire the combined kernel, which is fed into the supervised vector machine (SVM) to implement the cross-temporal classification task. Experiments are conducted on two real HSI data sets, and the results compared with several well-known methods demonstrate the effectiveness of the proposed method.

As a rapid developing research topic in the machine vision field, image-based hand posture recognition has the potential to be an efficient and intuitive tool of human-computer interaction. For improving the accuracy of multi-class hand postures and extending the algorithm generalization,a novel hand posture recognition method is proposed by integrating the multiple image features and multiple kernels learning support vector machine(SVM). Firstly, three types of feature descriptors are extracted to describe the characteristics of a hand posture image. Shape context descriptor represents distribution characteristics of the edge points of the hand posture image. Pyramid histogram of oriented gradient describes characteristics of local and global shape effectively. The Bag of Feature(BOF) algorithm describes the surface texture characteristics of the posture image. Secondly, the Chamfer kernel and histogram intersection kernel are rebuilt to obtain the basis kernels of the features. And the combined kernel is constructed by weighting the basis kernels.So the heterogeneous features fusion realizes. Finally, the classification model and optimal fusion weights are calculated by using multiple kernels learning algorithm. The unknown category posture can be recognized by the trained multiple kernels of SVM. Experiments on Jochen Triesch's hand posture dataset demonstrate that the proposed method obtains higher recognition rate than the traditional single-kernel classifier and other recent methods.

This work was supported in part by the National Science Fund for Excellent Young Scholars under Grant 61522107, by the Natural Science Foundation of China under Grant 61371180, by the China Aerospace Science and Technology Corporation–Harbin Institute of Technology Joint Center for Technology Innovation Fund under Grant CASC-HIT15-1C03, and by the Smart Sea Technical Innovation Foundation, CSSC-SERI.

In very high resolution (VHR) remote sensing (RS) classification tasks, conventional pixel-based contextual information extraction methods such as morphological profiles (MPs), extended MPs (EMPs) and MPs with partial reconstruction (MPPR) with limited numbers, sizes and shapes of structural elements (SEs) cannot perfectly match all sizes and shapes of the objects in an image. To overcome such limitation, we introduce novel spatial feature extractors, namely, the superpixel-guided morphological profiles (SPMPs), where the superpixels are used as SEs in opening by reconstruction and closing by reconstruction operations. Moreover, to avoid possible side effects from unusual maximum and minimum values within superpixels, the mean pixel value of superpixels is adopted (SPMPsM). Additionally, new decision forest based on penalizing the attributes in previous trees, the ForestPA is introduced and evaluated through a comparative investigation on three VHR multi-/hyperspectral RS image classification tasks. Support vector machine and benchmark ensemble classifiers, including bagging, AdaBoost, MultiBoost, ExtraTrees, Random Forest and Rotation Forest, are adopted. The experimental results confirm the effectiveness and superior performances of the proposed SPMPs and SPMPsM relative to those of the MPs and MPPR. Moreover, ForestPA outperforms only bagging and is not suitable for learning from large numbers of samples with high dimensionality from the computational efficiency and classification accuracy perspective.

A nonlinear subspace multiple kernel learning for financial distress prediction of Chinese listed companies

Multiple kernel learning (MKL) is a principled way for kernel fusion for various learning tasks such as classification, clustering, and dimensionality reduction. The least absolute shrinkage and selection operator (Lasso) allows computationally efficient feature selection based on the linear dependence between input features and output values. In this paper, we develop a novel MKL model based on a nonlinear Lasso, that is, the Hilbert–Schmidt independence criterion (HSIC) Lasso. In the proposed model, we first propose the HSIC Lasso‐based MKL formulation, which has a clear statistical interpretation that minimum redundant kernels with maximum dependence on output labels are found and combined, and also that the global optimal solution can be computed efficiently by solving a Lasso optimization problem. After the optimal kernel is obtained, the support vector machine (SVM) is used to select the prediction hypothesis. It is evident that the proposed MKL is a two‐stage kernel learning approach. Extensive experiments on real‐world datasets from the UCI benchmark repository validate the superiority of the proposed model in terms of prediction accuracy. © 2018 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Fisher discrimination multiple kernel dictionary learning for robust identification of nonlinear features in machinery health monitoring

Chips are a well-known product among Small and Medium Enterprises (SMEs). In order to enhance the quality of chips as an SME product, sentiment analysis is a crucial step. In this research, sentiment analysis of chip purchases on the Shopee E-marketplace was conducted using the Natural Language Processing (NLP) method, utilizing the N-Gram Model and Term Frequent-Inverse Document Frequency (TF-IDF) as feature extraction techniques, and the Support Vector Machine (SVM) algorithm for sentiment classification. The objective of this research is to identify the most suitable feature extraction model and optimal SVM kernel type from the options of Linear, Polynomial degree, Gaussian RBF, and Sigmoid kernels. Results from the experiments indicate that the TF-IDF and unigram feature extraction techniques offer the best performance for SVM classification when utilizing the Linear kernel. By labeling the dataset, it was observed that using a lexicon-based approach for sentiment classification resulted in 84.31% of the total reviews being positive. The words "price", "cheap" and "quality" in unigram have the highest weights above 0.040. In the unigram model, linear kernel accuracy and precision performance values are 88.4% and 87.3%. At the same time, the recall performance values is 88.4%. The results of the F1-Score assessment matrix from Unigram were 86.9%, Bigram was 78.5% and Trigram was 77.4%. Ultimately, the unigram model combined with a linear kernel in the SVM algorithm demonstrates strong potential for application in the development of various systems focused on detecting user reviews in the Indonesian language on the Shopee E-Marketplace.