A multiple attributes convolution kernel with reproducing property

A local–global mixed kernel with reproducing property

The use of regional covariance descriptors to generate feature data represented by Symmetric Positive Definite (SPD) matrices from images or videos has become increasingly common in machine learning. However, SPD data itself does not constitute a vector space, and dictionary learning involves a large number of linear operations, so dictionary learning cannot be performed directly on SPD data. For this reason, a more common method is to map the SPD data to the Reproducing Kernel Hilbert Space (RKHS). The so-called kernel learning is to find the most suitable RKHS for specific tasks. RKHS can be uniquely generated by a kernel function. Therefore, RKHS learning can also be considered as kernel learning. In this article, there are two main contributions. The first contribution is to propose a framework which based on Kernel Learning and Riemannian Metric (KLRM). Usually the learnable kernel function framework is to learn some parameters in the kernel function. The second contribution is dictionary learning by applying KLRM to SPD data. The SPD data is transformed into the RKHS generated by KLRM, and RKHS after training provides the most suitable working space for dictionary learning. Under the proposed framework, we design a positive definite kernel function, which is defined by the Log-Euclidean metric. This function can be transformed into a corresponding Riemannian kernel. The experimental results provided in this paper is compared with other state-of-the-art algorithms for SPD data dictionary learning and show that the proposed algorithm achieves better results.

Covariance matrices have attracted increasing attention for data representation in many computer vision tasks. The nonsingular covariance matrices are regarded as points on Riemannian manifolds rather than Euclidean space. A common technique for classification on Riemannian manifolds is to embed the covariance matrices into a reproducing kernel Hilbert space (RKHS), and then construct a map from RKHS to Euclidean space, while the explicit map from RKHS to Euclidean space in most kernel-based methods only depends on a linear hypothesis. In this paper, we propose a subspace learning framework to project Riemannian manifolds to Euclidean space, and give the theoretical derivation for it. Specifically, the Euclidean space is isomorphic to the subspace of RKHS. Under the framework, firstly we define an improved Log-Euclidean Gaussian radial basis function kernel for embedding. The first order statistical features of input images are incorporated into the kernel function to increase the discriminative power. After that we seek the optimal projection matrix of the subspace of the RKHS by conducting a graph embedding discriminant analysis. Texture recognition and object categorization experiments with region covariance descriptors demonstrate the considerable effectiveness of the improved Log-Euclidean Gaussian RBK kernel and the proposed method.

The nonparametric multivariate analysis of variance (NPMANOVA) testing procedure has been proven to be a valuable tool for comparing groups. In the present paper, we propose a kernel extension of this technique in order to effectively confront high-dimensionality, a recurrent problem in many fields of science. The new method is called kernel multivariate analysis of variance (KMANOVA). The basic idea is to take advantage of the kernel framework: we propose to project the data from the original data space to a Hilbert space generated by a given kernel function and then perform the NPMANOVA method in the reproducing kernel Hilbert space (RKHS). Dispersion of the embedded points can be measured by the distance induced by the inner product in the RKHS but also by many other distances best suited in high-dimensional settings. For this purpose, we study two promising distances: a Manhattan-type distance and a distance based on an orthogonal projection of the embedded points in the direction of the group centroids. We show that the NPMANOVA method and the KMANOVA method with the induced distance are essentially equivalent. We also show that the KMANOVA method with the other two distances performs considerably better than the NPMANOVA method. We illustrate the advantages of our approach in the context of genetic association studies and demonstrate its usefulness on Alzheimer’s disease data. We also provide a software implementation of the method that is available on GitHub https://github.com/8699vicente/Kmanova.

In the machine learning based on kernel tricks, people often put one variable of a kernel function on the given samples to produce the basic functions of a solution space of learning problem. If the collection of the given samples deviates from the data distribution, the solution space spanned by these basic functions will also deviate from the real solution space of learning problem. In this paper a multikernel-like learning algorithm based on data probability distribution (MKDPD) is proposed, in which the parameters of a kernel function are locally adjusted according to the data probability distribution, and thus produces different kernel functions. These different kernel functions will generate different Reproducing Kernel Hilbert Spaces (RKHS). The direct sum of the subspaces of these RKHS constitutes the solution space of learning problem. Furthermore, based on the proposed MKDPD algorithm, a new algorithm for labeling new coming data is proposed, in which the basic functions are retrained according to the new coming data, while the coefficients of the retrained basic functions remained unchanged to label the new coming data. The experimental results presented in this paper show the effectiveness of the proposed algorithms.

Kernel-based methods have had hefty success in a wide range of fields over the past decade, they are founded on the robust mathematical framework of reproducing kernel Hilbert spaces (RKHS), this space provides an interesting framework for the development of adaptive nonlinear filters. In this paper, we present a comparative study between the kernel method in Hilbert space with a reproducing kernel, and linear adaptive algorithms that is least mean square (LMS), normalized least mean square (NLMS) and recursive least square (RLS) algorithms. Simulation results show excellent performance of the kernel algorithm for identification of single-input single-output (SISO) systems, compared to the linear adaptive algorithms, this by adopting the very fast fading channels called Broadband Radio Access Network (BRAN A and BRAN C).

Wavelet SVM in Reproducing Kernel Hilbert Space for hyperspectral remote sensing image classification

In this note, we prove that an operator between reproducing kernel Hilbert spaces is a multiplication operator if and only if it leaves invariant zero sets. To be more precise, it is shown that an operator T between reproducing kernel Hilbert spaces is a multiplication operator if and only if (Tf)(z)=0 holds for all f and z satisfying f(z)=0. As possible applications, we deduce a general reflexivity result for multiplier algebras, and furthermore prove fully vector-valued generalizations of mulitplier lifting results of Beatrous and Burbea.

Modeling data relation as a hierarchical structure has proven beneficial for many learning scenarios, and the hyperbolic space, with negative curvature, can encode such data hierarchy without distortion. Several recent studies also show that the representation power of the hyperbolic space can be further improved by endowing the kernel methods. Unfortunately, the known kernel methods, developed in hyperbolic space, are limited by the adaptation capacity or distortion issues. This paper addresses the issues through a novel embedding function. To this end, we propose a curvature-aware isometric embedding, which establishes an isometry from the Poincar\'e model to a special reproducing kernel Hilbert space (RKHS). Then we can further define a series of kernels on this RKHS, including several positive definite kernels and an indefinite kernel. Thorough experiments are conducted to demonstrate the superiority of our proposals over existing-known hyperbolic and Euclidean kernels in various learning tasks, e.g., graph learning and zero-shot learning.

We propose a data analysis method that combines the objectives of nonlinear principal component analysis and nonlinear discriminant analysis with the kernel method in a reproducing kernel Hilbert space. This method addresses nonlinear data analysis problems in high-dimensional spaces, specifically the reproducing kernel Hilbert space, through the use of the kernel trick. Our proposed method can be considered as a semi-supervised data analysis approach. We evaluate our proposed method using various kernel functions and datasets, both visually and quantitatively. The evaluation results demonstrate that our proposal outperforms kernel principal component analysis and generalized discriminant analysis in terms of classification performance. This indicates the advantages and originality of our proposed method. Furthermore, we analyze and discuss our findings based on the evaluation results, and highlight potential areas for further research and future work related to our proposal.

In this paper, a contextual Support Vector Machine (SVM) technique based on the principle of Hilbert Space Embedding (HSE) of a local hyperspectral data distribution into an Reproducing Kernel Hilbert Space (RKHS) is proposed to optimally exploit the spectral and local spatial information of the hyperspectral image. The idea of embedding is to map hyperspectral pixels in a local neighborhood into a single point in the RKHS that can uniquely represent those pixels collectively. Previously, the authors have employed an HSE called empirical mean map to build the contextual SVM. In this work, a weighted empirical mean map is utilized to exploit the similarities and variation in the local spatial information. For every pixel, a small set of the neighboring pixels in a hyperspectral image are mapped into an RKHS induced by a certain kernel (Eg. Gaussian RBF kernel) and then, the embedded point of these group of pixels is obtained by calculating the weighted empirical mean of these mapped points. The weights are determined based on the distance between the pixel in consideration and its neighbors. An SVM separating hyperplane is built to maximize the margin between classes formed by weighted empirical means. The proposed technique showed significant improvement over the existing contextual and composite kernels on two hyperspectral image data sets.

This article proposes a novel approach to linear dimension reduction for regression using nonparametric estimation with positive-definite kernels or reproducing kernel Hilbert spaces (RKHSs). The purpose of the dimension reduction is to find such directions in the explanatory variables that explain the response sufficiently: this is called sufficient dimension reduction. The proposed method is based on an estimator for the gradient of the regression function considered for the feature vectors mapped into RKHSs. It is proved that the method is able to estimate the directions that achieve sufficient dimension reduction. In comparison with other existing methods, the proposed one has wide applicability without strong assumptions on the distributions or the type of variables, and needs only eigendecomposition for estimating the projection matrix. The theoretical analysis shows that the estimator is consistent with certain rate under some conditions. The experimental results demonstrate that the proposed method successfully finds effective directions with efficient computation even for high-dimensional explanatory variables.

Automatic target recognition has been widely studied over the years, yet it is still an open problem. The main obstacle consists in extended operating conditions, e.g.., depression angle change, configuration variation, articulation, and occlusion. To deal with them, this paper proposes a new classification strategy. We develop a new representation model via the steerable wavelet frames. The proposed representation model is entirely viewed as an element on Grassmann manifolds. To achieve target classification, we embed Grassmann manifolds into an implicit reproducing Kernel Hilbert space (RKHS), where the kernel sparse learning can be applied. Specifically, the mappings of training sample in RKHS are concatenated to form an overcomplete dictionary. It is then used to encode the counterpart of query as a linear combination of its atoms. By designed Grassmann kernel function, it is capable to obtain the sparse representation, from which the inference can be reached. The novelty of this paper comes from: 1) the development of representation model by the set of directional components of Riesz transform; 2) the quantitative measure of similarity for proposed representation model by Grassmann metric; and 3) the generation of global kernel function by Grassmann kernel. Extensive comparative studies are performed to demonstrate the advantage of proposed strategy.

This paper describes a rigorous framework for reconstructing MR images of the heart, acquired continuously over the cardiac and respiratory cycle. The framework generalizes existing techniques, commonly referred to as retrospective gating, and is based on the properties of reproducing kernel Hilbert spaces. The reconstruction problem is formulated as a moment problem in a multidimensional reproducing kernel Hilbert spaces (a two-dimensional space for cardiac and respiratory resolved imaging). Several reproducing kernel Hilbert spaces were tested and compared, including those corresponding to commonly used interpolation techniques (sinc-based and splines kernels) and a more specific kernel allowed by the framework (based on a first-order Sobolev RKHS). The Sobolev reproducing kernel Hilbert spaces was shown to allow improved reconstructions in both simulated and real data from healthy volunteers, acquired in free breathing.

Active learning based on minimization of the expected path-length of random walks on the learned manifold structure