Kernel Thesis Reproduction and Truncated Toeplitz Operators

In this paper, we specify what functions induce the bounded composition operators on a reproducing kernel Hilbert space (RKHS) associated with an analytic positive definite function defined on Rd. We prove that only affine transforms can do so in a certain large class of RKHS. Our result covers not only the Paley-Wiener space on the real line, studied in previous works, but also much more general RKHSs corresponding to analytic positive definite functions, where existing methods do not work. Our method only relies on intrinsic properties of the RKHSs, and we establish a connection between the behavior of composition operators and asymptotic properties of the greatest zeros of orthogonal polynomials on a weighted L2-space on the real line. We also investigate the compactness of the composition operators and show that any bounded composition operators cannot be compact in our situation.

The use of regional covariance descriptors to generate feature data represented by Symmetric Positive Definite (SPD) matrices from images or videos has become increasingly common in machine learning. However, SPD data itself does not constitute a vector space, and dictionary learning involves a large number of linear operations, so dictionary learning cannot be performed directly on SPD data. For this reason, a more common method is to map the SPD data to the Reproducing Kernel Hilbert Space (RKHS). The so-called kernel learning is to find the most suitable RKHS for specific tasks. RKHS can be uniquely generated by a kernel function. Therefore, RKHS learning can also be considered as kernel learning. In this article, there are two main contributions. The first contribution is to propose a framework which based on Kernel Learning and Riemannian Metric (KLRM). Usually the learnable kernel function framework is to learn some parameters in the kernel function. The second contribution is dictionary learning by applying KLRM to SPD data. The SPD data is transformed into the RKHS generated by KLRM, and RKHS after training provides the most suitable working space for dictionary learning. Under the proposed framework, we design a positive definite kernel function, which is defined by the Log-Euclidean metric. This function can be transformed into a corresponding Riemannian kernel. The experimental results provided in this paper is compared with other state-of-the-art algorithms for SPD data dictionary learning and show that the proposed algorithm achieves better results.

Covariance matrices have attracted increasing attention for data representation in many computer vision tasks. The nonsingular covariance matrices are regarded as points on Riemannian manifolds rather than Euclidean space. A common technique for classification on Riemannian manifolds is to embed the covariance matrices into a reproducing kernel Hilbert space (RKHS), and then construct a map from RKHS to Euclidean space, while the explicit map from RKHS to Euclidean space in most kernel-based methods only depends on a linear hypothesis. In this paper, we propose a subspace learning framework to project Riemannian manifolds to Euclidean space, and give the theoretical derivation for it. Specifically, the Euclidean space is isomorphic to the subspace of RKHS. Under the framework, firstly we define an improved Log-Euclidean Gaussian radial basis function kernel for embedding. The first order statistical features of input images are incorporated into the kernel function to increase the discriminative power. After that we seek the optimal projection matrix of the subspace of the RKHS by conducting a graph embedding discriminant analysis. Texture recognition and object categorization experiments with region covariance descriptors demonstrate the considerable effectiveness of the improved Log-Euclidean Gaussian RBK kernel and the proposed method.

A local–global mixed kernel with reproducing property

This paper describes a rigorous framework for reconstructing MR images of the heart, acquired continuously over the cardiac and respiratory cycle. The framework generalizes existing techniques, commonly referred to as retrospective gating, and is based on the properties of reproducing kernel Hilbert spaces. The reconstruction problem is formulated as a moment problem in a multidimensional reproducing kernel Hilbert spaces (a two-dimensional space for cardiac and respiratory resolved imaging). Several reproducing kernel Hilbert spaces were tested and compared, including those corresponding to commonly used interpolation techniques (sinc-based and splines kernels) and a more specific kernel allowed by the framework (based on a first-order Sobolev RKHS). The Sobolev reproducing kernel Hilbert spaces was shown to allow improved reconstructions in both simulated and real data from healthy volunteers, acquired in free breathing.

Bounded symbols and Reproducing Kernel Thesis for truncated Toeplitz operators

The reproducing kernel Hilbert space (RKHS) methodology has shown to be a suitable tool for the resolution of a wide range of problems in statistical signal processing both in the real and complex domains. It relies on the idea of transforming the original functional data into an infinite series representation by projection onto an specific RKHS, which usually simplifies the statistical treatment without any loss of efficiency. Moreover, the advantages of quaternion algebra over real-valued three and four-dimensional vector algebra in the modelling of multidimensional data have been proven useful in much relatively recent research. This paper accordingly proposes a generic RKHS framework for the statistical analysis of augmented quaternion random vectors, which provide a complete description of their second order characteristics. It will allow us to exploit the full advantages of the RKHS theory in widely linear processing applications, such as signal detection. In particular, we address the detection of a quaternion signal disturbed by additive Gaussian noise and the discrimination between two quaternion Gaussian signals in continuous time.

Adaptive channel equalization is a signal processing technique to mitigate inter-symbol interference in a time dispersive channel. For adaptive equalization, minimum mean square error (MMSE) criterion-based reproducing kernel Hilbert spaces (RKHS) approaches such as the kernel least mean squares (KLMS) algorithm and its variants have been suggested in the literature for nonlinear channels. Another optimality criterion, based on minimum bit/symbol error rate (MBER/MSER), is a better choice for adapting an equalizer as compared to MMSE criterion. A kernel-based minimum symbol error rate (KMSER) equalization algorithm combines minimum symbol error rate (MSER)-based approaches with RKHS techniques. However, most algorithms in RKHS such as KMSER/KLMS require infinite storage requirement and hence cannot be practically implemented. To curtail the infinite memory requirement, and make adaptive algorithm suitable for implementation with finite memory and processing power, we propose quantized KMSER (QKMSER) and fixed-budget quantized KMSER (FBQKMSER)-based equalizers in this paper. In this paper, we derive the dynamical equation for MSE evolution of the QKMSER and FBQKMSER and find their performance to be asymptotically close to the MSE behavior of the KMSER. Also, it is found via simulations that the tracking performance of FBQKMSER is better than all the compared algorithms in this paper which is particularly useful for non-stationary channels.

For a certain scaling of the initialization of stochastic gradient descent (SGD), wide neural networks (NN) have been shown to be well approximated by reproducing kernel Hilbert space (RKHS) methods. Recent empirical work showed that, for some classification tasks, RKHS methods can replace NNs without a large loss in performance. On the other hand, two-layers NNs are known to encode richer smoothness classes than RKHS and we know of special examples for which SGD-trained NN provably outperform RKHS. This is true even in the wide network limit, for a different scaling of the initialization. How can we reconcile the above claims? For which tasks do NNs outperform RKHS? If covariates are nearly isotropic, RKHS methods suffer from the curse of dimensionality, while NNs can overcome it by learning the best low-dimensional representation. Here we show that this curse of dimensionality becomes milder if the covariates display the same low-dimensional structure as the target function, and we precisely characterize this tradeoff. Building on these results, we present the spiked covariates model that can capture in a unified framework both behaviors observed in earlier work. We hypothesize that such a latent low-dimensional structure is present in image classification. We test numerically this hypothesis by showing that specific perturbations of the training distribution degrade the performances of RKHS methods much more significantly than NNs.

In this note, we prove that an operator between reproducing kernel Hilbert spaces is a multiplication operator if and only if it leaves invariant zero sets. To be more precise, it is shown that an operator T between reproducing kernel Hilbert spaces is a multiplication operator if and only if (Tf)(z)=0 holds for all f and z satisfying f(z)=0. As possible applications, we deduce a general reflexivity result for multiplier algebras, and furthermore prove fully vector-valued generalizations of mulitplier lifting results of Beatrous and Burbea.

In this paper, a contextual Support Vector Machine (SVM) technique based on the principle of Hilbert Space Embedding (HSE) of a local hyperspectral data distribution into an Reproducing Kernel Hilbert Space (RKHS) is proposed to optimally exploit the spectral and local spatial information of the hyperspectral image. The idea of embedding is to map hyperspectral pixels in a local neighborhood into a single point in the RKHS that can uniquely represent those pixels collectively. Previously, the authors have employed an HSE called empirical mean map to build the contextual SVM. In this work, a weighted empirical mean map is utilized to exploit the similarities and variation in the local spatial information. For every pixel, a small set of the neighboring pixels in a hyperspectral image are mapped into an RKHS induced by a certain kernel (Eg. Gaussian RBF kernel) and then, the embedded point of these group of pixels is obtained by calculating the weighted empirical mean of these mapped points. The weights are determined based on the distance between the pixel in consideration and its neighbors. An SVM separating hyperplane is built to maximize the margin between classes formed by weighted empirical means. The proposed technique showed significant improvement over the existing contextual and composite kernels on two hyperspectral image data sets.

Decentralized learning over a network with Nyström approximation using SGD

No fuss metric learning, a Hilbert space scenario

Given a time series arising as the observations of some dynamical system, it is possible to reconstruct the qualitative behaviour of the dynamical system. Given such a reconstruction, this provides a general approach for making accurate short-term predictions. The time series reconstruction problem can be viewed as a function approximation problem which can be addressed by reproducing kernel Hilbert spaces (RKHS). Such an approach is described and related to Bayesian estimation with Gaussian priors. It is also shown that an explicit formulation of the model as an input-dependent autoregressive model is possible. This linearization is, however, unlike other approaches, exact and the autoregressive coefficients can be calculated explicitly at each time step. A particular example of RKHS approximation is applied to two benchmark problems, one synthetic and one real. The results demonstrate various aspects related to the optimization of the models and also show that the RKHS method is competitive against the best alternatives.

Similarity measurement of two probability distributions is important in many applications of statistics. Embedding such distributions into a reproducing kernel Hilbert space (RKHS) has many favorable properties. The choice of the reproducing kernel is crucial in the approach. We study this question by considering the similarity of two distributions of the same class. In particular, we investigate when the RKHS embedding is “admissible” in the sense that the distance between the embeddings should become smaller when the expectations are getting closer or when the variance is increasing to infinity. We give conditions on the widely-used translation-invariant reproducing kernels to be admissible. We also extend the study to multivariate non-symmetric Gaussian distributions.