Kernel-Based Subspace Learning on Riemannian Manifolds for Visual Recognition

Active learning based on minimization of the expected path-length of random walks on the learned manifold structure

In this paper, we develop an approach to exploiting kernel methods with manifold-valued data. In many computer vision problems, the data can be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, usual Euclidean computer vision and machine learning algorithms yield inferior results on such data. In this paper, we define Gaussian radial basis function (RBF)-based positive definite kernels on manifolds that permit us to embed a given manifold with a corresponding metric in a high dimensional reproducing kernel Hilbert space. These kernels make it possible to utilize algorithms developed for linear spaces on nonlinear manifold-valued data. Since the Gaussian RBF defined with any given metric is not always positive definite, we present a unified framework for analyzing the positive definiteness of the Gaussian RBF on a generic metric space. We then use the proposed framework to identify positive definite kernels on two specific manifolds commonly encountered in computer vision: the Riemannian manifold of symmetric positive definite matrices and the Grassmann manifold, i.e., the Riemannian manifold of linear subspaces of a Euclidean space. We show that many popular algorithms designed for Euclidean spaces, such as support vector machines, discriminant analysis and principal component analysis can be generalized to Riemannian manifolds with the help of such positive definite Gaussian kernels.

Dimensionality reduction has emerged as one of the prominent fields of research since it provides a solution to a wide class of problems, such as compression, classification, regression, feature analysis, and visual recognition. Generally, subspace learning algorithms find a low-dimensional subspace from a given high-dimensional data, wherein samples from different classes can be well separated. In the past decades, several types of dimensionality reduction and subspace selection algorithms have been widely used for visual data analysis. Conventional algorithms such as principal component analysis (PCA) and linear discriminant analysis (LDA), perform well under the assumption that training and test data follow similar distributions. However, the main drawback still remains in the sense that the distribution of the training data and that of the test data are mismatched when samples are drawn from different but related sources. They also fail to incorporate temporal statistical variations in data distribution when applied to time-series data(videos). Consequently they result in unsatisfactory recognition performance when they deal with real world visual analysis problems. To overcome the above issues, in this research, our aim is to consider invariance and stationarity in subspace analysis pertaining to many computer vision applications such as video retrieval, human behaviour analysis, event analysis and activity recognition. More specifically, we will address the challenging and fundamental tasks of 1. Video Classification (Scene, dynamic texture and action recognition) 2. Visual Domain Adaptation. To address the first task, we propose a subspace learning approach that focuses on extracting the stationary parts of all videos in the same class. The notion of stationarity is intuitively well-adapted to model the temporal nature of the video signal and lets us make use of many image features. Instead of modeling temporal information in the features, our method explicitly accounts for it when learning the latent space. As a consequence, the resulting video representation is particularly well-suited for classification purpose. Our experimental evaluation shows that our approach outperforms baselines from the different groups of methods in several video classification tasks such as dynamic texture recognition, scene classification and action recognition.The second task addressed by this work is domain adaptation for visual recognition. In our proposed approach, we follow the intuitive idea of learning the domain invariant subspace by matching the distributions of the transformed training and test data using Maximum Mean Discrepancy (MMD). This, we believe, makes better use of the expressive power of the kernel in MMD compared to the other approaches like Transfer Component Analysis (TCA). Although motivated by MMD, in TCA, the distance between the sample means is measured in a lower-dimensional space rather than in Reproducing Kernel Hilbert Space (RKHS), which somewhat contradicts the intuition behind the use of kernels. Furthermore, we extend the framework for the semi-supervised scenario. Experiments on benchmark domain adaptation dataset for visual recognition, show that in comparison to well-known methods, the proposed approach obtains a significant improvement in classification accuracy. To address the second task, we propose another domain adaptation method that exploits the Riemannian structure of the statistical manifold when learning the invariant subspaces or samples. To this end, we introduce the use of the Hellinger distance which is related to the geodesic distance on the space of probability distributions. While the Hellinger distance has been employed for dimensionality reduction, to the best of our knowledge, our approach is the first attempt at exploiting the Riemannian geometry of the statistical manifold for domain adaptation. We showed that our sample-selection and subspace-based approaches in conjunction with Hellinger distance for distribution matching consistently outperform the similar approaches on the tasks of visual domain adaptation and WiFi localization.

A local–global mixed kernel with reproducing property

Multi-kernel sparse subspace clustering on the Riemannian manifold of symmetric positive definite matrices

Various kernel-based methods have been developed with great success in many fields, but very little research has been published that is concerned with a multiple attribute kernel in reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel elastic kernel called a multiple attribute convolution kernel with reproducing property (MACKRP) and present improved classification results over conventional approaches in the RKHS rather than the more commonly used Hilbert space. The MACKRP consists of two major steps. First, we find the basic solution of a generalized differential operator by the delta function, and then we design a convolution function using this solution. This convolution function is proven to be a specific reproducing kernel called a convolution reproducing kernel (CRK) in H3-space. Second, we prove that the CRK satisfies the condition of Mercer kernel. And the CRK is composed of three attributes (L1-norm, L2-norm and Laplace kernel), and each attribute can capture a different feature, with all attributes generating a novel kernel which we call an MACKRP. The experimental results demonstrate that the MACKRP possesses approximation and regularization performance and that classification results are consistently comparable or superior to a number of other state-of-the-art kernel functions.

No fuss metric learning, a Hilbert space scenario

we propose a regression method using partial least square (PLS) technique and kernel method, which we refer to as kernel partial least square regression II to distinguish the conventional kernel PLS regression. The motivation of the conventional kernel PLS regression is to establish a coordinator system where the independent and dependent variables have a stronger correlation, and which models a regression model using the coordinator system. However, it projects independent variables into a reproducing kernel Hilbert space (RKHS) alone. The proposal extends the basic framework of conventional kernel PLS regression. The proposed method does not only project independent variables into the RKHS, but also dependent variables, and establishes a coordinator system where the independent and dependent variables have a stronger correlation. We use two function regression cases to evaluate the proposed method compared with the conventional kernel PLS regression. The regression performance of the proposed method has almost the same regression accuracy arising from the evaluation result, and this depends on the regression tasks. We explain the correlation calculations of our proposed method, conventional kernel PLS regression, and PLS regression. The meaning of correlation depends on the application in question. We also analyse and discuss the algorithm implementation, correlation meaning, and other issues for further development of the proposal.

Retrieving videos of a particular person with face image as query via hashing technique has many important applications. While face images are typically represented as vectors in Euclidean space, characterizing face videos with some robust set modeling techniques (e.g. covariance matrices as exploited in this study, which reside on Riemannian manifold), has recently shown appealing advantages. This hence results in a thorny heterogeneous spaces matching problem. Moreover, hashing with handcrafted features as done in many existing works is clearly inadequate to achieve desirable performance for this task. To address such problems, we present an end-toend Deep Heterogeneous Hashing (DHH) method that integrates three stages including image feature learning, video modeling, and heterogeneous hashing in a single framework, to learn unified binary codes for both face images and videos. To tackle the key challenge of hashing on manifold, a well-studied Riemannian kernel mapping is employed to project data (i.e. covariance matrices) into Euclidean space and thus enables to embed the two heterogeneous representations into a common Hamming space, where both intra-space discriminability and inter-space compatibility are considered. To perform network optimization, the gradient of the kernel mapping is innovatively derived via structured matrix backpropagation in a theoretically principled way. Experiments on three challenging datasets show that our method achieves quite competitive performance compared with existing hashing methods.

The underlying function in reproducing kernel Hilbert space (RKHS) may be degraded by outliers or deviations, resulting in a symmetry ill-posed problem. This paper proposes a nonconvex minimization model with ℓ0-quasi norm based on RKHS to depict this degraded problem. The underlying function in RKHS can be represented by the linear combination of reproducing kernels and their coefficients. Thus, we turn to estimate the related coefficients in the nonconvex minimization problem. An efficient algorithm is designed to solve the given nonconvex problem by the mathematical program with equilibrium constraints (MPEC) and proximal-based strategy. We theoretically prove that the sequences generated by the designed algorithm converge to the nonconvex problem’s local optimal solutions. Numerical experiment also demonstrates the effectiveness of the proposed method.

Several descriptors have been proposed in the past for 3D shape analysis, yet none of them achieves best performance on all shape classes. In this paper we propose a novel method for 3D shape analysis using the covariance matrices of the descriptors rather than the descriptors themselves. Covariance matrices enable efficient fusion of different types of features and modalities. They capture, using the same representation, not only the geometric and the spatial properties of a shape region but also the correlation of these properties within the region. Covariance matrices, however, lie on the manifold of Symmetric Positive Definite (SPD) tensors, a special type of Riemannian manifolds, which makes comparison and clustering of such matrices challenging. In this paper we study covariance matrices in their native space and make use of geodesic distances on the manifold as a dissimilarity measure. We demonstrate the performance of this metric on 3D face matching and recognition tasks. We then generalize the Bag of Features paradigm, originally designed in Euclidean spaces, to the Riemannian manifold of SPD matrices. We propose a new clustering procedure that takes into account the geometry of the Riemannian manifold. We evaluate the performance of the proposed Bag of Covariance Matrices framework on 3D shape matching and retrieval applications and demonstrate its superiority compared to descriptor-based techniques.

Random objects are complex non-Euclidean data taking values in general metric spaces, possibly devoid of any underlying vector space structure. Such data are becoming increasingly abundant with the rapid advancement in technology. Examples include probability distributions, positive semidefinite matrices and data on Riemannian manifolds. However, except for regression for object-valued response with Euclidean predictors and distribution-on-distribution regression, there has been limited development of a general framework for object-valued response with object-valued predictors in the literature. To fill this gap, we introduce the notion of a weak conditional Fréchet mean based on Carleman operators and then propose a global nonlinear Fréchet regression model through the reproducing kernel Hilbert space (RKHS) embedding. Furthermore, we establish the relationships between the conditional Fréchet mean and the weak conditional Fréchet mean for both Euclidean and object-valued data. We also show that the state-of-the-art global Fréchet regression developed by Petersen and Müller (Ann. Statist. 47 (2019) 691-719) emerges as a special case of our method by choosing a linear kernel. We require that the metric space for the predictor admits a reproducing kernel, while the intrinsic geometry of the metric space for the response is utilized to study the asymptotic properties of the proposed estimates. Numerical studies, including extensive simulations and a real application, are conducted to investigate the finite-sample performance.

The decoding of motor imagery electroencephalography (MI-EEG) is constrained by core issues including low signal-to-noise ratio (SNR) and cross-session as well as cross-subject domain shift, which seriously impedes the practical deployment of brain-computer interfaces (BCIs). To address these challenges, this paper proposes a novel end-to-end MI-EEG decoding method named BARN-DA. Two innovative modules, Band-Aware Channel Attention (BACA) and Multi-Scale Kernel Perception (MSKP), are designed: one enhances discriminative channel features by modeling channel information fused with frequency band feature representation, and the other captures complex data correlations via multi-scale parallel convolutions to improve the discriminability of the network's feature extraction. Subsequently, the features are mapped onto the Riemannian manifold. For the source and target domain features residing on this manifold, a Riemannian Maximum Mean Discrepancy (R-MMD) loss is designed based on the log-Euclidean metric. This approach enables the effective embedding of Symmetric Positive Definite (SPD) matrices into the Reproducing Kernel Hilbert Space (RKHS), thereby reducing cross-domain discrepancies. Experimental results on four public datasets demonstrate that the BARN-DA method achieves average cross-session classification accuracies of 84.65% ± 8.97% (BCIC IV 2a), 89.19% ± 7.69% (BCIC IV 2b), and 61.76% ± 12.68% (SHU), as well as average cross-subject classification accuracies of 65.49% ± 11.64% (BCIC IV 2a), 78.78% ± 8.44% (BCIC IV 2b), and 78.14% ± 14.41% (BCIC III 4a). Compared with state-of-the-art methods, BARN-DA obtains higher classification accuracy and stronger cross-session and cross-subject generalization ability. These results confirm that BARN-DA effectively alleviates low SNR and domain shift problems in MI-EEG decoding, providing an efficient technical solution for practical BCI systems.

A regression-based framework for interpretable multi-way data imputation, termed Kernel Regression via Tensor Trains with Hadamard overparametrization (KReTTaH), is introduced. KReTTaH adopts a nonparametric formulation by casting imputation as regression via reproducing kernel Hilbert spaces. Parameter efficiency is achieved through tensors of fixed tensor-train (TT) rank, which reside on low-dimensional Riemannian manifolds, and is further enhanced via Hadamard overparametrization, which promotes sparsity within the TT parameter space. Learning is accomplished by solving a smooth inverse problem posed on the Riemannian manifold of fixed TT-rank tensors. As a representative application, the estimation of dynamic graph flows is considered. In this setting, KReTTaH exhibits flexibility by seamlessly incorporating graph-based (topological) priors via its inverse problem formulation. Numerical tests on real-world graph datasets demonstrate that KReTTaH consistently outperforms state-of-the-art alternatives-including a nonparametric tensor-and a neural-network-based methods-for imputing missing, time-varying edge flows.

In this paper, classification via sparse representation of monogenic signal on Grassmann manifolds is presented for target recognition in SAR image. To capture the broad spectral information with maximal spatial localization of SAR image, a recently proposed vector-valued analytic signal, namely monogenic signal is exploited. Different from the conventional methods, where a single feature descriptor is generated using the monogenic signal in an Euclidean space, the multiple components of monogenic signal at various scale spaces are viewed as points on a special type of Riemannian manifolds, Grassmann manifolds. The similarity between a pair of patterns (points) is measured by Grassmann distance metric. To exploit the nonlinear geometry structure further, we embed the sets of monogenic components into an implicit reproducing kernel Hilbert space (RKHS), where the kernel-based sparse signal modeling can be learnt to reach the inference. Specifically, the sets of monogenic components resulting from the training samples are concatenated first to build a redundant dictionary. Then, the counterpart of the query is efficiently approximated by superposition of atoms of the dictionary. Notably, the representation coefficients of superposition are very parsimonious. The inference is drawn by evaluating which class of training patterns could recover the query as accurately as possible. The novelty of this paper comes from 1) the development of Grassmann manifolds formed by the multiresolution monogenic signal; 2) the definition of similarity between the sets of monogenic components on Grassmann manifolds for target recognition; 3) the generalization of sparse signal modeling on Grassmann manifold; and 4) multiple comparative experiments for performance assessment.