Neighborhood preserving embedding on Grassmann manifold for image-set analysis

Discrete aggregation hashing for image set classification

Sparse and collaborative representation based kernel pairwise linear regression for image set classification

In this paper, we propose a novel image set representation and classification method by maximizing the margin of image sets. The margin of an image set is defined as the difference of the distance to its nearest image set from different classes and the distance to its nearest image set of the same class. By modeling the image sets by using both their image samples and their affine hull models, and maximizing the margins of the images sets, the image set representation parameter learning problem is formulated as an minimization problem, which is further optimized by an expectation -maximization (EM) strategy with accelerated proximal gradient (APG) optimization in an iterative algorithm. To classify a given test image set, we assign it to the class which could provide the largest margin. Experiments on two applications of video-sequence-based face recognition demonstrate that the proposed method significantly outperforms state-of-the-art image set classification methods in terms of both effectiveness and efficiency.

Joint dimensionality reduction and metric learning for image set classification

A semantic RBM-based model for image set classification

Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

We present an image set classification algorithm based on unsupervised clustering of labeled training and unlabeled test data where labels are only used in the stopping criterion. The probability distribution of each class over the set of clusters is used to define a true set based similarity measure. To this end, we propose an iterative sparse spectral clustering algorithm. In each iteration, a proximity matrix is efficiently recomputed to better represent the local subspace structure. Initial clusters capture the global data structure and finer clusters at the later stages capture the subtle class differences not visible at the global scale. Image sets are compactly represented with multiple Grassmannian manifolds which are subsequently embedded in Euclidean space with the proposed spectral clustering algorithm. We also propose an efficient eigenvector solver which not only reduces the computational cost of spectral clustering by many folds but also improves the clustering quality and final classification results. Experiments on five standard datasets and comparison with seven existing techniques show the efficacy of our algorithm.

Gait recognition has become popular due to the rising demand for nonintrusive biometrics. At its nascent stage of development, gait recognition faces a number of challenges. The performance of a gait recognition system is sensitive towards factors like viewing angle, clothing, shoe type, load carriage and speed changes. In this paper, the problems of gait are formulated on the Grassmann manifold. It is not difficult to obtain multiple snapshots of a walking subjects with the wide availability of camera networks. These sets of images can be modelled as low-dimensional subspaces, which can be realized naturally as points on the Grassmann manifold. Modelling image sets as low-dimensional subspaces provides not only possible clue of one’s gait, but also the common patterns of variation in the set. We present a method called Localized Grassmann Mean Representatives with Partial Least Squares Regression (LoGPLS) to infer a low-dimensional Euclidean approximation of the manifold. The notion of local mean representatives is introduced to construct multiple tangent spaces to better approximate the topological structure of the manifold. As the properties of the tangent spaces allows the Grassmann points to be evaluated in the vector space, partial least squares is applied to allow a more accurate classification of the points in a reduced space. Experiments have been conducted on four different publicly available gait databases. Empirical evidences demonstrate the effectiveness of the proposed approach in solving the various covariates in gait recognition.

Image set–based classification has attracted substantial research interest because of its broad applications. Recently, lots of methods based on feature learning or dictionary learning have been developed to solve this problem, and some of them have made gratifying achievements. However, most of them transform the image set into a 2D matrix or use 2D convolutional neural networks (CNNs) for feature learning, so the spatial and temporal information is missing. At the same time, these methods extract features from original images in which there may exist huge intra-class diversity. To explore a possible solution to these issues, we propose a simultaneous image reconstruction with deep learning and feature learning with 3D-CNNs (SIRFL) for image set classification. The proposed SIRFL approach consists of a deep image reconstruction network and a 3D-CNN-based feature learning network. The deep image reconstruction network is used to reduce the diversity of images from the same set, and the feature learning network can effectively retain spatial and temporal information by using 3D-CNNs. Extensive experimental results on five widely used datasets show that our SIRFL approach is a strong competitor for the state-of-the-art image set classification methods.

Modeling videos and image sets by linear subspaces has achieved great success in various visual recognition tasks. However, subspaces constructed from visual data are always notoriously embedded in a high-dimensional ambient space, which limits the applicability of existing techniques. This letter explores the possibility of proposing a geometry-aware framework for constructing lower-dimensional subspaces with maximum discriminative power from high-dimensional subspaces in the supervised scenario. In particular, we make use of Riemannian geometry and optimization techniques on matrix manifolds to learn an orthogonal projection, which shows that the learning process can be formulated as an unconstrained optimization problem on a Grassmann manifold. With this natural geometry, any metric on the Grassmann manifold can theoretically be used in our model. Experimental evaluations on several data sets show that our approach results in significantly higher accuracy than other state-of-the-art algorithms.

Traditional image clustering algorithms deal with single-task clustering (STC) problem on a single domain. However, with the increasing number of related images on the Web, it is challenging for STCs to perform related image clustering tasks independently without considering the between-task relationship, which mainly consists of similar visual features and image patterns among tasks. Therefore, it is intuitive to resort to multi-task clustering (MTC) algorithms. However, most existing MTCs learn a shared feature subspace, which may lead to negative transfer when facing the image clustering tasks that are not strongly related. In this paper, we propose a novel multi-task image clustering algorithm, which performs multiple image clustering tasks simultaneously and propagates the task correlation to improve clustering performance. Specifically, we first extend the information bottleneck method to cluster tasks independently. The related and unrelated images between the pairwise clusters of different tasks are then discovered. Meanwhile, two corresponding types of correlations are propagated among the tasks, where only the positive correlation benefits the clustering of each task. A sequential and collaborative method is further designed to ensure an optimal solution. Moreover, we perform a theoretical analysis of the properties on correlation propagation and the convergence of our algorithm. The experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art clustering methods.

Cooperative linear regression model for image set classification

In this paper, we propose a method to map data from a Grassmann manifold to a vector space while maximizing discrimination capability for classification. Subspaces are a practical and robust representation for image set recognition. However, as they exist on a Grassmann manifold, machine learning tools constructed on Euclidean geometry cannot be promptly utilized. Recently, methods to construct end-to-end learnable models for subspaces are starting to be explored, but they require multiple matrix decompositions and can be hard to compute and extend. Therefore we introduce a layer to map Grassmann manifold-valued data to vector space, in such a way that it can be seamlessly used as a layer along with other powerful tools defined on Euclidean space. The key idea of our method is to formulate the manifold logarithmic map (log) as a learnable model, where we seek to learn a tangency point that minimizes a loss function with respect to the data. The log effectively transforms a manifold point into a tangent vector. This log model can be learned with Riemannian stochastic gradient descent on the target manifold. We demonstrate the effectiveness of our proposed method on the applications of hand shape recognition, face identification and facial emotion recognition.

Data clustering is a fundamental topic in machine learning and data mining areas. In recent years, researchers have proposed a series of effective methods based on Low Rank Representation (LRR) which could explore low-dimension subspace structure embedded in original data effectively. The traditional LRR methods are usually designed for vectorial data from linear spaces with Euclidean distance. However, high-dimension data (such as video clip or imageset) are always considered as non-linear manifold data such as Grassmann manifold with non-linear metric. In addition, traditional LRR clustering method always adopt single nuclear norm as low rank constraint which would lead to suboptimal solution and decrease the clustering accuracy. In this paper, we proposed a new low rank method on Grassmann manifold for video or imageset data clustering task. In the proposed method, video or imageset data are formulated as sample data on Grassmann manifold first. And then a double low rank constraint is proposed by combining the nuclear norm and bilinear representation to better construct the representation matrix. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Constrained mutual convex cone method for image set based recognition