Multishot person reidentification using joint group sparse representation

Person re-identification is particularly challenging due to significant appearance changes across separate camera views. In order to re-identify people, a representative human signature should effectively handle differences in illumination, pose and camera parameters. While general appearance-based methods are modelled in Euclidean spaces, it has been argued that some applications in image and video analysis are better modelled via non-Euclidean manifold geometry. To this end, recent approaches represent images as covariance matrices, and interpret such matrices as points on Riemannian manifolds. As direct classification on such manifolds can be difficult, in this paper we propose to represent each manifold point as a vector of similarities to class representers, via a recently introduced form of Bregman matrix divergence known as the Stein divergence. This is followed by using a discriminative mapping of similarity vectors for final classification. The use of similarity vectors is in contrast to the traditional approach of embedding manifolds into tangent spaces, which can suffer from representing the manifold structure inaccurately. Comparative evaluations on benchmark ETHZ and iLIDS datasets for the person re-identification task show that the proposed approach obtains better performance than recent techniques such as Histogram Plus Epitome, Partial Least Squares, and Symmetry-Driven Accumulation of Local Features.

Person re-identification is a classical task for any multi-camera surveillance system. Most of the existing researches on re-identification are based on features extracted from RGB images. However, there are many deficiencies in RGB image processing, some of which are requiring a lot of illumination and high computation. In this paper, novel features are proposed for RGB-D person re-identification. First, the complex network approach in texture recognition is modified and its threshold function is changed for using in depth images extracted by RGB-D sensors. Then, two novel measurements named the histogram of the edge weight (HEW) and the histogram of the node strength (HNS) are introduced on complex networks. Our features fit both single-shot and multi-shot person re-identification. In the single-shot case, the HNS is extracted from only one frame while for the multi-shot case it is extracted from both one frame and multi-frames. These proposed measurements are called histogram of the spatial node strength (HSNS) and histogram of the temporal node strength (HTNS) respectively. Subsequently, these measurements are combined with skeleton features using score-level fusion. The method is evaluated using two benchmark databases and the results show that ours outperforms some state-of-the-art methods.

Joint graph regularized dictionary learning and sparse ranking for multi-modal multi-shot person re-identification

Image fusion is the process which aims to integrate the relevant and complementary information from a set of images into a single comprehensive image. Sparse representation (SR) is a powerful technique used in a wide variety of applications like denoising, compression and fusion. Building a compact and informative dictionary is the principal challenge in these applications. Hence, we propose a supervised classification based learning technique for the fusion algorithm. As an initial step, each patch of the training data set is pre-classified based on their gradient dominant direction. Then, a dictionary is learned using K-SVD algorithm. With this universal dictionary, sparse coefficients are estimated using greedy OMP algorithm to represent the given set of source images in the dominant direction. Finally, the Euclidean norm is used as a distance measure to reconstruct the fused image. Experimental results on different types of source images demonstrate the effectiveness of the proposed algorithm with conventional methods in terms of visual and quantitative evaluations.

Multi-Instance Convolutional Neural Network for multi-shot person re-identification

Recently, the person re-identification task becomes increasingly crucial in crowded scenarios, e.g., airports and schools. Many methods with high performance have been proposed to solve this problem. However, the existence of occlusion still challenges the development of person re-identification. In this work, we present the novel Pose - Guided Mixed Attention Network (PGMANet), an end-to-end framework to deal with pedestrian reidentification under occluded situations by fusing posture and second-order information. Specially, we employ two models. The first is Human Part - level Attention Model. We use key point information of a pedestrian to generate a heat map to enhance the pedestrian body part's feature. Simultaneously, we design Second - order Information Attention Model to investigate the correlation among features of different parts. Experimental results show that our method achieves state-of-the-art person re-identification performance on two challenging occlusion datasets Occluded-DukeMTMC and Occluded-Reid.

Person re-identification (re-ID) as a retrieval task often utilizes a re-ranking model to improve performance. Existing re-ranking methods are typically designed for conventional person re-ID with RGB images, while skeleton representation re-ranking for skeleton-based person re-ID still remains to be explored. To fill this gap, we revisit the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance re-ranking model in this letter, and propose a generic re-ranking method that exploits the salient skeleton features to perform <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance encoding for skeleton-based person re-ID re-ranking. In particular, we devise the skeleton sequence pooling to aggregate the most salient features of skeletons within a sequence, and combine both original Euclidean distance and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance to re-rank the skeleton sequence representations for person re-ID. Furthermore, we propose the context-based Rank-1 voting that jointly exploits the initial ranking list and re-ranking list to vote for the top candidate to enhance the Rank-1 matching. Extensive experiments on three public benchmarks demonstrate that our approach can effectively re-rank different state-of-the-art skeleton representations and significantly improve their person re-ID performance.

In this paper, we propose a novel image set representation and classification method by maximizing the margin of image sets. The margin of an image set is defined as the difference of the distance to its nearest image set from different classes and the distance to its nearest image set of the same class. By modeling the image sets by using both their image samples and their affine hull models, and maximizing the margins of the images sets, the image set representation parameter learning problem is formulated as an minimization problem, which is further optimized by an expectation -maximization (EM) strategy with accelerated proximal gradient (APG) optimization in an iterative algorithm. To classify a given test image set, we assign it to the class which could provide the largest margin. Experiments on two applications of video-sequence-based face recognition demonstrate that the proposed method significantly outperforms state-of-the-art image set classification methods in terms of both effectiveness and efficiency.

For face recognition, conventional dictionary learning (DL) methods have disadvantages. In the paper, we propose a novel robust, discriminative and comprehensive DL (RDCDL) model. The proposed model uses sample diversities of the same face image to make the dictionary robust. The model includes class-specific dictionary atoms and disturbance dictionary atoms, which can well represent the data from different classes. Both the dictionary and the representation coefficients of data on the dictionary introduce discriminative information, which improves effectively the discrimination capability of the dictionary. The proposed RDCDL is extensively evaluated on benchmark face image databases, and it shows superior performance to many state-of-the-art sparse representation and dictionary learning methods for face recognition.

Many problems in computer vision and machine learning area can be formulated as image set representation and learning. In real application, image set data often contains various kinds of noises or missing value corruptions, and are also usually sampled from nonlinear manifolds. These make the representation and learning process of image set data more challengeable. This paper proposes a robust manifold regularized low-rank approximation (MLRA) method for image set recovery and representation. MLRA provides an effective low-rank representation for image set whose elements are sampled from nonlinear manifolds. Comparing with original observed image set, MLRA of image set is generally noiseless and more regular, which can obviously encourage the robust learning and recognition process. We evaluate our method on several datasets to demonstrate the benefits of the method.

Person re-identification (ReID) is to identify the same person across non-overlapping camera views. After a decade of development, the methods based on deep networks have achieved high performance on benchmarks and become mainstream. In applications, the features of gallery images extracted by deep learning-based methods are stored to speed up the query process and protect the sensitive information contained in the images. Unfortunately, it is demonstrated that turning the images into features cannot properly protect privacy, as these features could be reversed to the corresponding images, revealing the sensitive information they contain. Therefore, for preventing privacy leakage, recent methods learn their features against some feature reversal methods, and most conventional reversal methods focus on minimizing the difference between a reconstruction and its original image. However, there could be many reasonable reconstruction results from a single feature, and the conventional reversal methods will inevitably generate reconstruction results that lie in a different distribution from one of the original images, which cannot properly assess the private information for learning to protect and thus hamper the privacy-protected feature learning. To mitigate this problem, we enforce the reconstructions to follow the same distribution as the original images by the generative adversarial network (GAN). We operate this GAN-based feature reversal module accompanied by the conventional ReID feature extraction module and form a novel GAN-based feature privacy-protected person ReID model, which is expected to protect feature privacy so as against reversal attack and maintain ReID utility. We demonstrate that optimizing ReID model to accommodate privacy protection faces a double adversarial objective and is thus challenging. As a remedy, we design a novel two-step training and lazy update strategy that alternatively optimizes the feature extraction module and stabilizes the update process of the GAN-based feature reversal module. To evaluate the efficiency of the model in balancing its ReID utility and feature privacy protection, we introduce a novel metric called utility-reversibility ratio (URR). Compared with existing privacy-protected feature extraction models, the proposed method achieves a better balance between privacy protection and person ReID performance. Extensive experiments validate that our model can effectively protect feature privacy at a tiny accuracy cost, and validate the effectiveness of our model with the emerging diffusion model.

Researches on person re-identification have become more prevalent as a core technique in visual surveillance systems. While a large volume of person re-identification (Re-ID) methods adopt experimental settings with a large patch size, relatively small ones are not much considered. Thus, this work focuses on investigating comparison of person Re-ID performance when color and shape features-based are adopted for small patch size images with low resolution. First, the deep decompositional network is used to divide the person into upper and lower body parts. Then, color and shape features are extracted. Finally, using single or combination of features, similarity-based ranked matching scores are computed. The person Re-ID performance is evaluated based on VIPeR dataset. From the experiment results, we found that color-based feature is better than shape-based features, and the combination of color and shape features-based can be meaningful.

A probe based approach is presented for the recognition of targets in a cluttered background using an infrared imager. A probe is a simple mathematical function which operates locally on image grey levels and produces an output that is more directly usable by an algorithm. A directional probe image is calculated by taking the difference in grey levels between pixels a set distance apart in a given direction, centered on the probe image pixel. A parametric statistical image background model which describes the probe images is introduced. The parameters of the probe image model can be readily estimated from the image. Knowledge of these parameters, together with target signatures obtained from Computer Aided Design (CAD) models, allows the likelihood ratio for a given object pose hypothesis versus the background null hypothesis to be written. The generalized likelihood ratio test is used to accept one of the object poses or to choose the null hypothesis. Results of the method applied to a large set of terrain model board images are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

We present a structural matching technique for robust classification based on image sets. In set based classification, a probe set is matched with a number of gallery sets and assigned the label of the most similar set. We represent each image set by a sparse dictionary and compute a similarity matrix by matching all the dictionary atoms of the gallery and probe sets. The similarity matrix comprises the sparse coding coefficients and forms a fully connected directed graph. The nodes of the graph are the dictionary atoms and the edges are the sparse coefficients. The graph is converted to an undirected graph with positive edge weights and spectral clustering is used to cut the graph into two balanced partitions using the normalized cut algorithm. This process is repeated until the graph reduces to critical and non-critical partitions. A critical partition contains atoms with the same gallery label along with one or more probe atoms whereas a noncritical partition either consists of only probe atoms or atoms with multiple gallery labels with no probe atom. Using the critical partitions, we define a novel set based similarity measure and assign the probe set the label of the gallery set with maximum similarity. The proposed algorithm is applied to image set based face recognition using two standard databases. Comparison with existing techniques shows the validity and robustness of our algorithm in the presence of outlier images.

For the instrument limitation and imperfect imaging optics, it is difficult to acquire high spatial resolution hyperspectral imagery. Low spatial resolution will result in a lot of mixed pixels and greatly degrade the detection and recognition performance, affect the related application in civil and military fields. As a powerful statistical image modeling technique, sparse representation can be utilized to analyze the hyperspectral image efficiently. Hyperspectral imagery is intrinsically sparse in spatial and spectral domains, and image super-resolution quality largely depends on whether the prior knowledge is utilized properly. In this article, we propose a novel hyperspectral imagery super-resolution method by utilizing the sparse representation and spectral mixing model. Based on the sparse representation model and hyperspectral image acquisition process model, small patches of hyperspectral observations from different wavelengths can be represented as weighted linear combinations of a small number of atoms in pre-trained dictionary. Then super-resolution is treated as a least squares problem with sparse constraints. To maintain the spectral consistency, we further introduce an adaptive regularization terms into the sparse representation framework by combining the linear spectrum mixing model. Extensive experiments validate that the proposed method achieves much better results.