Recurrent matching networks of spatial alignment learning for person re-identification

Person re-identification has become a hot research topic due to its importance in surveillance and forensics applications. The purpose of person re-identification is to find the same person from disjoint camera views at different time. Most of the existing methods try to identify the person by measuring the similarity of two still images from different camera views, which only uses intraimage features such as color, shape and texture. In this paper, we propose a person re-identification architecture which analyzes a sequence pair while not an image pair, so not only intra-image features but also the gait feature is also considered. In contrast to existing works that use handcrafted features, our method automatically learns spatio-temporal features optimal for the person re-identification task with a deep convolutional network. To learn a discriminant metric, we use a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA).The process of XQDA is also called transfer learning. Experiments show that our method significantly outperforms the state of the art on both a large dataset (CUHK03) and a medium-sized dataset (CUHK01). We also get a better performance on a small dataset (VIPeR) with a pre-trained network without fine-tuning.

The similarity metric between videos is integral to several key tasks, including video retrieval, classification and recommendation. Since there is no standard criterion for the similarity measurement between videos except measuring manually, it is difficult to collect large training dataset for distance metric learning algorithms. Moreover, the existing distance metric learning (DML) methods for multimedia data suffer from two critical limitations: (1) they typically attempt to learn a distance function on the single label setting, in which each item is only labeled with single label; (2) they are often designed for learning distance metrics on low-level features, which ignore the semantic similarity of the multimedia data. To address these problems, in this paper, we propose a novel framework of Intermediate Semantics based Distance Learning (ISDL) for video clips, which aims to integrate semantics of multiple modals optimally for distance metric learning. In particular, the proposed framework: (1) generates the training pairs automatically; (2) defines multi-modal concepts for similarity measure among videos; (3) learns the distance metric for video clips based on the intermediate semantics. We conduct an extensive set of experiments to evaluate the performance of the proposed algorithms, and the results validate the effectiveness of our proposed approach.

Multi-manifold metric learning for face recognition based on image sets

In person re-identification, extracting image features is an important step when retrieving pedestrian images. Most of the current methods only extract global features or local features of pedestrian images. Some inconspicuous details are easily ignored when learning image features, which is not efficient or robust to for scenarios with large differences. In this paper, we propose a Multi-level Feature Fusion model that combines both global features and local features of images through deep learning networks to generate more discriminative pedestrian descriptors. Specifically, we extract local features from different depths of network by the Part-based Multi-level Net to fuse low-to-high level local features of pedestrian images. Global-Local Branches are used to extract the local features and global features at the highest level. The experiments have proved that our deep learning model based on multi-level feature fusion works well in person re-identification. The overall results outperform the state of the art with considerable margins on three widely-used datasets. For instance, we achieve 96% Rank-1 accuracy on the Market-1501 dataset and 76.1% mAP on the DukeMTMC-reID dataset, outperforming the existing works by a large margin (more than 6%).

Person re-identification addresses the problem of matching pedestrian images across disjoint camera views. Design of feature descriptor and distance metric learning are the two fundamental tasks in person re-identification. In this paper, we propose a metric learning framework for person re-identification, where the discriminative metric space is learned using Kernel Fisher Discriminant Analysis (KFDA), to simultaneously maximize the inter-class variance as well as minimize the intra-class variance. We derive a Mahalanobis metric induced by KFDA and argue that KFDA is efficient to be applied for metric learning in person re-identification. We also show how the efficiency of KFDA in metric learning can be further enhanced for person re-identification by using two simple yet efficient multiple kernel learning methods. We conduct extensive experiments on three benchmark datasets for person re-identification and demonstrate that the proposed approaches have competitive performance with state-of-the-art methods.

Nowadays, person re-identification (re-id) receives much attraction and it is for matching person images across disjoint camera views. Although many methods are developed, none of the state-of-the-art models (especially for the deep models) can be deployed by a camera's chip due to limited memory and weak computation. To address this problem, we propose a framework called Multi-Cue Tiny Net, which is a combination of tiny convolutional neural networks (CNNs) with small model size. And we call the person re-identification based on limited memory and weak computation the Light Person Re-Identification. Three different tiny nets are used to learn the complementary features and four cues of images are used for learning features with different information. All the features will be concatenated to form a fusion feature. Besides, to reduce the dimension of the features, and keep the small model size and high performance accuracy, we pre-trained the tiny nets and then retrained them after adding a fully connected layer to the global average pooling layer. Experimental results on challenging person re-identification dataset show that our approach yields promising accuracy with light memory and computation.

Person re-identification has become a very popular research topic in the computer vision community owing to its numerous applications and growing importance in visual surveillance. Person re-identification remains challenging due to occlusion, illumination and significant intra-class variations across different cameras. In this paper, we propose a multi-task network base on an improved Res2Net model that simultaneously computes the identification loss and verification loss of two pedestrian images. Given a pair of pedestrian images, the system predicts the identities of the two input images and whether they belong to the same identity. In order to obtain deeper feature information of pedestrians, we propose to use the latest Res2Net model for feature extraction of each input image. Experiments on several large-scale person re-identification benchmark datasets demonstrate the accuracy of our approach. For example, rank-1 accuracies are 83.18% (+1.38) and 93.14% (+0.84) for the DukeMTMC and Market-1501 datasets, respectively. The proposed method shows encouraging improvements compared with state-of-the-art methods.

Metric learning aims to learn a good distance metric that can capture the relationships among instances, and its importance has long been recognized in many fields. In the traditional settings of metric learning, an implicit assumption is that the associated labels of the instances are deterministic. However, in many real-world applications, the associated labels come naturally with probabilities instead of deterministic values. Thus, the existing metric learning methods cannot work well in these applications. To tackle this challenge, in this paper, we study how to effectively learn the distance metric from datasets that contain probabilistic information, and then propose two novel metric learning mechanisms for two types of probabilistic labels, i.e., the instance-wise probabilistic label and the group-wise probabilistic label. Compared with the existing metric learning methods, our proposed mechanisms are capable of learning distance metrics directly from the probabilistic labels with high accuracy. We also theoretically analyze the two proposed mechanisms and provide theoretical bounds on the sample complexity for both of them. Additionally, extensive experiments based on real-world datasets are conducted to verify the desirable properties of the proposed mechanisms.

Person re-identification aims to match the same pedestrians across different camera views and has been applied to many important applications such as intelligent video surveillance. Due to the spatiotemporal uncertainty and visual ambiguity of pedestrian image pairs, person re-identification remains a difficult and challenging problem. The huge success of deep learning has focused attention on the use of deep features for person re-identification. However, for person re-identification, most deep learning methods minimize cross-entropy or triplet-based losses, thereby neglecting the fact that the similarities and differences between image pairs can be considered simultaneously to increase discrimination. In this paper, we propose a novel deep learning method called deep similarity feature learning DSFL to extract more effective deep features for image pairs. Extensive experiments on two representative person re-identification datasets CUHK-03 and GRID demonstrate the effectiveness and robustness of DSFL.

A review on video person re-identification based on deep learning

Person re-identification aims to match pedestrian images across multiple surveillance camera views. It is still a challenging task due to the partial occlusion of pedestrian images, variations in the illumination of surveillance cameras, and similar appearances of pedestrians and so on. In order to improve the representation ability of pedestrian features extracted from the convolutional neural networks, in this paper, we proposed an improved split-attention architecture for person re-identification. Specifically, we first divide the feature map into two sub-groups and then split the features in each subgroup into three more fine-grained sub-feature maps. Moreover, in order to minimize the inter-class similarity and maximize the intra-class similarity, we use circle loss and identification loss to optimize our network together. Circle loss makes the similarity scores learn at different paces, which benefits deep feature learning. The circle loss not only makes the model have higher optimization flexibility but also makes the convergence target of the model more definite. Unlike many methods that use complex convolutional neural networks to represent pedestrian feature maps in a layer-wise manner, our proposed method improves the representation ability of pedestrian features at a more fine-grained level. We evaluated the performance of our proposed network on two large-scale person re-identification benchmark datasets Market-1501 and DukeMTMC-reID. Experimental results show that the proposed split-attention network outperforms the state-of-the-art methods on both datasets with only using pedestrian global features.

Despite being often considered less challenging than image-based person re-identification (re-id), video-based person re-id is still appealing as it mimics a more realistic scenario owing to the availability of pedestrian sequences from surveillance cameras. In order to exploit the temporal information provided, a number of feature extraction methods have been proposed. Although the features could be equally learned at a significantly higher computational cost, the scarce nature of labelled re-id datasets encourages the development of robust hand-crafted feature representations as an efficient alternative, especially when novel distance metrics or multi-shot ranking algorithms are to be validated. This paper presents a novel hand-crafted feature representation for video-based person re-id based on a 3-dimensional hierarchical Gaussian descriptor. Compared to similar approaches, the proposed descriptor (i) does not require any walking cycle extraction, hence avoiding the complexity of this task, (ii) can be easily fed into off-shelf learned distance metrics, (iii) and consistently achieves superior performance regardless of the matching method adopted. The performance of the proposed method was validated on PRID2011 and iLIDS-VID datasets outperforming similar methods on both benchmarks.

As supervised person re-identification (Re-Id) requires massive labeled pedestrian data and it is very difficult to collect sufficient labeled data in reality, unsupervised Re-Id approaches attract much more attention than the former. Existing unsupervised person Re-Id models learn global features of pedestrian from whole images or several constant patches. These models ignore the difference of each region in the whole pedestrian images for feature representation, such as occluded and pose invariant regions, and thus reduce the robustness of models for cross-view feature learning. To solve these issues, we propose an Unsupervised Region Attention Network (URAN) that can learn the cross-view region attention features from the cropped pedestrian images, fixed by region importance weights on images. The proposed URAN designs a Pedestrian Region Biased Enhance (PRBE) loss to produce high attention weights for most important regions in pedestrian images. Furthermore, the URAN employs a first neighbor relation grouping algorithm and a First Neighbor Relation Constraint (FNRC) loss to provide the training direction of the unsupervised region attention network, such that the region attention features are discriminant enough for unsupervised person Re-Id task. In experiments, we consider two popular datasets, Market1501 and DukeMTMC-reID, as evaluation of PRBE and FNRC loss, and their balance parameter to demonstrate the effectiveness and efficiency of the proposed URAN, and the experimental results show that the URAN provides better performance than the-state-of-the-arts (higher than existing methods at least 1.1%).

Lumbar disc degeneration (LDD) is a commonly age-associated condition related to low back pain, while its consequences are responsible for over 90% of spine surgical procedures. In clinical practice, grading of LDD by inspecting MRI is a necessary step to make a suitable treatment plan. This step purely relies on physicians manual inspection so that it brings the unbearable tediousness and inefficiency. An automated method for grading of LDD is highly desirable. However, the technical implementation faces a big challenge from class ambiguity, which is typical in medical image classification problems with a large number of classes. This typical challenge is derived from the complexity and diversity of medical images, which lead to a serious class overlapping and brings a great challenge in discriminating different classes. To solve this problem, we proposed an automated grading approach, which is based on supervised distance metric learning to classify the input discs into four class labels (0: normal, 1: slight, 2: marked, 3: severe). By learning distance metrics from labeled instances, an optimal distance metric is modeled and with two attractive advantages: (1) keeps images from the same classes close, and (2) keeps images from different classes far apart. The experiments, performed in 93 subjects, demonstrated the superiority of our method with accuracy 0.9226, sensitivity 0.9655, specificity 0.9083, F-score 0.8615. With our approach, physicians will be free from the tediousness and patients will be provided an effective treatment.

This study aims to develop content-based image retrieval (CBIR) system for the retrieval of T1-weighted contrast-enhanced MR (CE-MR) images of brain tumors. When a tumor region is fed to the CBIR system as a query, the system attempts to retrieve tumors of the same pathological category. The bag-of-visual-words (BoVW) model with partition learning is incorporated into the system to extract informative features for representing the image contents. Furthermore, a distance metric learning algorithm called the Rank Error-based Metric Learning (REML) is proposed to reduce the semantic gap between low-level visual features and high-level semantic concepts. The effectiveness of the proposed method is evaluated on a brain T1-weighted CE-MR dataset with three types of brain tumors (i.e., meningioma, glioma, and pituitary tumor). Using the BoVW model with partition learning, the mean average precision (mAP) of retrieval increases beyond 4.6% with the learned distance metrics compared with the spatial pyramid BoVW method. The distance metric learned by REML significantly outperforms three other existing distance metric learning methods in terms of mAP. The mAP of the CBIR system is as high as 91.8% using the proposed method, and the precision can reach 93.1% when the top 10 images are returned by the system. These preliminary results demonstrate that the proposed method is effective and feasible for the retrieval of brain tumors in T1-weighted CE-MR Images.