- Research Article
15
- 10.1016/j.image.2020.115876
Segmentation mask guided end-to-end person search
- May 08, 2020
- Signal Processing: Image Communication
- Dingyuan Zheng + 3 more +3
Segmentation mask guided end-to-end person search
Person search aims to locate and retrieve specific pedestrians in scene images, including two subtasks, pedestrian detection and person re-identification. Recently, triplet loss has been widely used in person re-identification, which effectively improves the pedestrian features embedding and achieves superior performance. However, forming triplet in the person search is not an easy task. Most of the existing end-to-end person search methods are based on Faster R-CNN. The training process of person re-identification part is affected by the detector. It is difficult to form pedestrian triplets within a limited batch size. Also, there are many pedestrian identities in the person search dataset, but each pedestrian identity only has a few samples. It is difficult to learn a robust pedestrian feature representation for person search. To resolve the problem discussed above, a novel Feature Compactness (FC) Loss for the person search is designed, which efficiently improves the inter-class discrimination and intra-class compactness of pedestrian features embedding without the need for positive or negative pairs. Besides, we propose a pedestrian attention module (PAM) to help the network focuses more on pedestrian information and suppresses irrelevant background information. Our method achieves comparable performance on two benchmarks, CUHK-SYSU and PRW, and achieves 91.96% of mAP and 93.34% of rank1 accuracy on CUHK-SYSU.
Segmentation mask guided end-to-end person search
Segmentation mask guided end-to-end person search
Perceiving informative key-points: A self-attention approach for person search
Perceiving informative key-points: A self-attention approach for person search
Sequential End-to-end Network for Efficient Person Search
Person search aims at jointly solving Person Detection and Person Re-identification (re-ID). Existing works have designed end-to-end networks based on Faster R-CNN. However, due to the parallel structure of Faster R-CNN, the extracted features come from the low-quality proposals generated by the Region Proposal Network, rather than the detected high-quality bounding boxes. Person search is a fine-grained task and such inferior features will significantly reduce re-ID performance. To address this issue, we propose a Sequential End-to-end Network (SeqNet) to extract superior features. In SeqNet, detection and re-ID are considered as a progressive process and tackled with two sub-networks sequentially. In addition, we design a robust Context Bipartite Graph Matching (CBGM) algorithm to effectively employ context information as an important complementary cue for person matching. Extensive experiments on two widely used person search benchmarks, CUHK-SYSU and PRW, have shown that our method achieves state-of-the-art results. Also, our model runs at 11.5 fps on a single GPU and can be integrated into the existing end-to-end framework easily.
Read moreEnd-to-end feature diversity person search with rank constraint of cross-class matrix
End-to-end feature diversity person search with rank constraint of cross-class matrix
Person re-identification based on multi-appearance model
Person re-identification plays important roles in many practical applications. Due to various human poses, complex backgrounds and similarity of person clothes, person re-identification is still a challenging task. In this paper, we mainly focus on the robust and discriminative appearance feature representation and proposed a novel multi-appearance method for person re-identification. First, we proposed a deep feature fusion method and get the multi-appearance feature by combining two Convolutional Neural Networks. Then, in order to further enhance the representation of the appearance feature, the multi-part model was constructed by combining the whole body and the six body parts. Additionally, we optimized the feature extraction process by adding a pooling layer. Comprehensive and comparative experiments with the state-of-the-art methods over publicly available datasets demonstrated that the proposed method can get promising results.
Read morePerson Search via Anchor-Free Detection and Part-Based Group Feature Similarity Estimation
In order to solve the problems of insufficient accuracy of pedestrian bounding boxes in person search and large-scale person matching. A novel person search framework is proposed, which includes: (1) A multi-layer cascade heatmap mechanism (MCHM) is proposed, which aggregates pedestrian features by multi-layer heatmaps cascaded and improves the accuracy of the pedestrian bounding box by optimizating the offset between the center of the bounding box and the center point. (2) A learnable part-based pedestrian feature weight calculation module is proposed, which can learn the weight of the part according to the importance of the part-based feature instead of manually set hyperparameters. (3) A group feature correlation graph convolution network (GFCGCN) is proposed, which can calculate the similarity between group pedestrian features and provide a more accuracy end to end person search work. Some ablation studies and comparative experiments on datasets CUHK-SYSU, PRW show that our model can effectively achieve more accuracy pearch search with accuracy of 88.7% rank-1 and 78.2% mAP.
Read moreDhff: Robust Multi-Scale Person Search by Dynamic Hierarchical Feature Fusion
Person Search plays the role of the ultimate destination of person re-identification (re-ID) in real applications. It has many challenges that person re-ID doesn’t need to handle, such as mis-detections, false alarms and multi-scale matching. In contrast to previous works, we show that a strong multi-scale person matching system can result in a good person search performance with a common deep object detector (e.g. Faster-RCNN). In this work, we provide a robust person search method called Dynamic Hierarchical Feature Fusion (DHFF) which is based on multi-level feature fusion to tackle with multi-scale matching. In addition, A Multi-Metric loss is proposed to train the model effectively and stably with numerous identities. We evaluate our method on two large person search benchmark data sets: CUHK-SYSU and PRW. Experiments show that the proposed algorithm outperforms other state-of-the-art person search methods.
Read moreHSME: Hypersphere Manifold Embedding for Visible Thermal Person Re-Identification
Person Re-identification(re-ID) has great potential to contribute to video surveillance that automatically searches and identifies people across different cameras. Heterogeneous person re-identification between thermal(infrared) and visible images is essentially a cross-modality problem and important for night-time surveillance application. Current methods usually train a model by combining classification and metric learning algorithms to obtain discriminative and robust feature representations. However, the combined loss function ignored the correlation between classification subspace and feature embedding subspace. In this paper, we use Sphere Softmax to learn a hypersphere manifold embedding and constrain the intra-modality variations and cross-modality variations on this hypersphere. We propose an end-to-end dualstream hypersphere manifold embedding network(HSMEnet) with both classification and identification constraint. Meanwhile, we design a two-stage training scheme to acquire decorrelated features, we refer the HSME with decorrelation as D-HSME. We conduct experiments on two crossmodality person re-identification datasets. Experimental results demonstrate that our method outperforms the state-of-the-art methods on two datasets. On RegDB dataset, rank-1 accuracy is improved from 33.47% to 50.85%, and mAP is improved from 31.83% to 47.00%.
Read moreEnhancing Person Re-Identification through Attention-Driven Global Features and Angular Loss Optimization.
To address challenges related to the inadequate representation and inaccurate discrimination of pedestrian attributes, we propose a novel method for person re-identification, which leverages global feature learning and classification optimization. Specifically, this approach integrates a Normalization-based Channel Attention Module into the fundamental ResNet50 backbone, utilizing a scaling factor to prioritize and enhance key pedestrian feature information. Furthermore, dynamic activation functions are employed to adaptively modulate the parameters of ReLU based on the input convolutional feature maps, thereby bolstering the nonlinear expression capabilities of the network model. By incorporating Arcface loss into the cross-entropy loss, the supervised model is trained to learn pedestrian features that exhibit significant inter-class variance while maintaining tight intra-class coherence. The evaluation of the enhanced model on two popular datasets, Market1501 and DukeMTMC-ReID, reveals improvements in Rank-1 accuracy by 1.28% and 1.4%, respectively, along with corresponding gains in the mean average precision (mAP) of 1.93% and 1.84%. These findings indicate that the proposed model is capable of extracting more robust pedestrian features, enhancing feature discriminability, and ultimately achieving superior recognition accuracy.
Read moreWeakly Supervised Person Search
While existing person search methods have achieved good performance, they require the images used for training contain labels about the identity and bounding box location of each person. However, it is expensive and difficult to manually annotate these labels in the large scale scenario. To overcome this issue, we consider weakly supervised person search. The weakly supervised setting means during training we only know which identities appear in the image set and how many individuals present in each image, without any identity or location information on the image. Facing this challenge, we propose a clustering and patch based weakly supervised learning (CPBWSL) framework, which separately addresses two sub-tasks including pedestrian detection and person re-identification. Particularly, we introduce multiple detectors to provide more detection results as well as fuzzy c-means clustering algorithm to cluster these results and remove low membership ones. Moreover, a patch based learning network is designed to generate different patches and learn discriminative patch features. Extensive experiments on two benchmarks indicate that the proposed weakly supervised setting is feasible and our method can achieve performance comparable to some fully supervised person search methods.
Read moreHybrid Modality Metric Learning for Visible-Infrared Person Re-Identification
Visible-infrared person re-identification (Re-ID) has received increasing research attention for its great practical value in night-time surveillance scenarios. Due to the large variations in person pose, viewpoint, and occlusion in the same modality, as well as the domain gap brought by heterogeneous modality, this hybrid modality person matching task is quite challenging. Different from the metric learning methods for visible person re-ID, which only pose similarity constraints on class level, an efficient metric learning approach for visible-infrared person Re-ID should take both the class-level and modality-level similarity constraints into full consideration to learn sufficiently discriminative and robust features. In this article, the hybrid modality is divided into two types, within modality and cross modality. We first fully explore the variations that hinder the ranking results of visible-infrared person re-ID and roughly summarize them into three types: within-modality variation, cross-modality modality-related variation, and cross-modality modality-unrelated variation. Then, we propose a comprehensive metric learning framework based on four kinds of paired-based similarity constraints to address all the variations within and cross modality. This framework focuses on both class-level and modality-level similarity relationships between person images. Furthermore, we demonstrate the compatibility of our framework with any paired-based loss functions by giving detailed implementation of combing it with triplet loss and contrastive loss separately. Finally, extensive experiments of our approach on SYSU-MM01 and RegDB demonstrate the effectiveness and superiority of our proposed metric learning framework for visible-infrared person Re-ID.
Read moreAn Improved Split-Attention Architecture Based on Circle Loss for Person Re-Identification
Person re-identification aims to match pedestrian images across multiple surveillance camera views. It is still a challenging task due to the partial occlusion of pedestrian images, variations in the illumination of surveillance cameras, and similar appearances of pedestrians and so on. In order to improve the representation ability of pedestrian features extracted from the convolutional neural networks, in this paper, we proposed an improved split-attention architecture for person re-identification. Specifically, we first divide the feature map into two sub-groups and then split the features in each subgroup into three more fine-grained sub-feature maps. Moreover, in order to minimize the inter-class similarity and maximize the intra-class similarity, we use circle loss and identification loss to optimize our network together. Circle loss makes the similarity scores learn at different paces, which benefits deep feature learning. The circle loss not only makes the model have higher optimization flexibility but also makes the convergence target of the model more definite. Unlike many methods that use complex convolutional neural networks to represent pedestrian feature maps in a layer-wise manner, our proposed method improves the representation ability of pedestrian features at a more fine-grained level. We evaluated the performance of our proposed network on two large-scale person re-identification benchmark datasets Market-1501 and DukeMTMC-reID. Experimental results show that the proposed split-attention network outperforms the state-of-the-art methods on both datasets with only using pedestrian global features.
Read moreCross Domain Person ReID Based on Symmetric Coding and Pedestrian Similarity
Person re-identification (ReID) aims to find images of the same pedestrian in cross camera scenarios. Recent years have witnessed extensive studies on person ReID by exploiting machine learning techniques under single domain settings. In this paper, we propose a novel cross domain person ReID method based on symmetric coding and pedestrian similarity. Specifically, we first introduce a symmetric coding technique for pedestrian attributes, and compute the attribute similarity matrix. Then, we extract pedestrian features by utilizing convolutional neural networks, and compute the feature similarity matrix. By fusing the pedestrian attribute similarity and pedestrian feature similarity, we obtain the fusion similarity accordingly. The attribute-tensor graph can be represented by the edges composed of the pedestrian similarity, which is produced by pedestrian attribute similarity and pedestrian feature similarity. Finally, the person ReID results can be obtained by similarity ranking. We conduct comprehensive experiments to evaluate the performance of our proposed algorithm for cross domain person ReID tasks, in which the encouraging results validate the efficacy of our proposed cross domain person ReID method.KeywordsCross domainSymmetric codingPedestrian similarityPerson ReID
Read morePH-GCN: Person Retrieval With Part-Based Hierarchical Graph Convolutional Network
Compact feature representation of person image is important for person re-identification (Re-ID) task. Recently, part-based representation models have been widely studied for extracting the more compact and robust feature representation for person image to improve person Re-ID results. However, existing part-based representation models mostly extract the features of different parts independently which ignore the spatial relationship information among different parts. To address this issue, in this paper we propose a novel deep learning framework, named Part-based Hierarchical Graph Convolutional Network (PH-GCN) for person Re-ID problem. Given a person image, PH-GCN first constructs a hierarchical graph to represent the spatial relationships among different parts. Then, both local and global feature learning is achieved by the feature information passing in PH-GCN, which takes the information of other parts into account for part feature representation. Finally, a perceptron layer is adopted for the final person part label prediction and re-identification. The proposed framework provides a general solution that integrates <i>local</i>, <i>global</i> and <i>structural</i> feature learning simultaneously in a unified end-to-end network representation and learning. Extensive experiments on several widely used benchmark datasets demonstrate the effectiveness and benefits of the proposed PH-GCN approach for person Re-ID task.
Read moreAn enhanced siamese angular softmax network with dual joint-attention for person re-identification
For person re-identification (re-ID), a core problem is how to learn discriminative feature representations of pedestrians. In this paper, we propose a novel enhanced siamese angular softmax network (ES-ASnet) to integrate identification, verification and metric learning into a unified network. First, a dual joint-attention (DJA) based identification model is proposed that can focus on both key local information and global contextual dependencies in spatial and channel domains simultaneously. Then, we adopt angular softmax (A-Softmax) loss in the training phase, which directly integrates metric learning into classification to enhance the discriminative capability of features in the angular space. Furthermore, the alignment module in the unified network can reduce the impact of misalignment between image pairs, which can further learn robust discriminative feature representations effectively. Experiments on three main person re-ID datasets, including Market1501, DukeMTMC-reID and CUHK03-NP, demonstrate that the proposed network has achieved competitive performance compared with several state-of-the-art methods for person re-ID.
Read more