Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification

Two-phase self-supervised pretraining for object re-identification

Prevalent nighttime person re-identification (ReID) methods typically combine image relighting and ReID networks in a sequential manner. However, their performance (recognition accuracy) is limited by the quality of relighting images and insufficient collaboration between image relighting and ReID tasks. To handle these problems, we propose a novel Collaborative Enhancement Network called CENet, which performs the multilevel feature interactions in a parallel framework, for nighttime person ReID. In particular, the designed parallel structure of CENet can not only avoid the impact of the quality of relighting images on ReID performance, but also allow us to mine the collaborative relations between image relighting and person ReID tasks. To this end, we integrate the multilevel feature interactions in CENet, where we first share the Transformer encoder to build the low-level feature interaction, and then perform the feature distillation that transfers the high-level features from image relighting to ReID, thereby alleviating the severe image degradation issue caused by the nighttime scenario while avoiding the impact of relighting images. In addition, the sizes of existing real-world nighttime person ReID datasets are limited, and large-scale synthetic ones exhibit substantial domain gaps with real-world data. To leverage both small-scale real-world and large-scale synthetic training data, we develop a multi-domain learning algorithm, which alternately utilizes both kinds of data to reduce the inter-domain difference in training procedure. Extensive experiments on two real nighttime datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Night600</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RGBNT201_rgb</i> , and a synthetic nighttime ReID dataset are conducted to validate the effectiveness of CENet. We release the code and synthetic dataset at: https://github.com/Alexadlu/CENet.

Abstract. Person Re-Identification (Re-ID) is the task of matching the same person in multiple images captured by different cameras. Recently deep learning-based Re-ID algorithms demonstrated an exciting progress for terrestrial-based cameras, still, person Re-ID in aerial images poses multiple challenges including occlusion of human feature parts, image distortion, and dynamic camera location. In this paper, we propose a new Person Aerial Re-ID framework Robust to Occlusion and Thermal imagery (ParrotGAN). Our model is focused on cross-modality person Re-ID in aerial images. Furthermore, we collected a new large-scale synthetic multimodal AerialReID dataset with 30k images and 137 person ID. Our ParrotGAN model leverages two strategies to achieve robust performance in the task of person Re-ID in thermal and visible range. Firstly, we use a latent space of StyleGAN2 model to estimate the distance between two images of a person. Specifically, we project each real image into the latent space with a correspondent latent vector z. We use the distance between latent vectors to provide a Re-ID similarity metric. Secondly, we use a generative-adversarial network to translate a color image to a synthetic thermal image. We use synthetic image for a cross-modality Re-ID. We evaluate our ParrotGAN model and basleines on our AerialReID and PRAI-1581 datasets. The results of the evaluation are encouraging and demonstrate that our ParrotGAN model competes with baselines in visible range aerial person Re-ID and outperforms them in the cross-modality setting. We made our code and dataset publicly available.

PrGCN: Probability prediction with graph convolutional network for person re-identification

A review on video person re-identification based on deep learning

PA-Net: Learning local features using by pose attention for short-term person re-identification

Compact feature representation of person image is important for person re-identification (Re-ID) task. Recently, part-based representation models have been widely studied for extracting the more compact and robust feature representation for person image to improve person Re-ID results. However, existing part-based representation models mostly extract the features of different parts independently which ignore the spatial relationship information among different parts. To address this issue, in this paper we propose a novel deep learning framework, named Part-based Hierarchical Graph Convolutional Network (PH-GCN) for person Re-ID problem. Given a person image, PH-GCN first constructs a hierarchical graph to represent the spatial relationships among different parts. Then, both local and global feature learning is achieved by the feature information passing in PH-GCN, which takes the information of other parts into account for part feature representation. Finally, a perceptron layer is adopted for the final person part label prediction and re-identification. The proposed framework provides a general solution that integrates <i>local</i>, <i>global</i> and <i>structural</i> feature learning simultaneously in a unified end-to-end network representation and learning. Extensive experiments on several widely used benchmark datasets demonstrate the effectiveness and benefits of the proposed PH-GCN approach for person Re-ID task.

While person Re-identification (Re-ID) has progressed rapidly due to its wide real-world applications, it also causes severe risks of leaking personal information from training data. Thus, this paper focuses on quantifying this risk by membership inference (MI) attack. Most of the existing MI attack algorithms focus on classification models, while Re-ID follows a totally different training and inference paradigm. Re-ID is a fine-grained recognition task with complex feature embedding, and model outputs commonly used by existing MI like logits and losses are not accessible during inference. Since Re-ID focuses on modelling the relative relationship between image pairs instead of individual semantics, we conduct a formal and empirical analysis which validates that the distribution shift of the inter-sample similarity between training and test set is a critical criterion for Re-ID membership inference. As a result, we propose a novel membership inference attack method based on the inter-sample similarity distribution. Specifically, a set of anchor images are sampled to represent the similarity distribution conditioned on a target image, and a neural network with a novel anchor selection module is proposed to predict the membership of the target image. Our experiments validate the effectiveness of the proposed approach on both the Re-ID task and conventional classification task.

Person re-identification (Re-ID) aims to match pedestrian images across various scenes in video surveillance. There are a few works using attribute information to boost Re-ID performance. Specifically, those methods leverage attribute information to boost Re-ID performance by introducing auxiliary tasks like verifying the image level attribute information of two pedestrian images or recognizing identity level attributes. Identity level attribute annotations cost less manpower and are well-fitted for person re-identification task compared with image-level attribute annotations. However, the identity attribute information may be very noisy due to incorrect attribute annotation or lack of discriminativeness to distinguish different persons, which is probably unhelpful for the Re-ID task. In this paper, we propose a novel Attribute Attentional Block (AAB), which can be integrated into any backbone network or framework. Our AAB adopts reinforcement learning to drop noisy attributes based on our designed reward and then utilizes aggregated attribute attention of the remaining attributes to facilitate the Re-ID task. Experimental results demonstrate that our proposed method achieves state-of-the-art results on three benchmark datasets.

Searching accuracy and efficiency are two challenges in person and vehicle Re-identification (Re-ID), which one focuses on robust representations learning which usually generating high-dimensional features while the other has not been fully explored. Hashing is a suitable solution to make REID efficient. However, directly extending the existing hashing methods to fast Re-ID faces two challenges: one is the non-overlap between training and testing set which need more discriminative hash codes, the other is the large identities in Re-ID tasks which will lead to slow convergence and hard optimization. In this work, we propose an attention pooling operator to exploit both local and global visual attributes which can break limited discriminative power in hash methods. To further make training procedure converge faster and optimize the network more easily, we substitute non-differentiable $l_{1^{-}}$ regularization with smooth l 1 -regularization. In experiments, our work outperforms state-of-the-art hashing and quantization methods on both person and vehicle Re-ID datasets. Besides, the results can serve as a strong baseline in the field of deep hashing for fast Re-ID.11Code is available at https://github.com/cynthia951031/Hashing ReID

In recent years, considerable progress has been witnessed in the person re-identification (Re-ID). However, in a more realistic long-term scenario, the appearance shift arising from the clothes-changing inevitably deteriorates the conventional methods that heavily depend on the clothing color. Although the current clothes-changing person Re-ID methods introduce external human knowledge (i.e, contour, mask) and sophisticated feature decoupling strategy to alleviate the clothing shift, they still face the risk of overfitting to clothing due to the limited clothing diversity of training set. To more efficiently and effectively promote the clothes-irrelevant feature learning, we present a novel joint Identity-aware Mixstyle and Graph-enhanced Prototype method for clothes-changing person Re-ID. Specifically, by treating the cloth-changing as fine-grained domain/style shift, the identity-aware mixstyle (IMS) is proposed from the perspective of domain generalization, which mixes the instance-level feature statistics of samples within each identity to synthesize novel and diverse clothing styles, while retaining the correspondence between synthesized samples and latent label space. By incorporating the IMS module, the more diverse styles can be exploited to train a clothing-shift robust model. To further reduce the feature discrepancy caused by clothing variations, the graph-enhanced prototype constraint (GEP) module is proposed to explore the graph similarity structure of style-augmented samples across memory bank to build informative and robust prototypes, which serve as powerful exemplars for better clothing-irrelevant metric learning. The two modules are integrated into a joint learning framework and benefit each other. The extensive experiments conducted on clothes-changing person Re-ID datasets validate the superiority and effectiveness of our method. In addition, our method also shows good universality and corruption robustness on other Re-ID tasks.

Vehicle re-identification (Re-ID) task has far-reaching practical application value and research significance in busy traffic monitoring scenes. Most existing works are based on the single RGB modality or RGB-IR (Infrared) cross-modality. Still, it is difficult to be widely applied in multi-scene real life because of the inadequacy of a single modality at the limited daytime. Recently, multi-spectral Re-ID has been proposed to simultaneously use the RGB and IR spectral data to build a data cell that can complement each other to deal with this condition. For multi-spectral vehicle Re-ID, existing methods simply fuse features extracted from a data cell with multiple modalities, which ignores the large gap as well as the relationship between different modalities. To address this problem, in this work, we propose the Generative and Attentive Fusion Network (GAFNet) to fuse the multiple data sources for the vehicle Re-ID task. First, at the input-level, we enrich the data cell by adding the generated transitional modality images, which could involve different data distribution from the original modality images. Note that the newly generated images have high similarity with both the target modality and the source modality, therefore they would help reduce the gap between modalities and indirectly improve the consistency inside a data cell, and further enhance the weight of key information. Second, at the feature-level, we concatenate and fuse the features of the original data and the transitional modalities from different dimensions with the proposed attentive feature fusion module to further align the features and enhance their focused areas in the feature maps. Extensive experimental results on two standard benchmarks demonstrate that our method is effective and outperforms the state-of-the-art methods.

RGB-D cross-modal person re-identification (Re-ID) task aims to match the person images between the RGB and depth modalities. This task is rather challenging for the tremendous discrepancy between these two modalities in addition to common issues such as lighting conditions, human posture, camera angle, etc. Nowadays only few types of research focus on this task, and existing Re-ID methods tend to learn homogeneous structural relationships in an image, which have limited discriminability and weak robustness to noisy images. In this paper, we propose A Local-Global Interaction Network dedicated to processing cross-modal problems. The network can constrain the center distance between two modals, and improve the intra-class cross-modality similarity. Besides, it can also learn the local and global features of different modalities to enrich the features extracted from different modes. We validate the effectiveness of our approach on public benchmark datasets. Experimental results demonstrate our method outperforms other state-of-the-arts in terms of visual quality and quantitative measurement.KeywordsCross-modal person re-identificationGlobal self-attentionIntra-class cross-modality similarity

For the long-term person re-identification (ReID) task, pedestrians are likely to change clothes, which poses a key challenge in overcoming drastic appearance variations caused by these cloth changes. However, analyzing how cloth changes influence identity-invariant representation learning is difficult. In this context, varying cloth-changed samples are not adaptively utilized, and their effects on the resulting features are overshadowed. To address these limitations, this paper aims to estimate the effect of cloth-changing patterns at both the image and feature levels, presenting a Dual-Level Adaptive Weighting (DLAW) solution. Specifically, at the image level, we propose an adaptive mining strategy to locate the cloth-changed regions for each identity. This strategy highlights the informative areas that have undergone changes, enhancing robustness against cloth variations. At the feature level, we estimate the degree of cloth-changing by modeling the correlation of part-level features and re-weighting identity-invariant feature components. This further eliminates the effects of cloth variations at the semantic body part level. Extensive experiments demonstrate that our method achieves promising performance on several cloth-changing datasets. Code and models are available at https: //github.com/fountaindream/DLAW.

The goal of unsupervised person re-identification (Re-ID) is to use unlabeled person images to learn discriminative features. In recent years, many approaches have adopted clustered pseudo labels to construct proxies for contrastive learning, and have thereby achieved great success. However, existing methods of this kind only utilize local structures within IDs to design their proxies while ignoring the relations between samples of different IDs, which limits the improvement for inter-ID discriminative ability. To resolve this issue, we propose a Global Relation-Aware Contrast Learning (GRACL) method for the task of unsupervised Re-ID. Our method first sets up two proxies for each cluster to capture the inter- and intra-ID relations respectively, which enables us to both effectively increase inter-ID variances and reduce the intra-ID discrepancies. Specifically, the samples that are most different from those in different clusters are selected as inter-ID relation-aware proxies, while those that are least similar to samples from the same clusters are employed as intra-ID relation-aware proxies. With the aid of these proxies, we design both inter- and intra-ID relation-aware contrastive learning modules to facilitate model learning. By pulling each sample close to the positive proxy, we can obtain identity-invariant discriminative features. Experiments on five widely-used Re-ID datasets prove that our GRACL model outperforms current state-of-the-art approaches to a remarkable extent.