Multi-kernel metric learning for person re-identification

As a fundamental task in automated video surveillance, person re-identification, which has received increasing attention in recent years, aims to match people across non-overlapping camera views in a multi-camera surveillance system. It has been reported that KISS metric learning has been followed by most of the previous supervised work because of its state of the art performance for person re-identification on VIPeR dataset. However, given only a small number of labeled image pairs available for training, the matching model certainly suffers from unstable learning process and poor matching result. To address this serious practical issue, we proposed a novel semi-supervised KISS metric learning (SS-KISS) approach which makes use of unlabeled data to improve the re-identification performance by 1) combining both global and local information to select the most confident image pairs from the unlabeled data; 2) using an ensemble approach, which explores advantages of supervised and unsupervised learning by reconciling two matching models on which labeled and un-labeled data to an optimal one via smart weighting schema. Extensive experiments have been conducted on three datasets: VIPeR, ETHZ, and i-LiDS, experimental results demonstrate that our approach achieves a sound performance in the case of small amount of labeled data.

Person reidentification has attracted extensive research efforts in recent years. It is challenging due to the varied visual appearance from illumination, view angle, background, and possible occlusions, leading to the difficulties when measuring the relevance, i.e., similarities, between probe and gallery images. Existing methods mainly focus on pairwise distance metric learning for person reidentification. In practice, pairwise image matching may limit the data for comparison (just the probe and one gallery subject) and yet lead to suboptimal results. The correlation among gallery data can be also helpful for the person reidentification task. In this paper, we propose to investigate the high-order correlation among the probe and gallery data, not the pairwise matching, to jointly learn the relevance of gallery data to the probe. Recalling recent progresses on feature representation in person reidentification, it is difficult to select the best feature and each type of feature can benefit person description from different aspects. Under such circumstances, we propose a multihypergraph joint learning algorithm to learn the relevance in corporation with multiple features of the imaging data. More specifically, one hypergraph is constructed using one type of feature and multiple hypergraphs can be generated accordingly. Then, the learning process is conducted on the multihypergraph structure, and the identity of a probe is determined by its relevance to each gallery data. The merit of the proposed scheme is twofold. First, different from pairwise image matching, the proposed method jointly explores the relationships among different images. Second, multimodal data, i.e., different features, can be formulated in the multihypergraph structure, which can convey more information in the learning process and can be easily extended. We note that the proposed method is a general framework to incorporate with any combination of features, and thus is flexible in practice. Experimental results and comparisons with the state-of-the-art methods on three public benchmarking data sets demonstrate the superiority of the proposed method.

Person re-identification problem is a valuable computer vision task, which aims at matching pedestrian images of different cameras in a non-overlapping surveillance network. Existing metric learning based methods address this problem by learning a robust distance function. These methods learn a mapping subspace by forcing the distance of the positive pair much smaller than the negative pair by a strict constraint. The metric model is over-fitting to the training dataset. Due to drastic appearance variations, the handcrafted features are weak of representation for person re-identification. To address these problems, we propose a joint distance measure based approach, which learns a Mahalanobis distance and a Euclidean distance with a novel feature jointly. The novel feature is represented with a dictionary representation based method which considers pedestrian images of different camera views with the same dictionary. The joint distance combine the Mahalanobis distance based on metric learning method with the Euclidean distance based on the novel feature to measure the similarity between matching pairs. Extensive experiments are conducted on the publicly available bench marking datasets VIPeR and CUHK01. The identification results show that the proposed method reaches a good performance than the comparison methods.

In this paper, we propose a novel coupled dictionary learning method, namely coupled analysis-synthesis dictionary learning, to improve the performance of person re-identification in the non-overlapping fields of different camera views. Most of the existing coupled dictionary learning methods train a coupled synthesis dictionary directly on the original feature spaces, which limits the representation ability of the dictionary. To handle the diversities of different original spaces, We first employ local Fisher discriminant analysis (LFDA) to learn a common feature space for close relationship of the same people in different views. In order to enhance the representation power of the coupled synthesis dictionary, we then learn a coupled analysis dictionary by transforming the common feature space into the coupled feature space. Experimental results on two publicly available VIPeR and CUHK01 datasets have validated the effectiveness of the proposed method.

Prevalent nighttime person re-identification (ReID) methods typically combine image relighting and ReID networks in a sequential manner. However, their performance (recognition accuracy) is limited by the quality of relighting images and insufficient collaboration between image relighting and ReID tasks. To handle these problems, we propose a novel Collaborative Enhancement Network called CENet, which performs the multilevel feature interactions in a parallel framework, for nighttime person ReID. In particular, the designed parallel structure of CENet can not only avoid the impact of the quality of relighting images on ReID performance, but also allow us to mine the collaborative relations between image relighting and person ReID tasks. To this end, we integrate the multilevel feature interactions in CENet, where we first share the Transformer encoder to build the low-level feature interaction, and then perform the feature distillation that transfers the high-level features from image relighting to ReID, thereby alleviating the severe image degradation issue caused by the nighttime scenario while avoiding the impact of relighting images. In addition, the sizes of existing real-world nighttime person ReID datasets are limited, and large-scale synthetic ones exhibit substantial domain gaps with real-world data. To leverage both small-scale real-world and large-scale synthetic training data, we develop a multi-domain learning algorithm, which alternately utilizes both kinds of data to reduce the inter-domain difference in training procedure. Extensive experiments on two real nighttime datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Night600</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RGBNT201_rgb</i> , and a synthetic nighttime ReID dataset are conducted to validate the effectiveness of CENet. We release the code and synthetic dataset at: https://github.com/Alexadlu/CENet.

Existing person re-identification methods rely on the visual sensor to capture the pedestrians. The image or video data from visual sensor inevitably suffers the occlusion and dramatic variations of pedestrian postures, which degrades the re-identification performance and further limits its application to the open environment. On the other hand, for most people, one of the most important carry-on items is the mobile phone, which can be sensed by WiFi and cellular networks in the form of a wireless positioning signal. Such signal is robust to the pedestrian occlusion and visual appearance change, but suffers some positioning error. In this work, we approach person re-identification with the sensing data from both vision and wireless positioning. To take advantage of such cross-modality cues, we propose a novel recurrent context propagation module that enables information to propagate between visual data and wireless positioning data and finally improves the matching accuracy. To evaluate our approach, we contribute a new Wireless Positioning Person Re-identification (WP-ReID) dataset. Extensive experiments are conducted and demonstrate the effectiveness of the proposed algorithm. Code will be released at https://github.com/yolomax/WP-ReID.

Recently, the person re-identification task becomes increasingly crucial in crowded scenarios, e.g., airports and schools. Many methods with high performance have been proposed to solve this problem. However, the existence of occlusion still challenges the development of person re-identification. In this work, we present the novel Pose - Guided Mixed Attention Network (PGMANet), an end-to-end framework to deal with pedestrian reidentification under occluded situations by fusing posture and second-order information. Specially, we employ two models. The first is Human Part - level Attention Model. We use key point information of a pedestrian to generate a heat map to enhance the pedestrian body part's feature. Simultaneously, we design Second - order Information Attention Model to investigate the correlation among features of different parts. Experimental results show that our method achieves state-of-the-art person re-identification performance on two challenging occlusion datasets Occluded-DukeMTMC and Occluded-Reid.

Person re-identification (re-ID) as a retrieval task often utilizes a re-ranking model to improve performance. Existing re-ranking methods are typically designed for conventional person re-ID with RGB images, while skeleton representation re-ranking for skeleton-based person re-ID still remains to be explored. To fill this gap, we revisit the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance re-ranking model in this letter, and propose a generic re-ranking method that exploits the salient skeleton features to perform <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance encoding for skeleton-based person re-ID re-ranking. In particular, we devise the skeleton sequence pooling to aggregate the most salient features of skeletons within a sequence, and combine both original Euclidean distance and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -reciprocal distance to re-rank the skeleton sequence representations for person re-ID. Furthermore, we propose the context-based Rank-1 voting that jointly exploits the initial ranking list and re-ranking list to vote for the top candidate to enhance the Rank-1 matching. Extensive experiments on three public benchmarks demonstrate that our approach can effectively re-rank different state-of-the-art skeleton representations and significantly improve their person re-ID performance.

Fully-automated person re-identification in multi-camera surveillance system with a robust kernel descriptor and effective shadow removal method

Person re-identification is to match pedestrian targets observed from different camera views of multi-camera surveillance systems. Aiming at the issues that the color similarity pedestrian affect the recognition result, a person re-identification method based on multi-feature fusion in perceptual uniform color space is proposed, which is according to the characteristics of human vision system. Firstly, the color space which is suitable for similar pedestrian recognition is selected from five color spaces CIELAB, S-CIELAB, IPT, LAB2000HL and CAM02-SCD. Secondly, three kinds of pedestrian appearance features are extracted including spatial weighted histogram, local color and shape feature and global texture feature. Different distance measure methods are used to calculate the similarity of three different features. Finally, the linear fusion is performed by the adaptive weights. The experimental results based on VIPeR database and ETHZ database show that the proposed method is effective and can be used for the recognition of pedestrians with visual similarity.

In recent years, with the development of feature extraction and metric learning of deep neural network, the performance of person re-identification (Re-ID) based on deep learning has been greatly improved. However, the vulnerability of deep neural network is also affecting the security of person Re-ID model, and the adversarial examples become a common way to attack the deep models. In this paper, we propose a targeted adversarial attack method against person Re-ID by adding imperceptible noise on the query examples for the generation of adversarial examples, so that the retrieval results of query examples returned from the model are consistently biased towards the specific pedestrian. To test the effectiveness of the proposed method, several experiments are conducted on the Market-1501 dataset, and the results show that the targeted adversarial attack method can effectively attack the retrieval results of person Re-ID model.

For the long-term person re-identification (ReID) task, pedestrians are likely to change clothes, which poses a key challenge in overcoming drastic appearance variations caused by these cloth changes. However, analyzing how cloth changes influence identity-invariant representation learning is difficult. In this context, varying cloth-changed samples are not adaptively utilized, and their effects on the resulting features are overshadowed. To address these limitations, this paper aims to estimate the effect of cloth-changing patterns at both the image and feature levels, presenting a Dual-Level Adaptive Weighting (DLAW) solution. Specifically, at the image level, we propose an adaptive mining strategy to locate the cloth-changed regions for each identity. This strategy highlights the informative areas that have undergone changes, enhancing robustness against cloth variations. At the feature level, we estimate the degree of cloth-changing by modeling the correlation of part-level features and re-weighting identity-invariant feature components. This further eliminates the effects of cloth variations at the semantic body part level. Extensive experiments demonstrate that our method achieves promising performance on several cloth-changing datasets. Code and models are available at https: //github.com/fountaindream/DLAW.

Person re-identification is an important problem in computer vision fields due to its widely application. However, most of existing person re-identification methods are evaluated in daytime scenarios which is still far from real applications. In this paper, we pay attention to the night scenario person re-identification problem which most of works are not focused on. For this purpose, we contribute a large and real-scenario person re-identification dataset for night scenario named KnightReid, which aims to bridge the gap between theoretical research and practical application. To the best of our knowledge, the KnightReid dataset is the first night scenario dataset for the person re-identification which distinguishes existing works. Furthermore, by carefully examining the properties of night scenario data, we propose to combine image denoising networks with common used person re-identification networks to adapt to this kind of problem. Besides, we provide a comprehensive benchmark result that is evaluated on the dataset. The extensive experiments convince the effectiveness of the proposed model.

Person Re-ID is a challenging problem and is gaining more attention due to demands in security, intelligent system and other applications. Most person Re-ID works are vision-based, such as image, video, or broadly speaking, face recognition-based techniques. Recently, several multi-modal person Re-ID datasets were released, including RGB+IR, RGB+text, RGB+WiFi, which shows the potential of the multi-modal sensor-based person Re-ID approach. However, there are several common issues in public datasets, such as short time duration, lack of appearance change, and limited activities, resulting in un-robust models. For example, vision-based Re-ID models are sensitive to appearance change. In this work, a person Re-ID testbed with multi-modal sensors is created, allowing the collection of sensing modalities including RGB, IR, depth, WiFi, radar, and audio. This novel dataset will cover normal daily office activities with large time span over multi-seasons. Initial analytic results are obtained for evaluating different person Re-ID models, based on small datasets collected in this testbed.

Person re-identification (ReID) is to identify the same person across non-overlapping camera views. After a decade of development, the methods based on deep networks have achieved high performance on benchmarks and become mainstream. In applications, the features of gallery images extracted by deep learning-based methods are stored to speed up the query process and protect the sensitive information contained in the images. Unfortunately, it is demonstrated that turning the images into features cannot properly protect privacy, as these features could be reversed to the corresponding images, revealing the sensitive information they contain. Therefore, for preventing privacy leakage, recent methods learn their features against some feature reversal methods, and most conventional reversal methods focus on minimizing the difference between a reconstruction and its original image. However, there could be many reasonable reconstruction results from a single feature, and the conventional reversal methods will inevitably generate reconstruction results that lie in a different distribution from one of the original images, which cannot properly assess the private information for learning to protect and thus hamper the privacy-protected feature learning. To mitigate this problem, we enforce the reconstructions to follow the same distribution as the original images by the generative adversarial network (GAN). We operate this GAN-based feature reversal module accompanied by the conventional ReID feature extraction module and form a novel GAN-based feature privacy-protected person ReID model, which is expected to protect feature privacy so as against reversal attack and maintain ReID utility. We demonstrate that optimizing ReID model to accommodate privacy protection faces a double adversarial objective and is thus challenging. As a remedy, we design a novel two-step training and lazy update strategy that alternatively optimizes the feature extraction module and stabilizes the update process of the GAN-based feature reversal module. To evaluate the efficiency of the model in balancing its ReID utility and feature privacy protection, we introduce a novel metric called utility-reversibility ratio (URR). Compared with existing privacy-protected feature extraction models, the proposed method achieves a better balance between privacy protection and person ReID performance. Extensive experiments validate that our model can effectively protect feature privacy at a tiny accuracy cost, and validate the effectiveness of our model with the emerging diffusion model.