Enforcing Affinity Feature Learning through Self-attention for Person Re-identification

Compact feature representation of person image is important for person re-identification (Re-ID) task. Recently, part-based representation models have been widely studied for extracting the more compact and robust feature representation for person image to improve person Re-ID results. However, existing part-based representation models mostly extract the features of different parts independently which ignore the spatial relationship information among different parts. To address this issue, in this paper we propose a novel deep learning framework, named Part-based Hierarchical Graph Convolutional Network (PH-GCN) for person Re-ID problem. Given a person image, PH-GCN first constructs a hierarchical graph to represent the spatial relationships among different parts. Then, both local and global feature learning is achieved by the feature information passing in PH-GCN, which takes the information of other parts into account for part feature representation. Finally, a perceptron layer is adopted for the final person part label prediction and re-identification. The proposed framework provides a general solution that integrates <i>local</i>, <i>global</i> and <i>structural</i> feature learning simultaneously in a unified end-to-end network representation and learning. Extensive experiments on several widely used benchmark datasets demonstrate the effectiveness and benefits of the proposed PH-GCN approach for person Re-ID task.

Given images of a person, person re- identification (Person ReID) techniques aim to find images of the same person from previously collected images. Because of large data sets of person images and the advance of deep learning, convolutional neural networks (CNNs) successfully boost the accuracy of Person ReID algorithms, but it can be difficult to explain and to troubleshoot issues due to the complexity of CNNs. In this paper, we present a visualization-based approach to understand a CNN-based Person ReID algorithm. As Person ReID algorithms are often designed to map images of the same person into similar feature vectors, given two images, we design an algorithm to estimate how much each element in a CNN layer contributes to the similarity between their feature vectors. Based on the estimation, we build a visualization tool to interactively locate and visualize the activation of highly-contributing elements, other than manually examining all. Our visualization tool also supports various user interaction widgets to explore a Person ReID data set, locate difficult cases, and analyze the reason behind their similarities. We show a use case with our tool to understand and troubleshoot issues in a CNN-based Person ReID algorithm.

Image quality assessment (IQA) is very important for both end-users and service providers since a high-quality image can significantly improve the user's quality of experience (QoE) and also benefit lots of computer vision algorithms. Most existing blind image quality assessment (BIQA) models were developed for synthetically distorted images, however, they perform poorly on in-the-wild images, which are widely existed in various practical applications. In this paper, we propose a novel BIQA model for in-the-wild images by addressing two critical problems in this field: how to learn better quality-aware feature representation, and how to solve the problem of insufficient training samples in terms of their content and distortion diversity. Considering that perceptual visual quality is affected by both low-level visual features (e.g. distortions) and high-level semantic information (e.g. content), we first propose a staircase structure to hierarchically integrate the features from intermediate layers into the final feature representation, which enables the model to make full use of visual information from low-level to high-level. Then an iterative mixed database training (IMDT) strategy is proposed to train the BIQA model on multiple databases simultaneously, so the model can benefit from the increase in both training samples and image content and distortion diversity and can learn a more general feature representation. Experimental results show that the proposed model outperforms other state-of-the-art BIQA models on six in-the-wild IQA databases by a large margin. Moreover, the proposed model shows an excellent performance in the cross-database evaluation experiments, which further demonstrates that the learned feature representation is robust to images with diverse distortions and content. The code is available at https://github.com/sunwei925/StairIQA.

Attentive occlusion-adaptive deep network for facial landmark detection

Person re-identification has great applications in video surveillance. It can be viewed as recognizing the same person across non-overlapping cameras. Video-based person re-identification methods are gaining increased attention due to the better discriminative nature of spatio-temporal feature representations. Current video-based methods make use of RNN to extract temporal information. In this paper, we propose a novel Moving Average Recurrent Neural Network (MA-RNN) model that can build a strong feature representation by taking both previous and present inputs at each time stamp. Specifically, here the recurrent layer produces a better sequential information by looking back directly in to the past values where as general RNNs has only an indirect dependence on the previous values in the form of hidden-state information. The proposed model is tested on two publicly available datasets: iLIDS-VID and PRID-2011 and it performed better in comparison with the state-of-the-art methods with a significant margin. We also analyze the effect of the depth of previous input dependence of the MA-RNN model on the matching accuracy.

Due to variations in pose, angle and illumination condition, a person's appearance is significantly different in two different views, which makes person re-identification(re-id) intrinsically difficult. In this paper, we propose a person re-id method which learns Convolutional Neural Networks (CNNs) feature representations from joint-dataset learning. The CNN features extracted from all levels of the deep Convolutional Neural Network can preserve robust and generic feature representations. Experimental results show that our proposed method is effective, which outperforms other state-of-the-art methods on VIPeR, PRID 2011, CUHK01, CUHK03 and i-LIDS datasets by large margins.

Person re-identification is the task of matching pedestrian images across a network of non-overlapping camera views. It poses aggregated challenges resulted from random human pose, clutter from the background, illumination variations, and other factors. There has been a vast number of studies in recent years with promising success. However, key challenges have not been adequately addressed and continue to result in sub-optimal performance. Attention-based person re-identification gains more popularity in identifying discriminatory features from person images. Its potential in terms of extracting features common to a pair of person images across the feature extraction pipeline has not been be fully exploited. In this paper, we propose a novel attention-based Siamese network driven by a mutual-attention module decomposed into spatial and channel components. The proposed mutual-attention module not only leads feature extraction to the discriminative part of individual images, but also fuses mutual features symmetrically across pairs of person images to get informative regions common to both input images. Our model simultaneously learns feature embedding for discriminative cues and the similarity measure. The proposed model is optimized with multi-task loss, namely classification and verification loss. It is further optimized by a learnable mutual-attention module to facilitate an efficient and adaptive learning. The proposed model is thoroughly evaluated on extensively used large-scale datasets, Market-1501 and Duke-MTMC-ReID. Our experimental results show competitive results with the state-of-the-art works and the effectiveness of the mutual-attention module.

Person re-identification (Re-ID) has become increasingly popular in vision community. Many previous works rely on either feature representation learning and/or metric learning. Different from classical methods, we find that some text in images could be considered as a key cue for differentiating persons under some circumstances (e.g., racing bib number of marathon participants). Based on this observation, we propose to simplify the person Re-ID problem into an end-to-end text recognition problem. Thanks to many powerful state-of-the-art text recognition systems, we can largely improve the efficiency and accuracy of person re-identification in such circumstances. Moreover, we collect a dataset consisting of 9706 marathon images and propose an appropriate measurement to benchmark person identification. Our work provides a promising perspective to person Re-ID and end-to-end text recognition fields, showing also high potentials for video surveillance and image retrieval.

Person Re-IDentification (Re-ID) is a critical technology to identify a target person from captured person images by surveillance cameras. However, person Re-ID has triggered great concerns of personal image privacy. Although the law (e.g., GDPR) has stipulated person images are personal private data, there is no an efficient solution to tackle the image privacy concern for person Re-ID. To this end, we propose FREED, the first system solution for privacy-preserving person Re-ID, which supports the state-of-the-art person Re-ID operations on encrypted feature vectors of person images. To handle the encryption of feature vectors effectively and enable person Re-ID operations on encrypted feature vectors efficiently, FREED develops a suite of batch secure computing protocols based on a twin-server architecture and the threshold Paillier cryptosystem. We demonstrate our secure computing protocols are more efficient than existing protocols and FREED achieves a precision equal to the state-of-the-art plaintext method.

Person re-identification (Re-ID) aims to retrieve a person of interest across multiple nonoverlapping cameras. In the last few years, the construction of aerial person Re-ID datasets has been appealing because visual surveillance through unmanned aerial vehicle (UAV) platforms has become very valuable in real-world scenarios. However, great differences are exhibited between the pedestrian images captured by ground cameras and those captured by UAVs. Person Re-ID methods based on ground person images have difficulty when performing Re-ID based on aerial person images. In this paper, we first propose a novel meta-transfer learning method for person Re-ID in aerial imagery; this approach trains a generalisable Re-ID model to learn discriminative feature representations for aerial person images. Specifically, a meta-learning strategy is introduced to study a feature extractor, and a transfer learning strategy is introduced to utilise and further improve the acquired meta-knowledge. To overcome the convergence speed and recognition accuracy reductions caused by the presence of difficult categories in the given dataset, we propose a learning strategy based on curriculum sampling that is harmonised with our meta-transfer learning framework. In addition, a new metric formulation of sample similarity is introduced based on Mahalanobis distance to improve the optimisation of the model. Extensive comparative evaluation experiments are conducted on the large-scale aerial Re-ID dataset, and the results obtained show that our method achieves a Rank-1 accuracy of 63.63 $$\%$$ and a mean average precision (mAP) of 38.02 $$\%$$ , demonstrating its potential for completing person Re-ID in aerial images. Ablation studies also validate that each component contributes to improving the performance of the model.

EMBANet: A flexible efficient multi-branch attention network.

This paper introduces a groundbreaking approach to Person Re-identification (Re-ID), significantly enhancing accuracy and robustness through contrastive learning, especially under challenging real-world conditions like occlusions. The proposed method addresses the limitations of traditional supervised learning, which often demands extensive labeled data, and unsupervised learning, which can be less accurate due to issues such as occlusion. Our solution features a neural network designed to extract both global and local features essential for identifying individuals across different camera views. The network employs global pooling to generate overall identity representation and horizontal pooling to capture detailed local features crucial for distinguishing person identities. Additionally, we present a novel module that integrates a feature extraction encoder with an MLP to refine the comparison of features. To further enhance the model's performance, we utilize a contrastive loss function, an advancement over the standard InfoNCE Loss, which effectively differentiates between positive and negative samples. Experimental results demonstrate a significant improvement in Person Re-ID accuracy and reliability across various environments. By integrating global and local feature representation and leveraging contrastive learning, our model advances the state-of-the-art in Person Re-ID technology. This innovation has broad applications in video surveillance and automated monitoring systems, setting a new standard for accuracy and reliability in the field.

Person re-identification aims to match the same pedestrians across different camera views and has been applied to many important applications such as intelligent video surveillance. Due to the spatiotemporal uncertainty and visual ambiguity of pedestrian image pairs, person re-identification remains a difficult and challenging problem. The huge success of deep learning has focused attention on the use of deep features for person re-identification. However, for person re-identification, most deep learning methods minimize cross-entropy or triplet-based losses, thereby neglecting the fact that the similarities and differences between image pairs can be considered simultaneously to increase discrimination. In this paper, we propose a novel deep learning method called deep similarity feature learning DSFL to extract more effective deep features for image pairs. Extensive experiments on two representative person re-identification datasets CUHK-03 and GRID demonstrate the effectiveness and robustness of DSFL.

Captured images of outdoor scenes usually exhibit low visibility in cases of severe haze, which interferes with optical imaging and degrades image quality. Most of the existing methods solve the single-image dehazing problem by applying supervised training on paired images; however, in practice, the pairing of real-world images is not viable. Additionally, the processing speed of individual dehazing models is important in practical applications. In this study, a novel unsupervised single image dehazing network (USID-Net) based on disentangled representations without paired training images is explored. Furthermore, considering the trade-off between performance and memory storage, a compact multi-scale feature attention (MFA) module is developed, integrating multi-scale feature representation and attention mechanism to facilitate feature representation. To effectively extract haze information, a mechanism referred to as OctEncoder is designed to include multi-frequency representations that can capture more global information. Extensive experiments show that USID-Net achieves competitive dehazing results and a relatively high processing speed compared to state-of-the-art methods.

Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification