A Multi-task Deep Network for video-based Person Re-identification

As a result of its important role in video surveillance, pedestrian attribute recognition has become an attractive facet of computer vision research. Because of the changes in viewpoints, illumination, resolution and occlusion, the task is very challenging. In order to resolve the issue of unsatisfactory performance of existing pedestrian attribute recognition methods resulting from ignoring the correlation between pedestrian attributes and spatial information, in this paper, the task is regarded as a spatiotemporal, sequential, multi-label image classification problem. An attention-based neural network consisting of convolutional neural networks (CNN), channel attention (CAtt) and convolutional long short-term memory (ConvLSTM) is proposed (CNN-CAtt-ConvLSTM). Firstly, the salient and correlated visual features of pedestrian attributes are extracted by pre-trained CNN and CAtt. Then, ConvLSTM is used to further extract spatial information and correlations from pedestrian attributes. Finally, pedestrian attributes are predicted with optimized sequences based on attribute image area size and importance. Extensive experiments are carried out on two common pedestrian attribute datasets, PEdesTrian Attribute (PETA) dataset and Richly Annotated Pedestrian (RAP) dataset, and higher performance than other state-of-the-art (SOTA) methods is achieved, which proves the superiority and validity of our method.

A novel self-boosting dual-branch model for pedestrian attribute recognition

Person re-identification (ReID) aims to find images of the same pedestrian in cross camera scenarios. Recent years have witnessed extensive studies on person ReID by exploiting machine learning techniques under single domain settings. In this paper, we propose a novel cross domain person ReID method based on symmetric coding and pedestrian similarity. Specifically, we first introduce a symmetric coding technique for pedestrian attributes, and compute the attribute similarity matrix. Then, we extract pedestrian features by utilizing convolutional neural networks, and compute the feature similarity matrix. By fusing the pedestrian attribute similarity and pedestrian feature similarity, we obtain the fusion similarity accordingly. The attribute-tensor graph can be represented by the edges composed of the pedestrian similarity, which is produced by pedestrian attribute similarity and pedestrian feature similarity. Finally, the person ReID results can be obtained by similarity ranking. We conduct comprehensive experiments to evaluate the performance of our proposed algorithm for cross domain person ReID tasks, in which the encouraging results validate the efficacy of our proposed cross domain person ReID method.KeywordsCross domainSymmetric codingPedestrian similarityPerson ReID

Person re-identification (re-id) plays an important role in video surveillance and forensics applications. In many cases, person re-id should be conducted between video clips, i.e., given a query pedestrian video from one camera, the re-id system should retrieve the video clips containing the same person from other cameras. However, person re-id between videos, which we call video-based person re-id, has not been well studied. In this paper, we propose a visual-appearance-level and spatial-temporal-level dictionary learning (VSDL) approach for video-based person re-id. Specifically, we first employ two kinds of models to represent each walking cycle in the video, i.e., visual-appearance features of all frames within the walking cycle, and a spatial-temporal feature vector. By separately learning a visual-appearance-level dictionary and a spatial-temporal-level dictionary from two kinds of representations, each walking cycle can be represented as a coding coefficient. To enhance the discriminative ability of the obtained coding coefficients, we design a representation coefficient discriminant term for VSDL. Experiments on the public iLIDS-VID and PRID 2011 datasets demonstrate the effectiveness of VSDL.

Person re-identification has great applications in video surveillance. It can be viewed as recognizing the same person across non-overlapping cameras. Video-based person re-identification methods are gaining increased attention due to the better discriminative nature of spatio-temporal feature representations. Current video-based methods make use of RNN to extract temporal information. In this paper, we propose a novel Moving Average Recurrent Neural Network (MA-RNN) model that can build a strong feature representation by taking both previous and present inputs at each time stamp. Specifically, here the recurrent layer produces a better sequential information by looking back directly in to the past values where as general RNNs has only an indirect dependence on the previous values in the form of hidden-state information. The proposed model is tested on two publicly available datasets: iLIDS-VID and PRID-2011 and it performed better in comparison with the state-of-the-art methods with a significant margin. We also analyze the effect of the depth of previous input dependence of the MA-RNN model on the matching accuracy.

Deep and low-level feature based attribute learning for person re-identification

This paper introduces a groundbreaking approach to Person Re-identification (Re-ID), significantly enhancing accuracy and robustness through contrastive learning, especially under challenging real-world conditions like occlusions. The proposed method addresses the limitations of traditional supervised learning, which often demands extensive labeled data, and unsupervised learning, which can be less accurate due to issues such as occlusion. Our solution features a neural network designed to extract both global and local features essential for identifying individuals across different camera views. The network employs global pooling to generate overall identity representation and horizontal pooling to capture detailed local features crucial for distinguishing person identities. Additionally, we present a novel module that integrates a feature extraction encoder with an MLP to refine the comparison of features. To further enhance the model's performance, we utilize a contrastive loss function, an advancement over the standard InfoNCE Loss, which effectively differentiates between positive and negative samples. Experimental results demonstrate a significant improvement in Person Re-ID accuracy and reliability across various environments. By integrating global and local feature representation and leveraging contrastive learning, our model advances the state-of-the-art in Person Re-ID technology. This innovation has broad applications in video surveillance and automated monitoring systems, setting a new standard for accuracy and reliability in the field.

We investigated the application of multiresolution global and local texture features to reduce false-positive detection in a computerized mass detection program. One hundred and sixty-eight digitized mammograms were randomly and equally divided into training and test groups. From these mammograms, two datasets were formed. The first dataset (manual) contained four regions of interest (ROIs) selected manually from each of the mammograms. One of the four ROIs contained a biopsy-proven mass and the other three contained normal parenchyma, including dense, mixed dense/fatty, and fatty tissues. The second dataset (hybrid) contained the manually extracted mass ROIs, along with normal tissue ROIs extracted by an automated Density-Weighted Contrast Enhancement (DWCE) algorithm as false-positive detections. A wavelet transform was used to decompose an ROI into several scales. Global texture features were derived from the low-pass coefficients in the wavelet transformed images. Local texture features were calculated from the suspicious object and the peripheral subregions. Linear discriminant models using effective features selected from the global, local, or combined feature spaces were established to maximize the separation between masses and normal tissue. Receiver Operating Characteristic (ROC) analysis was conducted to evaluate the classifier performance. The classification accuracy using global features were comparable to that using local features. With both global and local features, the average area, Az, under the test ROC curve, reached 0.92 for the manual dataset and 0.96 for the hybrid dataset, demonstrating statistically significant improvement over those obtained with global or local features alone. The results indicated the effectiveness of the combined global and local features in the classification of masses and normal tissue for false-positive reduction.

In this study, we investigated how shapes are classified based on local and global features by four representative convolutional neural networks (CNNs), i.e., AlexNet, VGG, ResNet and Inception. While the local features are based on simple components, such as orientation of line segment, the global features are based on the whole object, such as whether an object has a hole. For example, solid triangles and solid squares are differentiated by local features, solid circles and rings are differentiated by a global feature. Two sets of experiments were performed in this research. In the first experiment, we examined how the four CNNs pre-trained on ImageNet (with transfer learning) learned to differentiate the regular shapes (equilateral-triangles, squares, circles and rings). Our results showed that the pre-trained CNNs exhibited faster learning rates in the tasks discriminating the local features than in the tasks discriminating the global feature. However, the transfer learning of discriminating the global feature in regular shapes were better generalized to irregular shapes than the transfer learning of discriminating local features. In the second experiment, the CNNs were trained from scratch (with random weights initialization) to discriminate local and global features in regular and irregular shapes. Different from the transfer learning, the CNNs exhibited faster learning rates in discriminating the global feature than the local features. Similar to transfer learning, the CNNs exhibited excellent learning generalization to discriminating the global feature of irregular shapes, but poor learning generalization to discriminating the local features in the irregular shapes. The overarching goal of this research is to create a paradigm and benchmark to directly compare how the CNNs and primate visual systems process geometrical invariants. In contrast to the ImageNet approach which employs natural images to train CNNs, we employed the “ShapeNet” approach which features geometrical shapes with well-defined properties. The ShapeNet approach will not only help elucidate the strengths and limitations of CNN computation, but also provide insights into visual information processing in the primates.

In person re-identification, extracting image features is an important step when retrieving pedestrian images. Most of the current methods only extract global features or local features of pedestrian images. Some inconspicuous details are easily ignored when learning image features, which is not efficient or robust to for scenarios with large differences. In this paper, we propose a Multi-level Feature Fusion model that combines both global features and local features of images through deep learning networks to generate more discriminative pedestrian descriptors. Specifically, we extract local features from different depths of network by the Part-based Multi-level Net to fuse low-to-high level local features of pedestrian images. Global-Local Branches are used to extract the local features and global features at the highest level. The experiments have proved that our deep learning model based on multi-level feature fusion works well in person re-identification. The overall results outperform the state of the art with considerable margins on three widely-used datasets. For instance, we achieve 96% Rank-1 accuracy on the Market-1501 dataset and 76.1% mAP on the DukeMTMC-reID dataset, outperforming the existing works by a large margin (more than 6%).

This study analyzes the effectiveness of the global (the whole face) and local (regions of eyes, nose, and mouth) features for face recognition. Features describing human faces are encoded in local ternary patterns. The two-class support vector machine is used as the supervised learning algorithm for training recognition models. In the recognition process, recognition modes based on the global features and local features are cascaded. For identifying a face image, the local features are used iteratively for filtering out candidates that can not be clearly identified by the global features, until the one with highest possibility is concluded. The experimental results show that cascading the recognition models of global and local features obtains better classification accuracy than the single classification process.

As a sub-direction of image retrieval, person re-identification (Re-ID) is usually used to solve the security problem of cross camera tracking and monitoring. A growing number of shopping centers have recently attempted to apply Re-ID technology. One of the development trends of related algorithms is using an attention mechanism to capture global and local features. We notice that these algorithms have apparent limitations. They only focus on the most salient features without considering certain detailed features. People’s clothes, bags and even shoes are of great help to distinguish pedestrians. We notice that global features usually cover these important local features. Therefore, we propose a dual branch network based on a multi-scale attention mechanism. This network can capture apparent global features and inconspicuous local features of pedestrian images. Specifically, we design a dual branch attention network (DBA-Net) for better performance. These two branches can optimize the extracted features of different depths at the same time. We also design an effective block (called channel, position and spatial-wise attention (CPSA)), which can capture key fine-grained information, such as bags and shoes. Furthermore, based on ID loss, we use complementary triplet loss and adaptive weighted rank list loss (WRLL) on each branch during the training process. DBA-Net can not only learn semantic context information of the channel, position, and spatial dimensions but can integrate detailed semantic information by learning the dependency relationships between features. Extensive experiments on three widely used open-source datasets proved that DBA-Net clearly yielded overall state-of-the-art performance. Particularly on the CUHK03 dataset, the mean average precision (mAP) of DBA-Net achieved 83.2%.

BDNet: A BERT-based dual-path network for text-to-image cross-modal person re-identification

Video-based person re-identification has become a hot topic in the field of research on computer vision and intelligent surveillance, which is more robust to the variations in a person's appearance than single-shot based methods and involves space-time information. However, the most existing spatiotemporal features have been proposed for action recognition that they mainly focus on the exact spatial changes over time. Unlike action recognition, pedestrians captured in person re-identification problem show similar and cyclic walking activities. The essential spatio-temporal information for person re-identification is the statistical information over time. In this paper, we propose a novel spatio-temporal feature, namely Fast Adaptive Spatio-Temporal 3D feature (FAST3D), for video-based person re-identification. The feature is able to extract the statistical motion information based on densely computed multi-direction gradients and an adaptive fusion process. We evaluate our method on two challenging datasets and the experimental results show the effectiveness and efficiency of the proposed feature.

A review on video person re-identification based on deep learning