- Research Article
73
- 10.1016/j.patcog.2017.07.005
Multi-type attributes driven multi-camera person re-identification
- Sep 12, 2017
- Pattern Recognition
- Chi Su + 4 more +4
Multi-type attributes driven multi-camera person re-identification
The visual appearance of a person is easily affected by many factors like pose variations, viewpoint changes and camera parameter differences. This makes person Re-Identification (ReID) among multiple cameras a very challenging task. This work is motivated to learn mid-level human attributes which are robust to such visual appearance variations. And we propose a semi-supervised attribute learning framework which progressively boosts the accuracy of attributes only using a limited number of labeled data. Specifically, this framework involves a three-stage training. A deep Convolutional Neural Network (dCNN) is first trained on an independent dataset labeled with attributes. Then it is fine-tuned on another dataset only labeled with person IDs using our defined triplet loss. Finally, the updated dCNN predicts attribute labels for the target dataset, which is combined with the independent dataset for the final round of fine-tuning. The predicted attributes, namely \emph{deep attributes} exhibit superior generalization ability across different datasets. By directly using the deep attributes with simple Cosine distance, we have obtained surprisingly good accuracy on four person ReID datasets. Experiments also show that a simple metric learning modular further boosts our method, making it significantly outperform many recent works.
Multi-type attributes driven multi-camera person re-identification
Multi-type attributes driven multi-camera person re-identification
Person Re-Identification With Joint Verification and Identification of Identity-Attribute Labels
One of the major challenges in person Re-Identification (ReID) is the inconsistent visual appearance of a person. Current works on visual feature and distance metric learning have achieved significant achievements, but still suffer from the limited robustness to pose variations, viewpoint changes, etc. This makes person ReID among multiple cameras still challenging. This work is motivated to learn mid-level human attributes which are robust to visual appearance variations and could be used as efficient features for person matching. We propose a supervised multi-task learning framework which considers attribute label information with joint identification-verification network to simultaneously learn an attribute-semantic and identity-discriminative feature representation. Specifically, this framework adopts the part-based deep neural network and learn three different tasks simultaneously: person identification, person verifications and attribute identification, so as to discover and capture concurrently complementary discriminative information about a person image from global and local image features and mid-level attribute features in one deep neural network. With the multi-task learning architecture, we obtain a discriminative model that reaches a synergy in distinguishing different person images, as manifested with the competitive accuracy on three person ReID datasets: Market1501, DukeMTMC-reID and VIPeR.
Read moreUnsupervised Person Re-Identification via Deep Attribute Learning
Driven by growing public security demands and the advancement of intelligent surveillance systems, person re-identification (ReID) has emerged as a prominent research focus in the field of computer vision. However, this task presents challenges due to its high sensitivity to variations in visual appearance caused by factors such as body pose and camera parameters. Although deep learning-based methods have achieved marked progress in ReID, the high cost of annotation remains a challenge that cannot be overlooked. To address this, we propose an unsupervised attribute learning framework that eliminates the need for costly manual annotations while maintaining high accuracy. The framework learns the mid-level human attributes (such as clothing type and gender) that are robust to substantial visual appearance variations and can hence boost the accuracy of attributes with a small amount of labeled data. To carry out our framework, we present a part-based convolutional neural network (CNN) architecture, which consists of two components for image and body attribute learning on a global level and upper- and lower-body image and attribute learning at a local level. The proposed architecture is trained to learn attribute-semantic and identity-discriminative feature representations simultaneously. For model learning, we first train our part-based network using a supervised approach on a labeled attribute dataset. Then, we apply an unsupervised clustering method to assign pseudo-labels to unlabeled images in a target dataset using our trained network. To improve feature compatibility, we introduce an attribute consistency scheme for unsupervised domain adaptation on this unlabeled target data. During training on the target dataset, we alternately perform three steps: extracting features with the updated model, assigning pseudo-labels to unlabeled images, and fine-tuning the model. Through a unified framework that fuses complementary attribute-label and identity label information, our approach achieves considerable improvements of 10.6% and 3.91% mAP on Market-1501→DukeMTMC-ReID and DukeMTMC-ReID→Market-1501 unsupervised domain adaptation tasks, respectively.
Read moreRepeated Pattern Detection Using CNN Activations
We propose a new approach for detecting repeated patterns on a grid in a single image. To do so, we detect repetitions in the space of pre-trained deep CNN filter responses at all layer levels. These encode features at several conceptual levels (from low-level patches to high-level semantics) as well as scales (from local to global). As a result, our repeated pattern detector is robust to challenging cases where repeated tiles show strong variation in visual appearance due to occlusions, lighting or background clutter. Our method contrasts with previous approaches that rely on keypoint extraction, description and clustering or on patch correlation. These generally only detect low-level feature clusters that do not handle variations in visual appearance of the patterns very well. Our method is simpler, yet incorporates high level features implicitly. As such, we can demonstrate detections of repetitions with strong appearance variations, organized on a nearly-regular axis-aligned grid Results show robustness and consistency throughout a varied database of more than 150 images.
Read moreTriplet CNN and pedestrian attribute recognition for improved person re-identification
In this paper, we propose a pedestrian attribute recognition approach and a CNN-based person re-identification framework enhanced by pedestrian attributes. The knowledge of person attributes can help video surveillance tasks like person re-identification as well as person search, semantic video indexing and retrieval to overcome viewpoint changes with their robustness to the inherent visual appearance variations. Compared to previous approaches, our attribute recognition method using Local Maximal Occurrence (LOMO) features and a Multi-Label Multi-Layer Perceptron (MLMLP) classifier proves to be more robust to different view points and is computationally more efficient. The experiments on three public benchmarks show that the proposed method improves the state-of-the art on attribute recognition. Furthermore, we integrate our attribute recognition algorithm into a triplet CNN similarity learning framework for person re-identification fusing both learned CNN features and attributes. This fusion leads to an overall improvement, and we achieve state-of-the-art results on person re-identification.
Read moreAppearance-invariant place recognition by discriminatively training a convolutional neural network
Appearance-invariant place recognition by discriminatively training a convolutional neural network
Person re-identification transformer with patch attention and pruning
Person re-identification transformer with patch attention and pruning
Person Re-identification and Tracking in Video Surveillance
Video surveillance system is one of the most essential topics in the computer vision field. As the rapid and continuous increasement of using video surveillance cameras to obtain portrait information in scenes, it becomes a very important system for security and criminal investigations. Video surveillance system includes many key technologies, including the object recognition, the object localization, the object re-identification, object tracking, and by which the system can be used to identify or suspect the movements of the objects and persons. In recent years, person re-identification and visual object tracking have become hot research directions in the computer vision field. The re-identification system aims to recognize and identify the target of the required attributes, and the tracking system aims at following and predicting the movement of the target after the identification process. Researchers have used deep learning and computer vision technologies to significantly improve the performance of person re-identification. However, the study of person re-identification is still challenging due to complex application environments such as lightning variations, complex background transformations, low-resolution images, occlusions, and a similar dressing of different pedestrians. The challenge of this task also comes from unavailable bounding boxes for pedestrians, and the need to search for the person over the whole gallery images. To address these critical issues in modern person identification applications, we propose an algorithm that can accurately localize persons by learning to minimize intra-person feature variations. We build our model upon the state-of-the-art object detection framework, i.e., faster R-CNN, so that high-quality region proposals for pedestrians can be produced in an online manner. In addition, to relieve the negative effects caused by varying visual appearances of the same individual, we introduce a novel center loss that can increase the intra-class compactness of feature representations. The engaged center loss encourages persons with the same identity to have similar feature characteristics. Besides the localization of a single person, we explore a more general visual object tracking problem. The main task of the visual object tracking is to predict the location and size of the tracking target accurately and reliably in subsequent image sequences when the target is given at the beginning of the sequence. A visual object tracking algorithm with high accuracy, good stability, and fast inference speed is necessary. In this thesis, we study the updating problem for two kinds of tracking algorithms among the mainstream tracking approaches, and improve the robustness and accuracy. Firstly, we extend the siamese tracker with a model updating mechanism to improve their tracking robustness. A siamese tracker uses a deep convolutional neural network to obtain features and compares the new frame features with the target features in the first frame. The candidate region with the highest similarity score is considered as the tracking result. However, these kinds of trackers are not robust against large target variation due to the no-update matching strategy during the whole tracking process. To combat this defect, we propose an ensemble siamese tracker, where the final similarity score is also affected by the similarity with tracking results in recent frames instead of solely considering the first frame. Tracking results in recent frames are used to adjust the model for a continuous target change. Meanwhile, we combine adaptive candidate sampling strategy and large displacement optical flow method to improve its performance further. Secondly, we investigate the classic correlation filter based tracking algorithm and propose to provide a better model selection strategy by reinforcement learning. Correlation filter has been proven to be a useful tool for a number of approaches in visual tracking, particularly for seeking a good balance between tracking accuracy and speed. However, correlation filter based models are susceptible to wrong updates stemming from inaccurate tracking results. To date, little effort has been devoted to handling the correlation filter update problem. In our approach, we update and maintain multiple correlation filter models in parallel, and we use deep reinforcement learning for the selection of an optimal correlation filter model among them. To facilitate the decision process efficiently, we propose a decision-net to deal with target appearance modeling, which is trained through hundreds of challenging videos using proximal policy optimization and a lightweight learning network. An exhaustive evaluation of the proposed approach on the OTB100 and OTB2013 benchmarks show the effectiveness of our approach.
Read moreCross-dataset person re-identification using deep convolutional neural networks: effects of context and domain adaptation
Over the past years, the impact of surveillance systems on public safety increases dramatically. One significant challenge in this domain is person re-identification, which aims to detect whether a person has already been captured by another camera in the surveillance network or not. Most of the work that has been conducted on person re-identification problem uses a single dataset, in which the training and test data are coming from the same source. However, as we have shown in this work, there is a strong bias among the person re-identification datasets, therefore, a method that has been trained and optimized on a specific person re-identification dataset may not generalize well and perform successfully on the other datasets. This is a problem for many real-world applications, since it is not feasible to collect and annotate sufficient amount of data from the target application to train or fine-tune a deep convolutional neural network model. Taking this issue into account, in this work, we have focused on cross-dataset person re-identification problem and first explored and analyzed in detail the use of the state-of-the-art deep convolutional neural network architectures, namely AlexNet, VGGNet, GoogLeNet, ResNet, and DenseNet that have been developed for generic image classification task. These deep CNN models have been adapted to the person re-identification domain by fine-tuning them for each human body part separately, as well as on the entire body, with the two relatively large person re-identification datasets: CUHK03 and Market-1501. Then, the performance of each adapted model has been evaluated on two different publicly available datasets: VIPeR and PRID2011. We have shown that, even just a domain adaptation leads comparable results to the state-of-the-art cross-dataset approaches. Another point that we have addressed in this paper is context adaptation. It has been known that person re-identification approaches implicitly utilizes background as context information. Therefore, to have a consistent background across different camera views, we have employed the cycle-consistent generative adversarial network. We have shown that this further improves the performance.
Read moreDeep distributed convolutional neural networks: Universality
Deep learning based on structured deep neural networks has provided powerful applications in various fields. The structures imposed on the deep neural networks are crucial, which makes deep learning essentially different from classical schemes based on fully connected neural networks. One of the commonly used deep neural network structures is generated by convolutions. The produced deep learning algorithms form the family of deep convolutional neural networks. Despite of their power in some practical domains, little is known about the mathematical foundation of deep convolutional neural networks such as universality of approximation. In this paper, we propose a family of new structured deep neural networks: deep distributed convolutional neural networks. We show that these deep neural networks have the same order of computational complexity as the deep convolutional neural networks, and we prove their universality of approximation. Some ideas of our analysis are from ridge approximation, wavelets, and learning theory.
Read moreEstimation and monitoring of product aesthetics: application to manufacturing of “engineered stone” countertops
A new machine vision approach for quantitatively estimating and monitoring the appearance and aesthetics of manufactured products is presented. The approach is composed of three steps: (1) wavelet-based textural feature extraction from product images, (2) estimation of measures of the product appearance through subspace projection of the textural features, and (3) monitoring of the appearance in the latent variable subspace of the textural features. The methodology is specifically designed to treat the stochastic nature of the visual appearance of many manufactured products. This nondeterministic aspect of product appearance has been an obstacle for the success of machine vision in many industries. The emphasis of this approach is on the consistent and quantitative estimation of continuous variations in visual appearance rather than on classification into discrete classes. This allows for the on-line monitoring and the eventual feedback control of product appearance. This approach is successfully applied to the estimation and monitoring of the aesthetic quality of manufactured stone countertops.
Read moreSubclass linear discriminant analysis for video-based face recognition
Subclass linear discriminant analysis for video-based face recognition
Enriched and Discriminative Human Features for Person Re-Identification Based on Explainable Behaviors of Convolutional Neural Networks
Understanding pedestrian behaviors such as their movement patterns in urban areas could contribute to the design of pedestrian-friendly facilities. With the commonly deployed surveillance cameras, pedestrian movement in a wide region could be identified by the person re-identification (ReID) technique across multiple cameras. Convolutional neural networks (CNNs) have been widely studied to automate the ReID task. CNN models equipped with deep learning techniques could extract discriminative human features from images and show promising ReID performance. However, some common challenges such as occlusion and appearance variation are still unsolved. Specifically, our study infers that over-relying on discriminative features only may compromise ReID performance. Therefore, this paper proposes a new model that extracts enriched features, which is more reliable against those ReID challenges. By adding a feature dropping strategy during model training, our model learns to focus on rich human features from different body parts. Moreover, this paper presents an explainable approach of model design, by visualizing which human parts a deep learning model focuses on. Based on an intuitive interpretation of model behaviors that lead to inaccurate results, specific improvement of model architecture is inspired. Our improved results suggest that making existing models explainable could effectively shed light on designing more robust models.
Read moreResearch on improved convolutional wavelet neural network
Artificial neural networks (ANN) which include deep learning neural networks (DNN) have problems such as the local minimal problem of Back propagation neural network (BPNN), the unstable problem of Radial basis function neural network (RBFNN) and the limited maximum precision problem of Convolutional neural network (CNN). Performance (training speed, precision, etc.) of BPNN, RBFNN and CNN are expected to be improved. Main works are as follows: Firstly, based on existing BPNN and RBFNN, Wavelet neural network (WNN) is implemented in order to get better performance for further improving CNN. WNN adopts the network structure of BPNN in order to get faster training speed. WNN adopts the wavelet function as an activation function, whose form is similar to the radial basis function of RBFNN, in order to solve the local minimum problem. Secondly, WNN-based Convolutional wavelet neural network (CWNN) method is proposed, in which the fully connected layers (FCL) of CNN is replaced by WNN. Thirdly, comparative simulations based on MNIST and CIFAR-10 datasets among the discussed methods of BPNN, RBFNN, CNN and CWNN are implemented and analyzed. Fourthly, the wavelet-based Convolutional Neural Network (WCNN) is proposed, where the wavelet transformation is adopted as the activation function in Convolutional Pool Neural Network (CPNN) of CNN. Fifthly, simulations based on CWNN are implemented and analyzed on the MNIST dataset. Effects are as follows: Firstly, WNN can solve the problems of BPNN and RBFNN and have better performance. Secondly, the proposed CWNN can reduce the mean square error and the error rate of CNN, which means CWNN has better maximum precision than CNN. Thirdly, the proposed WCNN can reduce the mean square error and the error rate of CWNN, which means WCNN has better maximum precision than CWNN.
Read moreBrain tumor segmentation with deep convolutional symmetric neural network
Brain tumor segmentation with deep convolutional symmetric neural network