- Research Article
13
- 10.1016/j.neucom.2021.03.128
Adversarial scale-adaptive neural network for crowd counting
- Apr 13, 2021
- Neurocomputing
- Xinyue Chen + 4 more +4
Adversarial scale-adaptive neural network for crowd counting
An efficient semi-supervised manifold embedding for crowd counting
Adversarial scale-adaptive neural network for crowd counting
Adversarial scale-adaptive neural network for crowd counting
AdaCrowd: Unlabeled Scene Adaptation for Crowd Counting
We address the problem of image-based crowd counting. In particular, we propose a new problem called <i>unlabeled scene-adaptive crowd counting</i>. Given a new target scene, we would like to have a crowd counting model specifically adapted to this particular scene based on the target data that capture some information about the new scene. In this paper, we propose to use one or more unlabeled images from the target scene to perform the adaptation. In comparison with the existing problem setups (e.g. fully supervised), our proposed problem setup is closer to the real-world applications of crowd counting systems. We introduce a novel <monospace>AdaCrowd</monospace> framework to solve this problem. Our framework consists of a crowd counting network and a guiding network. The guiding network predicts some parameters in the crowd counting network based on the unlabeled images from a particular scene. This allows our model to adapt to different target scenes. The experimental results on several challenging benchmark datasets demonstrate the effectiveness of our proposed approach compared with other alternative methods. Code is available at <uri>https://github.com/maheshkkumar/adacrowd</uri>
Read moreAccurate Deep Net Crowd Counting for Smart IoT Video acquisition devices
A novel deep neural network is proposed, for accurate and robust crowd counting. Crowd counting is a complex task, as it strongly depends on the deployed camera characteristics and, above all, the scene perspective. Crowd counting is essential in security applications where Internet of Things (IoT) cameras are deployed to help with crowd management tasks. The complexity of a scene varies greatly, and a medium to large scale security system based on IoT cameras must cater for changes in perspective and how people appear from different vantage points. To address this, our deep architecture extracts multi-scale features with a pyramid contextual module to provide long-range contextual information and enlarge the receptive field. Experiments were run on three major crowd counting datasets, to test our proposed method. Results demonstrate our method supersedes the performance of state-of-the-art methods.
Read moreDMPNet: densely connected multi-scale pyramid networks for crowd counting
Crowd counting has been widely studied by deep learning in recent years. However, due to scale variation caused by perspective distortion, crowd counting is still a challenging task. In this paper, we propose a Densely Connected Multi-scale Pyramid Network (DMPNet) for count estimation and the generation of high-quality density maps. The key component of our network is the Multi-scale Pyramid Network (MPN), which can extract multi-scale features of the crowd effectively while keeping the resolution of the input feature map and the number of channels unchanged. To increase the information transfer between the network layer, we used dense connections to connect multiple MPNs. In addition, we also designed a novel loss function, which can help our model achieve better convergence. To evaluate our method, we conducted extensive experiments on three challenging benchmark crowd counting datasets. Experimental results show that compared with the state-of-the-art algorithms, DMPNet performs well in both parameters and results. The code is available at: https://github.com/lpfworld/DMPNet.
Read moreEnd-to-end crowd counting via joint learning local and global count
Crowd counting is a very challenging task in crowded scenes due to heavy occlusions, appearance variations and perspective distortions. Current crowd counting methods typically operate on an image patch level with overlaps, then sum over the patches to get the final count. In this paper, we propose an end-to-end convolutional neural network (CNN) architecture that takes a whole image as its input and directly outputs the counting result. While making use of sharing computations over overlapping regions, our method takes advantages of contextual information when predicting both local and global count. In particular, we first feed the image to a pre-trained CNN to get a set of high level features. Then the features are mapped to local counting numbers using recurrent network layers with memory cells. We perform the experiments on several challenging crowd counting datasets, which achieve the state-of-the-art results and demonstrate the effectiveness of our method.
Read moreScene-adaptive accurate and fast vertical crowd counting via joint using depth and color information
Reliable and real-time crowd counting is one of the most important tasks in intelligent visual surveillance systems. Most previous works only count passing people based on color information. Owing to the restrictions of color information influences themselves for multimedia processing, they will be affected inevitably by the unpredictable complex environments (e.g. illumination, occlusion, and shadow). To overcome this bottleneck, we propose a new algorithm by multimodal joint information processing for crowd counting. In our method, we use color and depth information together with a ordinary depth camera (e.g. Microsoft Kinect). Specifically, we first detect each head of the passing or still person in the surveillance region with adaptive modulation ability to varying scenes on depth information. Then, we track and count each detected head on color information. The characteristic advantage of our algorithm is that it is scene adaptive, which means the algorithm can be applied into all kinds of different scenes directly without additional conditions. Based on the proposed approach, we have built a practical system for robust and fast crowd counting facing complicated scenes. Extensive experimental results show the effectiveness of our proposed method.
Read moreImproving Crowd Counting with Multi-Task Multi-Scale Convolutional Neural Network
Counting the number of person has received much attention in recent years. Most of the existing crowd counting methods adopted density map regression pipeline, which formulates the crowd counting problem to two fragmented part: density map regression and integration of the overall counting. To solve this problem, this paper presents a multi-task deep learning scheme to enhance the counting performance. More specifically, we firstly build a multi-scale deep convolutional neural network, based on combining the feature maps of conv layers with different filters, to solve the multi-scale problem in crowd counting. Secondly, we develop the multi-task structure that can simultaneously learn the density map and the global counting. Experiments on large scale crowd counting datasets, Shanghaitech and WorldExpo10, demonstrate that the proposed method achieves much reduction in counting error respectively
Read moreA multi-scale and multi-level feature aggregation network for crowd counting
A multi-scale and multi-level feature aggregation network for crowd counting
Residual Regression With Semantic Prior for Crowd Counting
Crowd counting is a challenging task due to factors such as large variations in crowdedness and severe occlusions. Although recent deep learning based counting algorithms have achieved a great progress, the correlation knowledge among samples and the semantic prior have not yet been fully exploited. In this paper, a residual regression framework is proposed for crowd counting utilizing the correlation information among samples. By incorporating such information into our network, we discover that more intrinsic characteristics can be learned by the network which thus generalizes better to unseen scenarios. Besides, we show how to effectively leverage the semantic prior to improve the performance of crowd counting. We also observe that the adversarial loss can be used to improve the quality of predicted density maps, thus leading to an improvement in crowd counting. Experiments on public datasets demonstrate the effectiveness and generalization ability of the proposed method. © 2019 IEEE.
Read moreResearch on Crowd Counting Network Model based on Fusion of Shuffle-net and Inception Model
The most challenging issue of crowd counting is scale change of the crowd. Therefore, an improved fully convolutional deep neural network model in end-to-end manner, which fully considers multi-scale features extracted and feature fusion, makes crowd counting more accurate. On the one hand, the Shuffle-net module is introduced into the model, using its feature fusion and channel shuffling features to enhance the ability to extract features; on the other hand, the improved multi-channel Inception module can not only merge the multi-scale features in the population, enlarge the receptive field during feature learning, but also improve the accuracy of model prediction. Aiming to verify the utility of the method proposed, many experiments on multiple public datasets are conducted, including ShanghaiTech part A, ShanghaiTech part B, UCSD, and Mall. From the experimental results, the improved method performs better than the existing methods.
Read moreLeveraging Unlabeled Data for Crowd Counting by Learning to Rank
We propose a novel crowd counting approach that leverages abundantly available unlabeled crowd imagery in a learning-to-rank framework. To induce a ranking of cropped images, we use the observation that any sub-image of a crowded scene image is guaranteed to contain the same number or fewer persons than the super-image. This allows us to address the problem of limited size of existing datasets for crowd counting. We collect two crowd scene datasets from Google using keyword searches and query-by-example image retrieval, respectively. We demonstrate how to efficiently learn from these unlabeled datasets by incorporating learning-to-rank in a multi-task network which simultaneously ranks images and estimates crowd density maps. Experiments on two of the most challenging crowd counting datasets show that our approach obtains state-of-the-art results.
Read moreFast Moving Crowd Counting for Unconstrained Videos
It is important to estimate the number of people in fast moving crowd scenarios for the surveillance systems. Regression-based techniques achieved promising results for counting the number of people in crowded scenes. However, appearance features used in the most existing techniques are not able to mirror the motion state of crowds. The motion state of crowds is important to solve the counting problem of fast moving crowds, since it is decided by the amount of people. In this study, we propose a novel method to address this problem from three perspectives: (1) train a crowd counting estimation model suited to all of the crowded scenes; (2) combine motion states with multiappearance features for crowd counting; and (3) count fast moving crowds in unconstrained videos. These ideas are implemented in a fast object segment framework, which can segment fast moving crowds in the unconstrained videos. Extensive experiments validate the effectiveness of our proposed method.
Read moreCrowd counting via Multi-Scale Adversarial Convolutional Neural Networks
The purpose of crowd counting is to estimate the number of pedestrians in crowd images. Crowd counting or density estimation is an extremely challenging task in computer vision, due to large scale variations and dense scene. Current methods solve these issues by compounding multi-scale Convolutional Neural Network with different receptive fields. In this paper, a novel end-to-end architecture based on Multi-Scale Adversarial Convolutional Neural Network (MSA-CNN) is proposed to generate crowd density and estimate the amount of crowd. Firstly, a multi-scale network is used to extract the globally relevant features in the crowd image, and then fractionally-strided convolutional layers are designed for up-sampling the output to recover the loss of crucial details caused by the earlier max pooling layers. An adversarial loss is directly employed to shrink the estimated value into the realistic subspace to reduce the blurring effect of density estimation. Joint training is performed in an end-to-end fashion using a combination of Adversarial loss and Euclidean loss. The two losses are integrated via a joint training scheme to improve density estimation performance.We conduct some extensive experiments on available datasets to show the significant improvements and supremacy of the proposed approach over the available state-of-the-art approaches.
Read moreA cross-modal crowd counting method combining CNN and cross-modal transformer
A cross-modal crowd counting method combining CNN and cross-modal transformer
A Deep Learning Approach for Crowd Counting in Highly Congested Scene
With the rapid progress of deep convolutional neural networks, several applications of crowd counting have been proposed and explored in the literature. In congested scene monitoring, a variety of crowd density estimating approaches has been developed. The understanding of highly congested scenes for crowd counting during Muslim gatherings of Hajj and Umrah is a challenging task, as a large number of individuals stand nearby and, it is hard for detection techniques to recognize them, as the crowd can vary from low density to high density. To deal with such highly congested scenes, we have proposed the Congested Scene Crowd Counting Network (CSCC-Net) using VGG-16 as a core network with its first ten layers due to its strong and robust transfer learning rate. A hole dilated convolutional neural network is used at the back end to widen the relevant field to extract a large range of information from the image without losing its original resolution. The dilated convolution neural network is mainly chosen to expand the kernel size without changing other parameters. Moreover, several loss functions have been applied to strengthen the evaluation accuracy of the model. Finally, the entire experiments have been evaluated using prominent data sets namely, ShanghaiTech parts A, B, UCF_CC_50, and UCF_QNRF. Our model has achieved remarkable results i.e., 68.0 and 9.0 MAE on ShanghaiTech parts A, B, 199.1 MAE on UCF_CC_50, and 99.8 on UCF_QNRF data sets respectively.
Read more