ACCNet: Attention-based Contextual Convolutional Network for Crowd Counting

As a research hotspot of computer vision, crowd counting methods have achieved success in natural images. But crowd counting in aerial images are rarely explored, and existing methods do not perform well because of the higher resolution, smaller object scale and more complex scene. Therefore, this paper proposes a lightweight dual-task network (LDNet) for crowd counting, which only uses bifurcated structure to overcome these new challenges in aerial images without complicated pipelines. To realize this, a complete but efficient Guidance Branch is proposed to assist Counting Branch in fitting crowd distribution. Furthermore, a scene attention mechanism is used to consider the complex scene information, which are never considered by existing methods. Our LD-Net outperforms existing methods on aerial crowd counting dataset (Visdrone), and gets better or comparable results on natural crowd counting datasets (UCF_CC_50, UCF_QNRF, ShanghaiTech Part A).

Dense crowd counting aims to predict thousands of human instances from an image, by calculating integrals of a density map over image pixels. Existing approaches mainly suffer from the extreme density variances. Such density pattern shift poses challenges even for multi-scale model ensembling. In this paper, we propose a simple yet effective approach to tackle this problem. First, a patch-level density map is extracted by a density estimation model and further grouped into several density levels which are determined over full datasets. Second, each patch density map is automatically normalized by an online center learning strategy with a multipolar center loss. Such a design can significantly condense the density distribution into several clusters, and enable that the density variance can be learned by a single model. Extensive experiments demonstrate the superiority of the proposed method. Our work outperforms the state-of-the-art by 4.2%, 14.3%, 27.1% and 20.1% in MAE, on ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF-QNRF datasets, respectively.

Counting the number of people in public places has received much attention, and researchers have devoted much effort to the task. However, the existing crowd counting approaches are mainly trained and tested in similar scenarios. The performance of crowd counting approaches degrades sharply when the test scenarios of the models are of different types from its training scenes. In practice, the crowd scenes are highly variable, and the lack of cross-scene capability could seriously limit the application of the existing approaches. We attribute the improvement in cross-scene crowd counting capability to the necessity of accommodating large changes in the scale of individuals and the ability to suppress the interference of cluttered backgrounds. To this end, we propose a multi-scale feature fusion network (MFFNet) with cascaded supervision. The multi-scale features extracted from the crowd images are upsampled and then combined into several feature blocks, followed by convolution and deconvolution operations on the feature blocks to derive feature matrices of different resolutions. The feature matrices are fused from bottom to top. In the process of feature fusion, the crowd density maps corresponding to the feature matrices of different resolutions are predicted separately. We devise cascaded supervision to synchronously optimize the network of different resolution density map prediction during training. The cross-scene crowd counting experiments are conducted on four types of scenes: ShanghaiTech Part_A (SHT A) with high-density crowd scenes and small-scale individuals, ShanghaiTech Part_B (SHT B) with sparse crowd distribution and medium-scale individuals, UCF_CC_50 dataset with extremely dense scenes and tiny scale individuals, and UCF-QNRF dataset with extreme variations. MFFNet exhibits the strongest scene adaptability relative to the state-of-the-art approaches, with an average decrease of 17.1% and 8.4% in mean absolute error (MAE) and mean square error (mse), respectively. The contributions of different components in our method are verified in the ablation study using the devised evaluation metrics. Our implementation will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/learnsharing/MFFNet</uri> .

Previous works of crowd counting prepossess the given label and convert it into a density map or count map used for learning. However, we revealed that density maps tend to have severe errors due to faulty occlusions, head size variation, and head shape variation. Directly learning the density map will often result in fatal over-fitting. On the other hand, Count-map did not fully utilize the detailed information of the image. These unsatisfactory preprocessing lead to the performance bottleneck despite recent advances in network architecture. To solve these problems, in this paper, we discovered that the distribution of errors throughout the density map is not uniform. Moreover, it is correlated with the distance to the nearest annotation point. Inspired by this finding, we introduce Fine-Grained Adaptive Losses to learn the density map differently in different regions of the density map. While our method is simple, it dictates that we should endeavor to obtain more supervision from the density map. Our effort subverts the traditional use of density maps and opens up a new vision for future counting research. Extensive experiments demonstrate that our approach significantly outperforms standard methods in crowd-counting datasets.

Automated people counting in crowd scenes is challenging due to large variations in scale, density, and background clutter. To tackle them, we propose a novel cross-level parallel network (CLPNet) by extracting multiple low-level features from VGG16 and fusing them with specific scale aggregation modules in the high-level stage. To deal with scale variation, we design five different aggregation modules for multiscale fusion. Furthermore, the ground truth is processed skillfully to eliminate the mismatches caused by the scale variation between heads and density maps. To cope with background clutter, cross-level feature fusion is implemented. Higher-level semantic information could effectively separate head from background and regain the lost low-level detailed information. To address the variation of density, we design a parallel network, in which two separate channels focus on different density-level estimation, and attain more accurate counting results. Finally, we evaluate the proposed CLPNet on four representative crowd counting datasets, i.e., ShanghaiTech, UCF_CC_50, WorldExpo'10, and UCF_QNRF. The experimental results demonstrate that with the cross-level and multiscale structure CLPNet achieves superior performance compared with the state-of-the-art crowd counting methods.

Density-aware and background-aware network for crowd counting via multi-task learning

Learning a deep network with cross-hierarchy aggregation for crowd counting

Crowd counting is an important task in the surveillance camera system. The methods of crowd counting have changed from counting each object in the scene to estimating the density map and the count will be reached by integrating the entire density map. The previous methods used multi-column convolutional neural networks (CNN) with different sizes of filters to describe objects of different scales. To increase the performance of the multi-column model, some methods suggest adding a classifier to select a single-column CNN that gives the expecting results. However, having multiple columns and filters will complicate the model and trained model with limited crowd data will estimate bad density maps. To solve this problem, we propose the "Single-Column CNN" (SCC) method which uses a single-column CNN with only one size of filters (3x3) and replaces normal convolution with different sizes of filters by the dilated convolution. Our approach makes the density map estimation model become a simple but strong CNN. Experiments show that our method achieves significant improvements compared to the method using multi-column CNN in the ShanghaiTech dataset.

Crowd counting has been widely studied by deep learning in recent years. However, due to scale variation caused by perspective distortion, crowd counting is still a challenging task. In this paper, we propose a Densely Connected Multi-scale Pyramid Network (DMPNet) for count estimation and the generation of high-quality density maps. The key component of our network is the Multi-scale Pyramid Network (MPN), which can extract multi-scale features of the crowd effectively while keeping the resolution of the input feature map and the number of channels unchanged. To increase the information transfer between the network layer, we used dense connections to connect multiple MPNs. In addition, we also designed a novel loss function, which can help our model achieve better convergence. To evaluate our method, we conducted extensive experiments on three challenging benchmark crowd counting datasets. Experimental results show that compared with the state-of-the-art algorithms, DMPNet performs well in both parameters and results. The code is available at: https://github.com/lpfworld/DMPNet.

Counting the number of person has received much attention in recent years. Most of the existing crowd counting methods adopted density map regression pipeline, which formulates the crowd counting problem to two fragmented part: density map regression and integration of the overall counting. To solve this problem, this paper presents a multi-task deep learning scheme to enhance the counting performance. More specifically, we firstly build a multi-scale deep convolutional neural network, based on combining the feature maps of conv layers with different filters, to solve the multi-scale problem in crowd counting. Secondly, we develop the multi-task structure that can simultaneously learn the density map and the global counting. Experiments on large scale crowd counting datasets, Shanghaitech and WorldExpo10, demonstrate that the proposed method achieves much reduction in counting error respectively

Crowd counting on still images is very challenging due to heavy occlusions and scale variations. In this paper, we aim to develop a method that can accurately estimate the crowd count from a still image. Recently, convolutional neural networks have been shown effective in many computer vision tasks including crowd counting. To this end, we propose a fully convolutional network (FCN) architecture to map the input image of arbitrary size or resolution to its density map. In order to address the perspective and scale variation issues, Inception-like modules with multiple kernel size filters are used to capture multi-scale features, which is necessary for higher crowd counting performance. We test our model on challenging ShanghaiTech dataset, the results show that our method outperforms the state-of-the-art methods.

Aiming to tackle the most intractable problems of scale variation and complex backgrounds in crowd counting, we present an innovative framework called Hierarchical Region-Aware Network (HRANet) for crowd counting in this paper, which can better focus on crowd regions to accurately predict crowd density. In our implementation, first, we design a Region-Aware Module (RAM) to capture the internal differences within different regions of the feature map, thus adaptively extracting contextual features within different regions. Furthermore, we propose a Region Recalibration Module (RRM) which adopts a novel region-aware attention mechanism (RAAM) to further recalibrate the feature weights of different regions. By the integration of the above two modules, the influence of background regions can be effectively suppressed. Besides, considering the local correlations within different regions of the crowd density map, a Region Awareness Loss (RAL) is designed to reduce false identification while producing the locally consistent density map. Extensive experiments on five challenging datasets demonstrate that the proposed method significantly outperforms existing methods in terms of counting accuracy and quality of the generated density map. In addition, a series of specific experiments in crowd gathering scenes indicate that our method can be practically applied to crowd localization.

The most challenging issue of crowd counting is scale change of the crowd. Therefore, an improved fully convolutional deep neural network model in end-to-end manner, which fully considers multi-scale features extracted and feature fusion, makes crowd counting more accurate. On the one hand, the Shuffle-net module is introduced into the model, using its feature fusion and channel shuffling features to enhance the ability to extract features; on the other hand, the improved multi-channel Inception module can not only merge the multi-scale features in the population, enlarge the receptive field during feature learning, but also improve the accuracy of model prediction. Aiming to verify the utility of the method proposed, many experiments on multiple public datasets are conducted, including ShanghaiTech part A, ShanghaiTech part B, UCSD, and Mall. From the experimental results, the improved method performs better than the existing methods.

In this paper, we focus on the task of estimating crowd count and high-quality crowd density maps. Among crowd counting methods, crowd density map estimation is especially promising because it preserves spatial information which makes it useful for both counting and localization (detection and tracking). Convolutional neural networks have enabled significant progress in crowd density estimation recently, but there are still open questions regarding suitable architectures. We revisit CNNs design and point out key adaptations, enabling plain a signal column CNNs to obtain high resolution and high-quality density maps on all major dense crowd counting datasets. The regular deep supervision utilizes the general ground truth to guide intermediate predictions. Instead, we build hierarchical supervisory signals with additional multi-scale labels to consider the diversities in deep neural networks. We begin by obtaining multi-scale labels based on different Gaussian kernels. These multi-scale labels can be seen as diverse representations in the supervision and can achieve high performance for better quality crowd density map estimation. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on the ShanghaiTech, UCF_CC_50 and UCSD datasets.

Crowd counting is a very challenging task in crowded scenes due to heavy occlusions, appearance variations and perspective distortions. Current crowd counting methods typically operate on an image patch level with overlaps, then sum over the patches to get the final count. In this paper, we propose an end-to-end convolutional neural network (CNN) architecture that takes a whole image as its input and directly outputs the counting result. While making use of sharing computations over overlapping regions, our method takes advantages of contextual information when predicting both local and global count. In particular, we first feed the image to a pre-trained CNN to get a set of high level features. Then the features are mapped to local counting numbers using recurrent network layers with memory cells. We perform the experiments on several challenging crowd counting datasets, which achieve the state-of-the-art results and demonstrate the effectiveness of our method.