HRANet: Hierarchical region-aware network for crowd counting

In recent years, crowd counting has been widely used in computer vision. But on many issues like large scale variations, perspective distortions and complex backgrounds, it remains a challenging task. Previous methods usually adopted dilation convolution to enlarge the receptive field, these methods can generate poor density maps. To solve this problem, in this paper, we proposed a novel structure called Attention-based Contextual Convolution Network for crowd counting. ACCNet contains two components. The Contextual Convolution Network first gets the relative influence of different scale-aware feature and attention module to further enlarge the range of scales while getting the final crowd density map in the network. The key section of our network is the dense dilated module, this method can make the front and back dilation layers tightly connected to preserve information from changing scales. Other than that, attention mechanism also plays an important role in learning density maps from input images. To verify the feasibility of our method, we compare it against the state-of-the-art structure on four crowd counting datasets (ShanghaiTech Part_A, ShanghaiTech Part_B, WorldEXPO'10 and UCF_CC_50), which demonstrates that ACCNet is effective and robust for complex scenes.

Accurately estimate the crowd count from a still image with arbitrary perspective and arbitrary crowd density is one of the difficulties of crowd analysis in surveillance videos. Conventional methods are scene-specific and subject to occlusions. In this paper, we propose a Multi-task Multi-column Convolutional Neural Network (MMCNN) architecture for crowd counting and crowd density estimation in still images of surveillance scenes. The MMCNN architecture is an end-to-end system which is robust for images with different perspective and different crowd density. By promoting MCNN with \(3\times 3\) filter, the MMCNN could utilize local spatial features from each column. Furthermore, the ground truth density map is generated based on Perspective-Adaptive Gaussian kernels which can better represent the heads of pedestrians. Finally, we use an iterative switching process in our deep crowd model to alternatively optimize the crowd density map estimation task and crowd counting task. We conduct experiments on the WorldExpo’10 dataset and our method achieves better results.

Density-aware and background-aware network for crowd counting via multi-task learning

In this paper, we focus on the task of estimating crowd count and high-quality crowd density maps. Among crowd counting methods, crowd density map estimation is especially promising because it preserves spatial information which makes it useful for both counting and localization (detection and tracking). Convolutional neural networks have enabled significant progress in crowd density estimation recently, but there are still open questions regarding suitable architectures. We revisit CNNs design and point out key adaptations, enabling plain a signal column CNNs to obtain high resolution and high-quality density maps on all major dense crowd counting datasets. The regular deep supervision utilizes the general ground truth to guide intermediate predictions. Instead, we build hierarchical supervisory signals with additional multi-scale labels to consider the diversities in deep neural networks. We begin by obtaining multi-scale labels based on different Gaussian kernels. These multi-scale labels can be seen as diverse representations in the supervision and can achieve high performance for better quality crowd density map estimation. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on the ShanghaiTech, UCF_CC_50 and UCSD datasets.

The task of crowd counting and density map estimation is riddled with many challenges, such as occlusions, non-uniform density, intra-scene and inter-scene variations in scale and perspective. Due to the development of deep learning and large crowd datasets in recent years, most crowd counting methods have achieved notable success. This paper aims to solve crowd density estimation problem for both sparse and dense conditions. To this end, we make two contributions: (1) a network named Patch Scale Discriminant Regression Network (PSDR). Given an input crowd image, it divides the image into patches and sends image patches of different density levels into different regression networks to get the corresponding density maps. It combines all patch density maps to predict the entire density map as the output. (2) A person classification activation map (CAM) method. CAM provides person location information and guides the generation of the entire density map in the final stage. Experiment confirms that CAM allows PSDR to gain another round of performance boost. For instance, on the SmartCity dataset, we achieve (8.6–1.1) MAE and (11.6–1.4) MSE. Our method combining above two methods performs better than state-of-the-art methods.

A multi-scale and multi-level feature aggregation network for crowd counting

As a research hotspot of computer vision, crowd counting methods have achieved success in natural images. But crowd counting in aerial images are rarely explored, and existing methods do not perform well because of the higher resolution, smaller object scale and more complex scene. Therefore, this paper proposes a lightweight dual-task network (LDNet) for crowd counting, which only uses bifurcated structure to overcome these new challenges in aerial images without complicated pipelines. To realize this, a complete but efficient Guidance Branch is proposed to assist Counting Branch in fitting crowd distribution. Furthermore, a scene attention mechanism is used to consider the complex scene information, which are never considered by existing methods. Our LD-Net outperforms existing methods on aerial crowd counting dataset (Visdrone), and gets better or comparable results on natural crowd counting datasets (UCF_CC_50, UCF_QNRF, ShanghaiTech Part A).

Video-based crowd counting and density estimation (CCDE) is vital for crowd monitoring. The existing solutions lack in addressing issues like cluttered background and scale variation in crowd videos. To this end, a multiscale head attention-guided multiscale density maps fusion for video-based CCDE via multi-attention Spatial-Temporal CNN (MHAMD-MST-CNN) is proposed. The MHAMD-MST-CNN has three modules: a multi attention spatial stream (MASS), a multi attention temporal stream (MATS), and a final density map generation (FDMG) module. The spatial head attention modules (SHAMs) and temporal head attention modules (THAMs) are designed to eliminate the background influence from the MASS and the MATS, respectively, by mapping the multiscale spatial or temporal features to head maps. The multiscale de-backgrounded features are utilised by the density map generation (DMG) modules to generate multiscale density maps to deal with scale variation due to perspective distortion. The multiscale density maps are fused and fed into the FDMG module to obtain the final crowd density map. The MHAMD-MST-CNN has been trained and validated on three publicly available benchmark datasets: the Venice, the Mall, and the UCSD. The MHAMD-MST-CNN provides competitive results as compared with the state-of-the-arts in terms of mean absolute error (MAE) and root mean squared error (RMSE).

We address the problem of image-based crowd counting. In particular, we propose a new problem called <i>unlabeled scene-adaptive crowd counting</i>. Given a new target scene, we would like to have a crowd counting model specifically adapted to this particular scene based on the target data that capture some information about the new scene. In this paper, we propose to use one or more unlabeled images from the target scene to perform the adaptation. In comparison with the existing problem setups (e.g. fully supervised), our proposed problem setup is closer to the real-world applications of crowd counting systems. We introduce a novel <monospace>AdaCrowd</monospace> framework to solve this problem. Our framework consists of a crowd counting network and a guiding network. The guiding network predicts some parameters in the crowd counting network based on the unlabeled images from a particular scene. This allows our model to adapt to different target scenes. The experimental results on several challenging benchmark datasets demonstrate the effectiveness of our proposed approach compared with other alternative methods. Code is available at <uri>https://github.com/maheshkkumar/adacrowd</uri>

Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a weakly-supervised manner. Based on the derived segmentation, we design a crowd-aware domain adaptation mechanism consisting of two crowd-aware adaptation modules, i.e., Crowd Region Transfer (CRT) and Crowd Density Alignment (CDA). The CRT module is designed to guide crowd features transfer across domains beyond background distractions. The CDA module dedicates to regularising target-domain crowd density generation by its own crowd density distribution. Our method outperforms previous approaches consistently in the widely-used adaptation scenarios.

Counting the number of people in public places has received much attention, and researchers have devoted much effort to the task. However, the existing crowd counting approaches are mainly trained and tested in similar scenarios. The performance of crowd counting approaches degrades sharply when the test scenarios of the models are of different types from its training scenes. In practice, the crowd scenes are highly variable, and the lack of cross-scene capability could seriously limit the application of the existing approaches. We attribute the improvement in cross-scene crowd counting capability to the necessity of accommodating large changes in the scale of individuals and the ability to suppress the interference of cluttered backgrounds. To this end, we propose a multi-scale feature fusion network (MFFNet) with cascaded supervision. The multi-scale features extracted from the crowd images are upsampled and then combined into several feature blocks, followed by convolution and deconvolution operations on the feature blocks to derive feature matrices of different resolutions. The feature matrices are fused from bottom to top. In the process of feature fusion, the crowd density maps corresponding to the feature matrices of different resolutions are predicted separately. We devise cascaded supervision to synchronously optimize the network of different resolution density map prediction during training. The cross-scene crowd counting experiments are conducted on four types of scenes: ShanghaiTech Part_A (SHT A) with high-density crowd scenes and small-scale individuals, ShanghaiTech Part_B (SHT B) with sparse crowd distribution and medium-scale individuals, UCF_CC_50 dataset with extremely dense scenes and tiny scale individuals, and UCF-QNRF dataset with extreme variations. MFFNet exhibits the strongest scene adaptability relative to the state-of-the-art approaches, with an average decrease of 17.1% and 8.4% in mean absolute error (MAE) and mean square error (mse), respectively. The contributions of different components in our method are verified in the ablation study using the devised evaluation metrics. Our implementation will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/learnsharing/MFFNet</uri> .

This paper aims to utilize the deep learning architecture to break through the limitations of camera perspective, image background, uneven crowd density distribution and pedestrian occlusion to estimate crowd density accurately. In this paper, we proposed a new neural network called Deep Scale-Adaptive Convolutional Neural Network (DSA-CNN), which can convert a single crowd image to density map for crowd counting directly. For a crowd image with any size and resolution, our algorithm can output the density map of the crowd image by end-to-end method and finally estimate the number of the crowd in the image. The proposed DSA-CNN consists of two parts: the seven layers CNN network structure and DSA modules. In order to ensure the proposed method is robust to camera perspective effect, DSA-CNN has adopted different sizes of filters in the network and combines them ingeniously. In order to reduce the depth of the data to increase the speed of training, the proposed method utilized 1 × 1 filter in DSA module. To validate the effectiveness of the proposed model, we conducted comparative experiments on four popular public datasets (ShanghiTech dataset, UCF_CC_50 dataset, WorldExpo’10 dataset and UCSD dataset). We compare the proposed method with other well-known algorithms on the MAE and MSE indicators, such as MCNN, Switching-CNN, CSRNet, CP-CNN and Cascaded-MTL. Experimental results show that the proposed method has excellent performance. In addition, we found that the proposed model is easily trained, which further increases the usability of the proposed model.

Counting challenging crowds robustly using a multi-column multi-task convolutional neural network

Crowd counting plays an important role in crowd analysis and monitoring. To this end, we propose a novel method called Adaptive Weighted Crowd Receptive Field Network (AWRFN) for crowd counting to estimate the number of people and the spatial distribution of input crowd images. The proposed AWRFN is composed of four modules: backbone, crowd receptive field block (CRFB), recurrent block (RB), and channel attention block (CAB). Backbone utilizes the first ten layers of VGG16 to extract base features of input images. CRFB is a multi-branch architecture simulating a real human visual system for further obtaining refined and discriminative crowd features. RB generates strong semantic and global information by recurrently stacking convolutional layers with the same parameters. CAB outputs appropriate weights to supervise each channel of the feature maps output from CRFB, which uses the outputs of RB as guidance. Different from previous works using Euclidean Loss, we employ L1_Smooth Loss to train our network in an end-to-end fashion. To demonstrate the effectiveness of our proposed method, we implement AWRFN on two representative datasets including the ShanghaiTech dataset and the UCF_CC_50 dataset. The experimental results prove that our method is both effective and robust compared with the state-of-the-art approaches.

Crowd counting in still images is a challenging problem in practice due to huge crowd-density variations, large perspective changes, severe occlusion, and variable lighting conditions. The state-of-the-art patch rescaling module (PRM) based approaches prove to be very effective in improving the crowd counting performance. However, the PRM module requires an additional and compromising crowd-density classification process. To address these issues and challenges, the paper proposes a new multi-resolution fusion based end-to-end crowd counting network. It employs three deep-layers based columns/branches, each catering the respective crowd-density scale. These columns regularly fuse (share) the information with each other. The network is divided into three phases with each phase containing one or more columns. Three input priors are introduced to serve as an efficient and effective alternative to the PRM module, without requiring any additional classification operations. Along with the final crowd count regression head, the network also contains three auxiliary crowd estimation regression heads, which are strategically placed at each phase end to boost the overall performance. Comprehensive experiments on three benchmark datasets demonstrate that the proposed approach outperforms all the state-of-the-art models under the RMSE evaluation metric. The proposed approach also has better generalization capability with the best results during the cross-dataset experiments.