Crowd counting via Multi-Scale Adversarial Convolutional Neural Networks

Counting the number of people in public places has received much attention, and researchers have devoted much effort to the task. However, the existing crowd counting approaches are mainly trained and tested in similar scenarios. The performance of crowd counting approaches degrades sharply when the test scenarios of the models are of different types from its training scenes. In practice, the crowd scenes are highly variable, and the lack of cross-scene capability could seriously limit the application of the existing approaches. We attribute the improvement in cross-scene crowd counting capability to the necessity of accommodating large changes in the scale of individuals and the ability to suppress the interference of cluttered backgrounds. To this end, we propose a multi-scale feature fusion network (MFFNet) with cascaded supervision. The multi-scale features extracted from the crowd images are upsampled and then combined into several feature blocks, followed by convolution and deconvolution operations on the feature blocks to derive feature matrices of different resolutions. The feature matrices are fused from bottom to top. In the process of feature fusion, the crowd density maps corresponding to the feature matrices of different resolutions are predicted separately. We devise cascaded supervision to synchronously optimize the network of different resolution density map prediction during training. The cross-scene crowd counting experiments are conducted on four types of scenes: ShanghaiTech Part_A (SHT A) with high-density crowd scenes and small-scale individuals, ShanghaiTech Part_B (SHT B) with sparse crowd distribution and medium-scale individuals, UCF_CC_50 dataset with extremely dense scenes and tiny scale individuals, and UCF-QNRF dataset with extreme variations. MFFNet exhibits the strongest scene adaptability relative to the state-of-the-art approaches, with an average decrease of 17.1% and 8.4% in mean absolute error (MAE) and mean square error (mse), respectively. The contributions of different components in our method are verified in the ablation study using the devised evaluation metrics. Our implementation will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/learnsharing/MFFNet</uri> .

High-density crowd events like public concerts, sporting events, or religious festivals represent significant safety challenges due to high crowd density. Methods of monitoring crowds using manual observation or passive surveillance often don't provide real-time information. Therefore, we present a real-time crowd density estimation solution that utilizes YOLOv8 for people detection and CSRNet for density estimation. The crowd density estimation system utilizes live video feeds from direct surveillance cameras/CCTV or drone footage. The system will assess the crowd density in four levels across four distinct areas: Low, Medium, High, or Critical crowd density. The density estimator has a web-based dashboard that provides real-time analytics reports, heatmap density estimates, and historical records, which can assist in making quick, informed decisions and assessments following an event. The system is validated on benchmark datasets and real-world video streams with 95.3% detection accuracy, 7.4 MAE in crowd counting and 28 FPS processing with off-the-shelf GPU hardware. The results show high accuracy with low latency, making it feasible for real-world applications for large-scale events. The main contributions of the work include using YOLOv8 integrated with CSRNet to jointly detect and estimate crowds, developing a real-time dashboard to provide transparent crowd analytics, and system validation with quantitative metrics and real-world evidence.

Counting the number of person has received much attention in recent years. Most of the existing crowd counting methods adopted density map regression pipeline, which formulates the crowd counting problem to two fragmented part: density map regression and integration of the overall counting. To solve this problem, this paper presents a multi-task deep learning scheme to enhance the counting performance. More specifically, we firstly build a multi-scale deep convolutional neural network, based on combining the feature maps of conv layers with different filters, to solve the multi-scale problem in crowd counting. Secondly, we develop the multi-task structure that can simultaneously learn the density map and the global counting. Experiments on large scale crowd counting datasets, Shanghaitech and WorldExpo10, demonstrate that the proposed method achieves much reduction in counting error respectively

Accurately estimate the crowd count from a still image with arbitrary perspective and arbitrary crowd density is one of the difficulties of crowd analysis in surveillance videos. Conventional methods are scene-specific and subject to occlusions. In this paper, we propose a Multi-task Multi-column Convolutional Neural Network (MMCNN) architecture for crowd counting and crowd density estimation in still images of surveillance scenes. The MMCNN architecture is an end-to-end system which is robust for images with different perspective and different crowd density. By promoting MCNN with \(3\times 3\) filter, the MMCNN could utilize local spatial features from each column. Furthermore, the ground truth density map is generated based on Perspective-Adaptive Gaussian kernels which can better represent the heads of pedestrians. Finally, we use an iterative switching process in our deep crowd model to alternatively optimize the crowd density map estimation task and crowd counting task. We conduct experiments on the WorldExpo’10 dataset and our method achieves better results.

Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a weakly-supervised manner. Based on the derived segmentation, we design a crowd-aware domain adaptation mechanism consisting of two crowd-aware adaptation modules, i.e., Crowd Region Transfer (CRT) and Crowd Density Alignment (CDA). The CRT module is designed to guide crowd features transfer across domains beyond background distractions. The CDA module dedicates to regularising target-domain crowd density generation by its own crowd density distribution. Our method outperforms previous approaches consistently in the widely-used adaptation scenarios.

This paper aims to utilize the deep learning architecture to break through the limitations of camera perspective, image background, uneven crowd density distribution and pedestrian occlusion to estimate crowd density accurately. In this paper, we proposed a new neural network called Deep Scale-Adaptive Convolutional Neural Network (DSA-CNN), which can convert a single crowd image to density map for crowd counting directly. For a crowd image with any size and resolution, our algorithm can output the density map of the crowd image by end-to-end method and finally estimate the number of the crowd in the image. The proposed DSA-CNN consists of two parts: the seven layers CNN network structure and DSA modules. In order to ensure the proposed method is robust to camera perspective effect, DSA-CNN has adopted different sizes of filters in the network and combines them ingeniously. In order to reduce the depth of the data to increase the speed of training, the proposed method utilized 1 × 1 filter in DSA module. To validate the effectiveness of the proposed model, we conducted comparative experiments on four popular public datasets (ShanghiTech dataset, UCF_CC_50 dataset, WorldExpo’10 dataset and UCSD dataset). We compare the proposed method with other well-known algorithms on the MAE and MSE indicators, such as MCNN, Switching-CNN, CSRNet, CP-CNN and Cascaded-MTL. Experimental results show that the proposed method has excellent performance. In addition, we found that the proposed model is easily trained, which further increases the usability of the proposed model.

Human beings perceive images through their properties, like colour, shape, size, and texture. Texture is a fertile source of information about the physical environment. Images of low density crowds tend to present coarse textures, while images of dense crowds tend to present fine textures. This paper describes a new technique for automatic estimation of crowd density, which is a part of the problem of automatic crowd monitoring, using texture information based on grey-level transition probabilities on digitised images. Crowd density feature vectors are extracted from such images and used by a self organising neural network which is responsible for the crowd density estimation. Results obtained respectively to the estimation of the number of people in a specific area of Liverpool Street Railway Station in London (UK) are presented.

The estimation of crowd density is a crucial area of study in computer vision because of its extensive use in intelligence collection, urban planning, and surveillance. The crowd density prediction that comes from a thorough analysis considers several factors, including inter-blocking in dense crowds, background elements, and individual appearance similarity. To monitor populous areas and avoid congestion, we are interested in using machine learning for crowd control in this research. We suggest using a Single Convolutional Neural Network with Three Layers (S-CNN3) model to estimate the crowd size and count the number of individuals in a scene. Next, a comparison study for density counting determines how well the suggested model performs in comparison to switched convolutional neural networks (SCNN) and convolutional neural networks with four layers (single CNN4). This study makes use of the Shanghai Tech dataset, which is regarded as the biggest database for crowd counting. With an average test accuracy of 99.88% and an average validation loss of 0.02 for crowd density estimate, the suggested model demonstrates remarkable efficacy and efficiency. These outcomes outperform the state-of-the-art models already in use.

A multi-scale and multi-level feature aggregation network for crowd counting

Crowd counting aims to estimate the number, density, and distribution of crowds in an image. While CNN-based crowd counting methods have been effective, head-scale variation and complex background remain two major challenges for crowd counting. Therefore, we propose a multiscale region calibration network called MRCNet to effectively address these challenges. To address the former challenge, we design a multiscale aware module that utilizes multi-branch dilated convolutional parallelism to obtain multiscale receptive fields and cope with drastic changes in head size. For the latter challenge, we design a regional calibration module that calibrates the attention weights of each region after obtaining the attention map to effectively handle challenges in complex contexts. Additionally, we improve the loss function by combining L2 loss and binary cross-entropy loss to help MRCNet achieve excellent results. Extensive experiments were conducted on three mainstream datasets to demonstrate the robustness and competitiveness of our approach.

In this paper, we focus on the task of estimating crowd count and high-quality crowd density maps. Among crowd counting methods, crowd density map estimation is especially promising because it preserves spatial information which makes it useful for both counting and localization (detection and tracking). Convolutional neural networks have enabled significant progress in crowd density estimation recently, but there are still open questions regarding suitable architectures. We revisit CNNs design and point out key adaptations, enabling plain a signal column CNNs to obtain high resolution and high-quality density maps on all major dense crowd counting datasets. The regular deep supervision utilizes the general ground truth to guide intermediate predictions. Instead, we build hierarchical supervisory signals with additional multi-scale labels to consider the diversities in deep neural networks. We begin by obtaining multi-scale labels based on different Gaussian kernels. These multi-scale labels can be seen as diverse representations in the supervision and can achieve high performance for better quality crowd density map estimation. Extensive experiments demonstrate that our approach achieves the state-of-the-art performance on the ShanghaiTech, UCF_CC_50 and UCSD datasets.

This paper aims to propose a convolutional neural network that can accurately estimate the density of pedestrians in crowd images, and analyze the primary factors that affect the performance of the network employed in crowd counting. In contrast to the multi-column convolutional neural network (MCNN) which is mainly used in previous works, the proposed network is designed by using the improved Inception-ResNet-A module in the manner of end-to-end and therefore it is convenient for training and easy to be employed. Experiments have been conducted on four datasets: Mall dataset, UCSD dataset, ShanghaiTech dataset and UCF_CC_SO dataset. Besides, the performances of different convolutional neural network (CNN) architectures have been compared to prove the effectiveness of the proposed network. The experimental results show that the proposed network is more accurate and efficient against the existing state-of-art methods, overcoming the problems generated by scale variation, pedestrian occlusion and appearance change. To the best of our knowledge, the proposed network is the first network to use the Inception-ResNet-A structure for crowd counting.

Crowd counting and precise localization in dense scenes are critical tasks in computer vision. Although the point-based prediction framework P2PNet eliminates complex post-processing steps through ensemble prediction, it still faces challenges such as insufficient multi-scale feature extraction and limited ability to perceive key information in complex scenes. To address these issues, this paper improves P2PNet by proposing a model based on a far-near network and a dynamic dual attention mechanism. Specifically, it introduces a far-near adaptive network (FN-Net) and a dynamic dual attention mechanism (DDAM). FN-Net explicitly models continuous scale variations caused by perspective effects by dividing the image into regions based on spatial position and assigning differentiated receptive fields. DDAM focuses on crowded areas through parallel spatial attention and channel attention sub-modules, selects discriminative features, and integrates a dynamic weighted fusion mechanism to adaptively combine the advantages of both attentions. Experiments show that our approach effectively enhances key features while suppressing background noise thus improves crowd counting accuracy.

During the past five years, there has been an increasing trend of weakly supervised crowd counting methods being developed since such methods just rely on count-level annotations and avoid a laborious labeling process. But, the existing weakly supervised methods usually fail to achieve comparable counting performance to the fully supervised methods. To improve the accuracy of crowd counting tasks, we propose to combine the convolutional neural network (CNN) and Transformer frameworks. Since CNN focuses on capturing local detail information and Transformer can effectively extract global context information, we believe that the combination of CNN and Transformer could learn more efficient feature representations for crowd images. Our proposed framework is named CrowdCCT (Crowd Counting via CNN and Transformer), and it is composed of a CNN feature extraction part, a Transformer feature extraction part, and a counting regression part. In the CNN part, we utilize DenseNet121 to learn rich semantic features with its inherent dense connection structure. In the Transformer part, we introduce two attention modules, Multi-Scale Dilated Attention (MSDA) and Location-Enhanced Attention (LEA), working together to extract more expressive features. The output features are then fed into the regression part to generate the predicted counting results. Experiments on four crowd counting benchmark datasets demonstrate that our proposed CrowdCCT can achieve superior performance. Also, the experimental results validate the feasibility and effectiveness of combining CNN and Transformer for weakly supervised counting tasks. Our work could be expected to promote further combination research on CNN and Transformer.

The task of crowd counting and density map estimation is riddled with many challenges, such as occlusions, non-uniform density, intra-scene and inter-scene variations in scale and perspective. Due to the development of deep learning and large crowd datasets in recent years, most crowd counting methods have achieved notable success. This paper aims to solve crowd density estimation problem for both sparse and dense conditions. To this end, we make two contributions: (1) a network named Patch Scale Discriminant Regression Network (PSDR). Given an input crowd image, it divides the image into patches and sends image patches of different density levels into different regression networks to get the corresponding density maps. It combines all patch density maps to predict the entire density map as the output. (2) A person classification activation map (CAM) method. CAM provides person location information and guides the generation of the entire density map in the final stage. Experiment confirms that CAM allows PSDR to gain another round of performance boost. For instance, on the SmartCity dataset, we achieve (8.6–1.1) MAE and (11.6–1.4) MSE. Our method combining above two methods performs better than state-of-the-art methods.