- Research Article
13
- 10.1016/j.neucom.2021.03.128
Adversarial scale-adaptive neural network for crowd counting
- Apr 13, 2021
- Neurocomputing
- Xinyue Chen + 4 more +4
Adversarial scale-adaptive neural network for crowd counting
Crowd counting is a fundamental yet challenging task, which desires rich information to generate pixel-wise crowd density maps. However, most previous methods only used the limited information of RGB images and cannot well discover potential pedestrians in unconstrained scenarios. In this work, we find that incorporating optical and thermal information can greatly help to recognize pedestrians. To promote future researches in this field, we introduce a large-scale RGBT Crowd Counting (RGBT-CC) benchmark, which contains 2,030 pairs of RGB-thermal images with 138,389 annotated people. Furthermore, to facilitate the multimodal crowd counting, we propose a cross-modal collaborative representation learning framework, which consists of multiple modality-specific branches, a modality-shared branch, and an Information Aggregation-Distribution Module (IADM) to capture the complementary information of different modalities fully. Specifically, our IADM incorporates two collaborative information transfers to dynamically enhance the modality-shared and modality-specific representations with a dual information propagation mechanism. Extensive experiments conducted on the RGBT-CC benchmark demonstrate the effectiveness of our framework for RGBT crowd counting. Moreover, the proposed approach is universal for multimodal crowd counting and is also capable to achieve superior performance on the ShanghaiTechRGBD [22] dataset. Finally, our source code and benchmark have been released at http://lingboliu.com/RGBT_Crowd_Counting.html.
Adversarial scale-adaptive neural network for crowd counting
Adversarial scale-adaptive neural network for crowd counting
PaDNet: Pan-Density Crowd Counting.
Crowd counting is a highly challenging problem in computer vision and machine learning. Most previous methods have focused on consistent density crowds, i.e., either a sparse or a dense crowd, meaning they performed well in global estimation while neglecting local accuracy. To make crowd counting more useful in the real world, we propose a new perspective, named pan-density crowd counting, which aims to count people in varying density crowds. Specifically, we propose the Pan-Density Network (PaDNet) which is composed of the following critical components. First, the Density-Aware Network (DAN) contains multiple subnetworks pretrained on scenarios with different densities. This module is capable of capturing pandensity information. Second, the Feature Enhancement Layer (FEL) effectively captures the global and local contextual features and generates a weight for each density-specific feature. Third, the Feature Fusion Network (FFN) embeds spatial context and fuses these density-specific features. Further, the metrics Patch MAE (PMAE) and Patch RMSE (PRMSE) are proposed to better evaluate the performance on the global and local estimations. Extensive experiments on four crowd counting benchmark datasets, the ShanghaiTech, the UCF-CC-50, the UCSD, and the UCFQNRF, indicate that PaDNet achieves state-of-the-art recognition performance and high robustness in pan-density crowd counting.
Read moreA cross-modal crowd counting method combining CNN and cross-modal transformer
A cross-modal crowd counting method combining CNN and cross-modal transformer
NF-Net: Crowd Counting Based on Near-Far Network and Dynamic Dual Attention Mechanism
Crowd counting and precise localization in dense scenes are critical tasks in computer vision. Although the point-based prediction framework P2PNet eliminates complex post-processing steps through ensemble prediction, it still faces challenges such as insufficient multi-scale feature extraction and limited ability to perceive key information in complex scenes. To address these issues, this paper improves P2PNet by proposing a model based on a far-near network and a dynamic dual attention mechanism. Specifically, it introduces a far-near adaptive network (FN-Net) and a dynamic dual attention mechanism (DDAM). FN-Net explicitly models continuous scale variations caused by perspective effects by dividing the image into regions based on spatial position and assigning differentiated receptive fields. DDAM focuses on crowded areas through parallel spatial attention and channel attention sub-modules, selects discriminative features, and integrates a dynamic weighted fusion mechanism to adaptively combine the advantages of both attentions. Experiments show that our approach effectively enhances key features while suppressing background noise thus improves crowd counting accuracy.
Read moreLate better than early: A decision-level information fusion approach for RGB-Thermal crowd counting with illumination awareness
Late better than early: A decision-level information fusion approach for RGB-Thermal crowd counting with illumination awareness
Read moreACCNet: Attention-based Contextual Convolutional Network for Crowd Counting
In recent years, crowd counting has been widely used in computer vision. But on many issues like large scale variations, perspective distortions and complex backgrounds, it remains a challenging task. Previous methods usually adopted dilation convolution to enlarge the receptive field, these methods can generate poor density maps. To solve this problem, in this paper, we proposed a novel structure called Attention-based Contextual Convolution Network for crowd counting. ACCNet contains two components. The Contextual Convolution Network first gets the relative influence of different scale-aware feature and attention module to further enlarge the range of scales while getting the final crowd density map in the network. The key section of our network is the dense dilated module, this method can make the front and back dilation layers tightly connected to preserve information from changing scales. Other than that, attention mechanism also plays an important role in learning density maps from input images. To verify the feasibility of our method, we compare it against the state-of-the-art structure on four crowd counting datasets (ShanghaiTech Part_A, ShanghaiTech Part_B, WorldEXPO'10 and UCF_CC_50), which demonstrates that ACCNet is effective and robust for complex scenes.
Read moreA Single-Column Convolutional Neural Networks for Crowd Counting
Crowd counting is an important task in the surveillance camera system. The methods of crowd counting have changed from counting each object in the scene to estimating the density map and the count will be reached by integrating the entire density map. The previous methods used multi-column convolutional neural networks (CNN) with different sizes of filters to describe objects of different scales. To increase the performance of the multi-column model, some methods suggest adding a classifier to select a single-column CNN that gives the expecting results. However, having multiple columns and filters will complicate the model and trained model with limited crowd data will estimate bad density maps. To solve this problem, we propose the "Single-Column CNN" (SCC) method which uses a single-column CNN with only one size of filters (3x3) and replaces normal convolution with different sizes of filters by the dilated convolution. Our approach makes the density map estimation model become a simple but strong CNN. Experiments show that our method achieves significant improvements compared to the method using multi-column CNN in the ShanghaiTech dataset.
Read moreNWPU-Crowd: A Large-Scale Benchmark for Crowd Counting and Localization.
In the last decade, crowd counting and localization attract much attention of researchers due to its wide-spread applications, including crowd monitoring, public safety, space design, etc. Many convolutional neural networks (CNN) are designed for tackling this task. However, currently released datasets are so small-scale that they can not meet the needs of the supervised CNN-based algorithms. To remedy this problem, we construct a large-scale congested crowd counting and localization dataset, NWPU-Crowd, consisting of 5,109 images, in a total of 2,133,375 annotated heads with points and boxes. Compared with other real-world datasets, it contains various illumination scenes and has the largest density range ( 0∼20,033). Besides, a benchmark website is developed for impartially evaluating the different methods, which allows researchers to submit the results of the test set. Based on the proposed dataset, we further describe the data characteristics, evaluate the performance of some mainstream state-of-the-art (SOTA) methods, and analyze the new problems that arise on the new data. What's more, the benchmark is deployed at https://www.crowdbenchmark.com/, and the dataset/code/models/results are available at https://gjy3035.github.io/NWPU-Crowd-Sample-Code/.
Read moreIncreasing Reciprocal Social Interactions Between Children Who Use Augmentative and Alternative Communication and Peers Using a Collaborative Learning Framework.
Children with complex communication needs who rely on augmentative and alternative communication (AAC) strategies to express themselves face significant difficulties interaction with peers. This study sought to design, implement, and evaluate a collaborative photography intervention designed to increase reciprocal social interaction between children who use AAC and their same-age peers. A single-subject, withdrawal design (ABAB) was used to explore the functional relationship between engagement in a collaborative photography intervention and the frequency of reciprocal social interactions between children who use AAC and their same-age peers. Partial-interval time sampling was used to code the number of reciprocal social interactions across four dyads. Increased frequencies of reciprocal social interactions were observed in intervention phases across all four dyads. Very large levels of effect and 100% nonoverlapping data were noted for Dyads 1 and 4. Moderate levels of effect and 70% of nonoverlapping data were noted for Dyads 2 and 3. Collaborative learning frameworks may be used to increase reciprocal social interactions between children who use AAC and their peers. Speech-language pathologists should consider utilizing collaborative learning elements in activities with children who use AAC. Future research is needed to further explore collaborative learning frameworks for interventions for children who use AAC. https://doi.org/10.23641/asha.25464064.
Read moreGroup-wise Deep Co-saliency Detection
In this paper, we propose an end-to-end group-wise deep co-saliency detection approach to address the co-salient object discovery problem based on the fully convolutional network (FCN) with group input and group output. The proposed approach captures the group-wise interaction information for group images by learning a semantics-aware image representation based on a convolutional neural network, which adaptively learns the group-wise features for co-saliency detection. Furthermore, the proposed approach discovers the collaborative and interactive relationships between group-wise feature representation and single-image individual feature representation, and model this in a collaborative learning framework. Finally, we set up a unified end-to-end deep learning scheme to jointly optimize the process of group-wise feature representation learning and the collaborative learning, leading to more reliable and robust co-saliency detection results. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches.
Read moreCollaborative learning based on centroid-distance-vector for wearable devices
Collaborative learning based on centroid-distance-vector for wearable devices
A Constructivist Framework for Online Collaborative Learning
The purpose of this chapter is to review and discuss theoretical perspectives that help to frame collaborative learning online. The chapter investigates literature about the type of learning and behavior that are anticipated and researched among participants learning collaboratively and discusses how these attributes explain computer-supported collaborative learning. The literature about learning is influenced by perspectives from a number of fields, particularly philosophy, psychology, and sociology. This chapter describes some of these perspectives from the fields of cognitive psychology, adult learning, and collaborative group learning. Recent research into computer-supported collaborative learning that applies these theories will also be discussed.
Read moreCross-modal collaborative feature representation via Transformer-based multimodal mixers for RGB-T crowd counting
Cross-modal collaborative feature representation via Transformer-based multimodal mixers for RGB-T crowd counting
A Framework for Collaborative and Convenient Learning on Cloud Computing Platforms
The depth of learning resides in collaborative work with more engagement and fun. Technology can enhance collaboration with a higher level of convenience and cloud computing can facilitate this in a cost effective and scalable manner. However, to deploy a successful online learning environment, elementary components of learning pedagogy must be embedded in the technology framework. A cloud computing technology based learning platform built on the proven learning pedagogies will be most successful attempt to facilitate collaborative and convenient learning. Such platform will support the learners to connect and coordinate the online resources in a more efficient, effective and practical way. This paper proposes a new theoretical framework for collaborative and convenient learning on cloud computing technology by reviewing various learning theories and integrating them with the technology framework. Architectural details of the framework have been presented with suitable examples for a better understanding. The work has a lot of significance for deploying cloud computing based collaborative learning solutions in the educational institutions.
Read moreDynamic Group Formation With Intelligent Tutor Collaborative Learning: A Novel Approach for Next Generation Collaboration
Group Formation (GF) strongly influences the collaborative learning process in Computer-Supported Collaborative Learning (CSCL). Various factors affect GF that include personal characteristics, social, cultural, psychological, and cognitive diversity. Although different group formation methods aim to solve the group compatibility problem, an optimal solution for dynamic group formation is still not addressed. In addition, the research lacks to supplement collaborative group formation with a collaborative platform. In this study, the next level of collaboration in CSCL and Intelligent Tutoring System (ITS) platforms is achieved. First, initial groups are formed based on students learning styles, and knowledge level, i.e. for knowledge level, an activity-based dynamic group formation technique is proposed. In this activity, swapping of students takes place on each permutation based on their knowledge level. Second, the formed heterogeneous balanced groups are used to augment the collaborative learning system. For this purpose, a hybrid framework of Intelligent Tutor Collaborative Learning (ITSCL) is used that provides a unique and real-time collaborative learning platform. Third, an experiment is conducted to evaluate the significance of the proposed study. Inferential and descriptive statistics of Paired T-Tests are applied for comprehensive analysis of recorded observations. The statistical results show that the proposed ITSCL framework positively impacts student learning and results in higher learning gains.
Read more