Density-aware and background-aware network for crowd counting via multi-task learning

ADMET property prediction via multi-task graph learning under adaptive auxiliary task selection

As a research hotspot of computer vision, crowd counting methods have achieved success in natural images. But crowd counting in aerial images are rarely explored, and existing methods do not perform well because of the higher resolution, smaller object scale and more complex scene. Therefore, this paper proposes a lightweight dual-task network (LDNet) for crowd counting, which only uses bifurcated structure to overcome these new challenges in aerial images without complicated pipelines. To realize this, a complete but efficient Guidance Branch is proposed to assist Counting Branch in fitting crowd distribution. Furthermore, a scene attention mechanism is used to consider the complex scene information, which are never considered by existing methods. Our LD-Net outperforms existing methods on aerial crowd counting dataset (Visdrone), and gets better or comparable results on natural crowd counting datasets (UCF_CC_50, UCF_QNRF, ShanghaiTech Part A).

- Crowd density prediction is a critical task in modern public safety, urban planning, and large-scale event management. Accurately estimating the number of people and their spatial distribution enables authorities to prevent dangerous overcrowding and optimize resource deployment. This paper presents a hybrid crowd density prediction framework combining Convolutional Neural Networks (CNN) with Histogram of Oriented Gradients (HOG) feature descriptors. The CNN branch employs a modified VGG-16 backbone with dilated convolutions to capture high-level spatial representations, while the HOG branch extracts complementary gradient-based structural information from local image regions. Feature embeddings from both branches are fused through fully connected layers with dropout regularization to generate spatial density maps. The system is trained and validated on the ShanghaiTech Part A and Part B benchmark datasets. The proposed CNN+HOG hybrid achieves a Mean Absolute Error (MAE) of 75.4 and Root Mean Square Error (RMSE) of 112.3 on Part A, and MAE of 11.8, RMSE of 18.5 on Part B - outperforming pure CNN and HOG baselines. Training convergence analysis and error distribution studies confirm the robustness and generalization of the proposed approach across varying crowd densities and scene complexities. KeyWords: Crowd density prediction, convolutional neural network (CNN), histogram of oriented gradients (HOG), deep learning, density map estimation, ShanghaiTech dataset, feature fusion.

In recent years, crowd counting has been widely used in computer vision. But on many issues like large scale variations, perspective distortions and complex backgrounds, it remains a challenging task. Previous methods usually adopted dilation convolution to enlarge the receptive field, these methods can generate poor density maps. To solve this problem, in this paper, we proposed a novel structure called Attention-based Contextual Convolution Network for crowd counting. ACCNet contains two components. The Contextual Convolution Network first gets the relative influence of different scale-aware feature and attention module to further enlarge the range of scales while getting the final crowd density map in the network. The key section of our network is the dense dilated module, this method can make the front and back dilation layers tightly connected to preserve information from changing scales. Other than that, attention mechanism also plays an important role in learning density maps from input images. To verify the feasibility of our method, we compare it against the state-of-the-art structure on four crowd counting datasets (ShanghaiTech Part_A, ShanghaiTech Part_B, WorldEXPO'10 and UCF_CC_50), which demonstrates that ACCNet is effective and robust for complex scenes.

As a classical machine learning method, multi‐task learning (MTL) has been widely applied in computer vision technology. Due to deep convolutional neural network (D‐CNN) having strong ability of feature representation, the combination of MTL and D‐CNN has attracted much attention from researchers recently. However, this kind of combination has rarely been explored in the field of vehicle analysis. The authors propose a D‐CNN enhanced with weighted multi‐attribute strategy for extensive exploration of comprehensive vehicle attributes over surveillance images. Specifically, regarding to recognising vehicle model and make/manufacturer, several related attributes as auxiliary tasks are incorporated in the training process of D‐CNN structure. The proposed strategy focuses more on the main task compared with traditional MTL methods, which has assigned different weights for the main task and auxiliary tasks rather than treating all involved tasks equally. To the extent of their knowledge, this is the first report relating to the combination of D‐CNN and weighted MTL for exploration of comprehensive vehicle attributes. The following experiments will show that the proposed approach outperforms the state‐of‐the‐art method for the vehicle recognition and improves the accuracy rate by about 10% for the analysis of other vehicle attributes on the recently public CompCars dataset.

In view of the continuous improvement of people's quality of life and the gradual increase of the number of cars in the city, the driving safety and efficiency of traditional cars have become a common concern. Whether it is some advanced driving assistance systems or auto drive system, it is an indispensable part of the whole system to accurately predict the future trajectory of other vehicles in the driving environment. Therefore, this paper studies the modeling and prediction of vehicle trajectory based on the Internet of vehicles. A multi task learning method is proposed, and the vehicle trajectory modeling and prediction based on multi task learning are discussed. At the same time, the analysis and modeling of vehicle collision warning at intersections are studied. Finally, by testing the impact of the dimension of the activity area on the trajectory coordinate prediction performance, it is verified that the model proposed in this paper can achieve better results, and the prediction performance of the model with auxiliary tasks is better than that without auxiliary tasks, it is of great significance to the driving safety and efficiency of vehicles in the future.

SummaryDeep learning has made great achievements in the field of speech recognition. With the popularization of embedded devices such as intelligent speaker and the demand for dialect interaction scenes, it poses great challenges to far‐field speech recognition and dialect language recognition. In order to solve the dialect language recognition of embedded devices in far‐field speech recognition, we propose a deep learning neural network model with multitask learning. First, the audio is passed through the end‐to‐end noise reduction model to improve the effect of audio recognition. Then we define dialect recognition as the main task and dialect area as the auxiliary task, using the multitask learning method to improve the accuracy of dialect classification. The experimental results show that the end‐to‐end noise reduction model can improve the accuracy of audio recognition, and the best effect can be 7.54% higher than the baseline, and the accuracy of dialect language recognition can be improved by about 5% through multi task learning model.

Crowd counting is an important task in the surveillance camera system. The methods of crowd counting have changed from counting each object in the scene to estimating the density map and the count will be reached by integrating the entire density map. The previous methods used multi-column convolutional neural networks (CNN) with different sizes of filters to describe objects of different scales. To increase the performance of the multi-column model, some methods suggest adding a classifier to select a single-column CNN that gives the expecting results. However, having multiple columns and filters will complicate the model and trained model with limited crowd data will estimate bad density maps. To solve this problem, we propose the "Single-Column CNN" (SCC) method which uses a single-column CNN with only one size of filters (3x3) and replaces normal convolution with different sizes of filters by the dilated convolution. Our approach makes the density map estimation model become a simple but strong CNN. Experiments show that our method achieves significant improvements compared to the method using multi-column CNN in the ShanghaiTech dataset.

Multi-task learning (MTL) has achieved success over a wide range of problems, where the goal is to improve the performance of a primary task using a set of relevant auxiliary tasks. However, when the usefulness of the auxiliary tasks w.r.t. the primary task is not known a priori, the success of MTL models depends on the correct choice of these auxiliary tasks and also a balanced mixing ratio of these tasks during alternate training. These two problems could be resolved via manual intuition or hyper-parameter tuning over all combinatorial task choices, but this introduces inductive bias or is not scalable when the number of candidate auxiliary tasks is very large. To address these issues, we present AutoSeM, a two-stage MTL pipeline, where the first stage automatically selects the most useful auxiliary tasks via a Beta-Bernoulli multi-armed bandit with Thompson Sampling, and the second stage learns the training mixing ratio of these selected auxiliary tasks via a Gaussian Process based Bayesian optimization framework. We conduct several MTL experiments on the GLUE language understanding tasks, and show that our AutoSeM framework can successfully find relevant auxiliary tasks and automatically learn their mixing ratio, achieving significant performance boosts on several primary tasks. Finally, we present ablations for each stage of AutoSeM and analyze the learned auxiliary task choices.

Dense crowd counting aims to predict thousands of human instances from an image, by calculating integrals of a density map over image pixels. Existing approaches mainly suffer from the extreme density variances. Such density pattern shift poses challenges even for multi-scale model ensembling. In this paper, we propose a simple yet effective approach to tackle this problem. First, a patch-level density map is extracted by a density estimation model and further grouped into several density levels which are determined over full datasets. Second, each patch density map is automatically normalized by an online center learning strategy with a multipolar center loss. Such a design can significantly condense the density distribution into several clusters, and enable that the density variance can be learned by a single model. Extensive experiments demonstrate the superiority of the proposed method. Our work outperforms the state-of-the-art by 4.2%, 14.3%, 27.1% and 20.1% in MAE, on ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF-QNRF datasets, respectively.

Aiming to tackle the most intractable problems of scale variation and complex backgrounds in crowd counting, we present an innovative framework called Hierarchical Region-Aware Network (HRANet) for crowd counting in this paper, which can better focus on crowd regions to accurately predict crowd density. In our implementation, first, we design a Region-Aware Module (RAM) to capture the internal differences within different regions of the feature map, thus adaptively extracting contextual features within different regions. Furthermore, we propose a Region Recalibration Module (RRM) which adopts a novel region-aware attention mechanism (RAAM) to further recalibrate the feature weights of different regions. By the integration of the above two modules, the influence of background regions can be effectively suppressed. Besides, considering the local correlations within different regions of the crowd density map, a Region Awareness Loss (RAL) is designed to reduce false identification while producing the locally consistent density map. Extensive experiments on five challenging datasets demonstrate that the proposed method significantly outperforms existing methods in terms of counting accuracy and quality of the generated density map. In addition, a series of specific experiments in crowd gathering scenes indicate that our method can be practically applied to crowd localization.

Multi-task learning aims to learn multiple tasks jointly by exploiting their relatedness to improve the generalization performance for each task. Traditionally, to perform multi-task learning, one needs to centralize data from all the tasks to a single machine. However, in many real-world applications, data of different tasks may be geo-distributed over different local machines. Due to heavy communication caused by transmitting the data and the issue of data privacy and security, it is impossible to send data of different task to a master machine to perform multi-task learning. Therefore, in this paper, we propose a distributed multi-task learning framework that simultaneously learns predictive models for each task as well as task relationships between tasks alternatingly in the parameter server paradigm. In our framework, we first offer a general dual form for a family of regularized multi-task relationship learning methods. Subsequently, we propose a communication-efficient primal-dual distributed optimization algorithm to solve the dual problem by carefully designing local subproblems to make the dual problem decomposable. Moreover, we provide a theoretical convergence analysis for the proposed algorithm, which is specific for distributed multi-task relationship learning. We conduct extensive experiments on both synthetic and real-world datasets to evaluate our proposed framework in terms of effectiveness and convergence.

Counting the number of people in public places has received much attention, and researchers have devoted much effort to the task. However, the existing crowd counting approaches are mainly trained and tested in similar scenarios. The performance of crowd counting approaches degrades sharply when the test scenarios of the models are of different types from its training scenes. In practice, the crowd scenes are highly variable, and the lack of cross-scene capability could seriously limit the application of the existing approaches. We attribute the improvement in cross-scene crowd counting capability to the necessity of accommodating large changes in the scale of individuals and the ability to suppress the interference of cluttered backgrounds. To this end, we propose a multi-scale feature fusion network (MFFNet) with cascaded supervision. The multi-scale features extracted from the crowd images are upsampled and then combined into several feature blocks, followed by convolution and deconvolution operations on the feature blocks to derive feature matrices of different resolutions. The feature matrices are fused from bottom to top. In the process of feature fusion, the crowd density maps corresponding to the feature matrices of different resolutions are predicted separately. We devise cascaded supervision to synchronously optimize the network of different resolution density map prediction during training. The cross-scene crowd counting experiments are conducted on four types of scenes: ShanghaiTech Part_A (SHT A) with high-density crowd scenes and small-scale individuals, ShanghaiTech Part_B (SHT B) with sparse crowd distribution and medium-scale individuals, UCF_CC_50 dataset with extremely dense scenes and tiny scale individuals, and UCF-QNRF dataset with extreme variations. MFFNet exhibits the strongest scene adaptability relative to the state-of-the-art approaches, with an average decrease of 17.1% and 8.4% in mean absolute error (MAE) and mean square error (mse), respectively. The contributions of different components in our method are verified in the ablation study using the devised evaluation metrics. Our implementation will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/learnsharing/MFFNet</uri> .

The task of crowd counting and density map estimation is riddled with many challenges, such as occlusions, non-uniform density, intra-scene and inter-scene variations in scale and perspective. Due to the development of deep learning and large crowd datasets in recent years, most crowd counting methods have achieved notable success. This paper aims to solve crowd density estimation problem for both sparse and dense conditions. To this end, we make two contributions: (1) a network named Patch Scale Discriminant Regression Network (PSDR). Given an input crowd image, it divides the image into patches and sends image patches of different density levels into different regression networks to get the corresponding density maps. It combines all patch density maps to predict the entire density map as the output. (2) A person classification activation map (CAM) method. CAM provides person location information and guides the generation of the entire density map in the final stage. Experiment confirms that CAM allows PSDR to gain another round of performance boost. For instance, on the SmartCity dataset, we achieve (8.6–1.1) MAE and (11.6–1.4) MSE. Our method combining above two methods performs better than state-of-the-art methods.

Adverse drug reactions (ADRs) are one of the leading causes of mortality in health care. Current ADR surveillance systems are often associated with a substantial time lag before such events are officially published. On the other hand, online social media such as Twitter contain information about ADR events in real-time, much before any official reporting. Current state-of-the-art in ADR mention extraction uses Recurrent Neural Networks (RNN), which typically need large labeled corpora. Towards this end, we propose a multi-task learning based method which can utilize a similar auxiliary task (adverse drug event detection) to enhance the performance of the main task, i.e., ADR extraction. Furthermore, in absence of the auxiliary task dataset, we propose a novel joint multi-task learning method to automatically generate weak supervision dataset for the auxiliary task when a large pool of unlabeled tweets is available. Experiments with \(\sim \)0.48M tweets show that the proposed approach outperforms the state-of-the-art methods for the ADR mention extraction task by \(\sim \)7.2 % in terms of F1 score.