SEGMENT-AND-COUNT: VEHICLE COUNTING IN AERIAL IMAGERY USING ATROUS CONVOLUTIONAL NEURAL NETWORKS

Towards improving quality of video-based vehicle counting method for traffic flow estimation

Mapping road networks is currently both expensive and labor-intensive. High-resolution aerial imagery provides a promising avenue to automatically infer a road network. Prior work uses convolutional neural networks (CNNs) to detect which pixels belong to a road (segmentation), and then uses complex post-processing heuristics to infer graph connectivity. We show that these segmentation methods have high error rates because noisy CNN outputs are difficult to correct. We propose RoadTracer, a new method to automatically construct accurate road network maps from aerial images. RoadTracer uses an iterative search process guided by a CNN-based decision function to derive the road network graph directly from the output of the CNN. We compare our approach with a segmentation method on fifteen cities, and find that at a 5% error rate, RoadTracer correctly captures 45% more junctions across these cities.

The study aimed to determine the efficacy and capabilities of using high-resolution aerial imagery and a convolutional neural network (CNN) to identify plant species and monitor land cover and land change in the context of remote sensing. The full capabilities of a CNN were examined, including testing whether the platform could be used for land cover and the evaluation of land change over time. An unmanned aerial vehicle (UAV) was used to collect the aerial data of the study area. The CNN was encoded and operated in RStudio, while digitised data from the input imagery were used by the programme as training and validation data. The object in this respect was to learn about the relevant features of the landscape, and thereafter to classify the Opuntia invasive plant species. Accuracy assessments were carried out on the results to test the efficacy of the aerial imagery in terms of its accuracy and reliability. The classification achieved an overall accuracy of 93%, while the kappa coefficient score was 0.86. CNN was also able to predict the land coverage area of Opuntia to be within four percent (4%) of the ground truthing data. A change in land cover over time was detected by the programme after the manual clearing of the plant had been undertaken. This research has determined that the use of a CNN in remote sensing is a very powerful tool for supervised image classifications. It can be used for monitoring land cover in that it is able to accurately estimate the spatial distribution of plant species and to monitor the growth or decline in the species over time. As such, it is an efficient methodology and its use in remote sensing could be extended.

Deep understanding of urban mobility is of great significance for many real-world applications, such as urban traffic management and autonomous driving. This thesis develops deep learning methodologies to extract vehicle counts from streaming realtime video captured by multiple low resolution web cameras and construct maps of traffic density in a city environment; in particular, we focus on cameras installed in the Manhattan borough of NYC. The large-scale videos from these web cameras have low spatial and temporal resolution, high occlusion, large perspective, and variable environment conditions, making most existing methods to lose their efficacy. To overcome these challenges, the thesis develops several techniques: 1. a block-level regression model with a rank constraint to map the dense image feature into vehicle densities; 2. a deep multi-task learning framework based on fully convolutional neural networks to jointly learn vehicle density and vehicle count; 3. deep spatio-temporal networks for vehicle counting to incorporate temporal information of the traffic flow; and 4. multi-source domain adaptation mechanisms with adversarial learning to adapt the deep counting model to multiple cameras. To train and validate the proposed system, we have collected a largescale webcam traffic dataset CityCam that contains 60 million frames from 212 webcams installed in key intersections of NYC. Of there, 60; 000 frames have been annotated with rich information, leading to about 900; 000 annotated objects. To the best of our knowledge, it is the first and largest webcam traffic dataset with such large number of elaborate annotations. The proposed methods are integrated into the CityScapeEye system that has been extensively evaluated and compared to existing techniques on different counting tasks and datasets, with experimental results demonstrating the effectiveness and robustness of CityScapeEye.

A vision-based traffic surveillance processor is being developed at JPL. This processor uses innovative image segmentation and classification techniques for vehicles in freeway images, including those with large shadows. These results enable the computation of many useful traffic parameters. A wavelet-based algorithm has been developed for vehicle detection and segmentation. Specifically, two types of mother wavelet has been created and tested: the first for shape-size discrimination of vehicles from their background; and the second for locating where vehicles join their shadows, thus enabling segmentation of the vehicles from their shadows. Combining these two wavelets enables robust segmentation of vehicles from busy freeways. This method reduces the false-alarm rate in vehicle counts, since shadows are no longer mistaken for vehicles. We use neural networks for vehicle classification. To reduce system complexity and training time, we use, as preprocessors, several feature extraction methods, such as invariant-moment and Hermite-moment computations. This preprocessing enables orders of magnitude reductions in training time and a great increase in classification accuracy.

Vehicle counting is a process to estimate traffic density on roads to assess the traffic conditions for intelligent transportation systems (ITS). Real-time traffic management systems have become popular recently due to the availability of high end cameras and technology. The present traffic management systems focus on speed detection, signal jumping, zebra crossing but not on traffic density estimation. Proposed video-based vehicle counting and tracking method using a video captured on CCTV and handheld mobile cameras. The system can be used in smart cities to create smart traffic light signals, in which duration of each signal depends on real time vehicle density in a particular lane of road. Vehicle counting is performed in two steps: the captured video is sent to You Only Look Once (YOLO) based deep learning framework to detect, count and classify the vehicles. Multi vehicular tracking is adopted using Deep SORTalgorithm to track the vehicles in video frames. Model was trained for six different classes, using Google Colaboratory. Datasets of vehicles specifically pertaining to Indian roads environment is considered for implementation. The performance of the model was analyzed, proposed model has tested and obtained an average counting accuracy of 86.56% while the average precision is 93.85%. The model can be implemented for ascertaining the traffic density on roads and this provides knowledge for infrastructural development to authorities. It can also be an integral part of smart city projects to develop intelligent and smart traffic surveillance system.

We propose a vehicle tracking algorithm that takes a new occlusion reasoning approach. We consider two different types of occlusions: explicit occlusion and implicit occlusion. We also propose a traffic flow extraction method with the velocity and trajectory of the moving vehicles. The proposed vehicle tracking system is composed of three parts: vehicle segmentation, vehicle tracking, and traffic parameter extraction. The vehicle segmentation part separates moving vehicles from their background. We employ the adaptive background approach, which does not update the background of moving objects. We also design a 2D token-based algorithm for vehicle tracking using Kalman filtering that has a modest amount of computational complexity. The traffic parameter extraction part estimates the traffic parameters, such as the vehicle count and the average speed. It also extracts the traffic flows. Finally, we have evaluated the proposed algorithm with some MPEG-7 test sequences.

Vehicle counting and traffic volume estimation on traffic videos are challenging tasks for traffic monitoring and management. Most previous methods are based on a process of vehicle detection and tracking or a process of time-spatial image (TSI)-based background subtraction, which suffer from occlusion and small size of vehicles. To overcome these difficulties, a time-spatial-based structure is proposed for vehicle counting and traffic volume estimation, without relying on tracking and TSI-based background subtraction. This structure has three main parts. First, a TSI-based density map generation model is proposed for generating density maps for TSIs, which makes it possible to automatically generate TSI training samples. Second, the time-spatial multiscale net (TM-Net) is proposed for estimating density map of TSI; with the stacked multiscale modules and the spatial attention module, the proposed TM-Net can partly overcome the difficulties brought from occlusion, small size, and deformation. Finally, a vehicle counting and traffic volume estimation model is designed for counting and volume estimation on the results of TM-Net. Experiments performed on the public UA-DETRAC data set show that the proposed TM-Net-based vehicle counting method outperforms the tested representative counting methods, and the proposed framework also can estimate traffic volume efficiently.

Automatic vehicle detection and counting are considered vital in improving traffic control and management. This work presents an effective algorithm for vehicle detection and counting in complex traffic scenes by combining both convolution neural network (CNN) and the optical flow feature tracking-based methods. In this algorithm, both the detection and tracking procedures have been linked together to get robust feature points that are updated regularly every fixed number of frames. The proposed algorithm detects moving vehicles based on a background subtraction method using CNN. Then, the vehicle’s robust features are refined and clustered by motion feature points analysis using a combined technique between KLT tracker and K-means clustering. Finally, an efficient strategy is presented using the detected and tracked points information to assign each vehicle label with its corresponding one in the vehicle’s trajectories and truly counted it. The proposed method is evaluated on videos representing challenging environments, and the experimental results showed an average detection and counting precision of 96.3% and 96.8%, respectively, which outperforms other existing approaches.

Automatic data labeling by neural networks for the counting of objects in videos

Large wood (LW) is vital for river ecosystems, influencing hydraulics, sediment dynamics, and habitat complexity. Quantifying LW storage is crucial for river management, habitat restoration, and flood mitigation but is challenging due to its spatial and temporal variability. Traditional field surveys are labour-intensive and limited in scope, while manual mapping of aerial imagery, though detailed, is equally time-consuming. This study presents an automated approach to detect and measure LW using high-resolution imagery and convolutional neural networks (CNNs). Two models, YOLOv10 for wood detection and YOLOv8 for wood segmentation, were trained using data from eight rivers in the Swiss Alps and Argentinean Andes. A separate river dataset was used for independent testing. The detection model achieved 90 % accuracy in estimating wood volume compared to field data and detected 97 % of large wood pieces at a 0.3 confidence threshold. The segmentation model achieved a mean Average Precision of 70 %. Although wood diameters were underestimated, the method reliably captured spatial distributions of wood. Expanding the training dataset with more diverse examples could improve performance. This automated approach offers a scalable and efficient tool for monitoring riverine wood, overcoming the limitations of traditional field-based surveys.

In Unmanned Aerial Vehicle (UAV) networks, multi-class aerial image classification (AIC) is crucial in various applications, from environmental monitoring to infrastructure inspection. Deep Learning (DL), a powerful tool in artificial intelligence (AI), proves significant in this context, enabling the model to analyze and classify complex aerial images effectually. By utilizing advanced neural network architectures, such as convolutional neural networks (CNN), DL models outperform at identifying complex features and patterns within the aerial imagery. These models can extract spectral and spatial information from the captured data, classifying diverse terrains, structures, and objects precisely. Furthermore, the integration of Snake Optimization algorithms assists in fine-tuning the classification process, improving accuracy. As UAV networks continue to expand, DL-powered multi-class AIC significantly enhances the performance of surveillance, reconnaissance, and remote sensing tasks, contributing to the advancement of autonomous aerial systems. This study proposes a Snake Optimization Algorithm with Deep Learning for Multi-Class Aerial Image Classification (SOADL-MCAIC) methodology on UAV Networks. The main purpose of SOADL-MCAIC methodology is to recognize the presence of multiple classes of aerial images on the UAV networks. To accomplish this, the SOADL-MCAIC technique utilizes Gaussian filtering (GF) for pre-processing. In addition, the SOADL-MCAIC technique employs the Efficient DenseNet model to learn difficult and intrinsic features in the image. The SOA-based hyperparameter tuning process is used to enhance the performance of the Efficient DenseNet technique. Finally, the kernel extreme learning machine (KELM)-based classification algorithm is implemented to identify and classify the presence of various classes in aerial images. The simulation outcomes of the SOADL-MCAIC method are examined under the UCM land use dataset. The experimental analysis of the SOADL-MCAIC method portrayed a superior accuracy value of 99.75% over existing models.

In the past few years, automatic building detection in aerial images has become an emerging field in computer vision. Detecting the specific types of houses will provide information in urbanization, change detection, and urban monitoring that play increasingly important roles in modern city planning and natural hazard preparedness. In this paper, we demonstrate the effectiveness of detecting various types of houses in aerial imagery using Faster Region-based Convolutional Neural Network (Faster-RCNN). After formulating the dataset and extracting bounding-box information, pre-trained ResNet50 is used to get the feature maps. The fully convolutional Region Proposal Network (RPN) first predicts the bounds and objectness score of objects (in this case house) from the feature maps. Then, the Region of Interest (RoI) pooling layer extracts interested regions to detect objects that are present in the images. To the best of our knowledge, this is the first attempt at detecting houses using Faster R-CNN that has achieved satisfactory results. This experiment opens a new path to conduct and extent the works not only for civil and environmental domain but also other applied science disciplines.

Landslides are one of the most frequent and destructive natural disasters, posing severe threats to human lives, infrastructure, and the environment. Accurate prediction and classification of landslide occurrences using remote sensing and aerial imagery have become vital for disaster management and mitigation. However, existing deep learning models often fail to capture fine-grained spatial variations and multi-class distinctions leads to inaccurate segmentation and misclassification of landslide-affected regions. To overcome these challenges, a novel penta classification of landslide via deep learning based SegNet and RegNet (PENTA-SERE) model has been proposed. Initially, the 180 aerial images from INRIA Aerial Image Labelling Dataset were collected for landslides classification. The input aerial images are denoised using adaptive bilateral filter (ABF) to reduce the noise artifacts. The denoised images are fed into the SegNet to identify and separate the regions corresponding to different landslide areas. Then, the segmented regions are extracted using RegNet for efficient feature representation and enhanced discrimination among different types of landslides. Finally, the Deep Belief Network (DBN) is used for penta classification such as Rock falls, Debris flaws, Creep, Solifluction and earth slides. The performance analysis of the proposed PENTA-SERE model is evaluated in terms of accuracy, specificity, precision, recall and F1score based on the INRIA Aerial Image Labelling Dataset. The proposed PENTA-SERE model achieves an overall accuracy of 99.29%, and F1 Score of 96.62% based on the gathered dataset. The Proposed PENTA-SERE model improves the overall accuracy by 17.41, 27.45, 6.83, and 2.44% better than Bidirectional Encoder Representations from Transformers (BERT), Support Vector Machine (SVM), Deep convolutional neural network (DCNN), InceptionV3 respectively.

Small unmanned aerial vehicles applications had erupted in many fields including conservation management. Automatic object detection methods for such aerial imagery were in high demand to facilitate more efficient and economical wildlife management and research. This paper aimed to detect hoofed animals in aerial images taken from a quad-rotor in Southern Africa. Objects captured in this way were small both in absolute pixels and from an object-to-image ratio point of view, which were not perfectly suit for general purposed object detectors. We proposed a method based on the iconic Faster region-based convolutional neural networks (R-CNN) framework with atrous convolution layers in order to retain the spatial resolution of the feature map to detect small objects. A good choice of anchors was of prime importance in detecting small objects. The performance of the proposed Faster R-CNN with atrous convolutional filters in the backbone network was proven to be outstanding in our scenario by comparing to other object detection architectures.