Fast multidirectional vehicle detection on aerial images using region based convolutional neural networks

With the rapid development of Unmanned Aerial Vehicles, vehicle detection in aerial images plays an important role in different applications. Comparing with general object detection problems, vehicle detection in aerial images is still a challenging research topic since it is plagued by various unique factors, e.g. different camera angle, small vehicle size and complex background. In this paper, a Feature Fusion Deep-Projection Convolution Neural Network is proposed to enhance the ability to detect small vehicles in aerial images. The backbone of the proposed framework utilizes a novel residual block named stepwise res-block to explore high-level semantic features as well as conserve low-level detail features at the same time. A specially designed feature fusion module is adopted in the proposed framework to further balance the features obtained from different levels of the backbone. A deep-projection deconvolution module is used to minimize the impact of the information contamination introduced by down-sampling/up-sampling processes. The proposed framework has been evaluated by UCAS-AOD, VEDAI, and DOTA datasets. According to the evaluation results, the proposed framework outperforms other state-of-the-art vehicle detection algorithms for aerial images.

In the past few years, automatic building detection in aerial images has become an emerging field in computer vision. Detecting the specific types of houses will provide information in urbanization, change detection, and urban monitoring that play increasingly important roles in modern city planning and natural hazard preparedness. In this paper, we demonstrate the effectiveness of detecting various types of houses in aerial imagery using Faster Region-based Convolutional Neural Network (Faster-RCNN). After formulating the dataset and extracting bounding-box information, pre-trained ResNet50 is used to get the feature maps. The fully convolutional Region Proposal Network (RPN) first predicts the bounds and objectness score of objects (in this case house) from the feature maps. Then, the Region of Interest (RoI) pooling layer extracts interested regions to detect objects that are present in the images. To the best of our knowledge, this is the first attempt at detecting houses using Faster R-CNN that has achieved satisfactory results. This experiment opens a new path to conduct and extent the works not only for civil and environmental domain but also other applied science disciplines.

Small unmanned aerial vehicles applications had erupted in many fields including conservation management. Automatic object detection methods for such aerial imagery were in high demand to facilitate more efficient and economical wildlife management and research. This paper aimed to detect hoofed animals in aerial images taken from a quad-rotor in Southern Africa. Objects captured in this way were small both in absolute pixels and from an object-to-image ratio point of view, which were not perfectly suit for general purposed object detectors. We proposed a method based on the iconic Faster region-based convolutional neural networks (R-CNN) framework with atrous convolution layers in order to retain the spatial resolution of the feature map to detect small objects. A good choice of anchors was of prime importance in detecting small objects. The performance of the proposed Faster R-CNN with atrous convolutional filters in the backbone network was proven to be outstanding in our scenario by comparing to other object detection architectures.

Intelligent transportation systems (ITS) are the integration of information and communications technologies with applications which are significant in traffic control and management. The increased number of on road vehicles in urban areas urges the need of development of automated methods for traffic management. Vehicle identification, classification and analysis enable the intelligent transportation systems to make decisions. In this paper, an automated method for video analysis for vehicle identification using a modified Region based Convolution Neural Network (RCNN) has been proposed. The traffic videos collected by CCTV cameras installed on the roads are analyzed for vehicle identification in a given frame. The pretrained google net is used to extract features. These features are used by the Region based Convolution Neural Network for vehicle identification. The vehicles are identified using probability score computed using intersection of objects (IoU). The identified vehicles are classified into ten different vehicle classes. The proposed network concatenates features from previous layers to reduce loss and consequently improve the vehicle identification accuracy. The vehicle identification method is further extended for vehicle counting and behavioral analysis. The vehicle counting information can be used for congestion control in smart cities. The behavioral analysis includes computation of speed of vehicles. The speed information is useful for traffic law enforcement in smart cities. The proposed method is applied on MIO-TCD vehicle dataset and EBVT video dataset. The results are calculated using three different metrics namely average accuracy, mean precision and mean recall. Obtained results are also compared with other state of the art methods. The results show significant improvement and thus the method can be effectively used for video analysis.

One-stage object detection methods are usually more computationally efficient than two-stage methods, which makes it more likely to be applied in practice. However, one-stage methods often suffer from lower detection accuracies, especially when the objects to be detected are small. In this paper, we propose a multi-flow sub-network and multiple connections for single shot detection (MSSD), which is built upon a one-stage strategy to inherit the computational efficiency and improve the detection accuracy. The multi-flow sub-network in MSSD aims to extract high quality feature maps with high spatial resolution, sufficient non-linear transformation, and multiple receptive fields, which facilitates detection of small objects in particular. In addition, MSSD uses multiple connections, including up-sampling, down-sampling, and resolution-invariant connections, to combine feature maps of different layers, which helps the model capture fine-grained details and improve feature representation. Extensive experiments on PASCAL VOC and MS COCO demonstrate that MSSD achieves competitive detection accuracy with high computational efficiency compared to state-of-the-art methods. MSSD with input size of 320\(\,\times \,\)320 achieves 80.6% mAP on VOC2007 at 45 FPS and 29.7% mAP on COCO, both with a Nvidia Titan X GPU.

The image processing technologies have become a popular tool for biological related researches. In order to detect the specific animal from aerial videos, this paper attempts to use the region-based convolution neural network to implement stingray detection on aerial images obtained by UAV. The experimental shows that using Faster R-CNN algorithm as the target detector can achieve good detection accuracy with short computing time. It suggests that deep learning-based methods have considerable potential to aid real-time applications.

In the actual operation scenario, we usually want to measure the trajectory of the UAV performing the task. This can be achieved by many different means. For example, we can obtain the motion state estimation of UAV by obtaining sensor information such as GPS, RTK and IMU. Alternatively, the velocity and acceleration of the UAV can be measured and its displacement can be calculated by integration. The device (including hardware and algorithm) to complete this motion estimation is called odometry. Considering the efficiency and cost, we hope to complete the devise of odometry visually. The target of visual odometry (VO) is to estimate the motion of the camera according to the captured image. This paper presents a method based on feature extraction and incremental pose estimation, which can accurately calculate motion state of UAVs through aerial images taken by its own.

Agricultural productivity depends heavily on the economy. This stands as the main reason for the detection of the plant disease in the field of agriculture, because diseases in plants are quite natural. Crop diseases serve as a major food supply threat. The identification of diseases can lead to more rapid interventions in order to reduce the effects of plant disease. Automatic detection of plant leaf disease is beneficial because of the limited requirement of overviewing in large areas of crops and detects the diseases at an earliest, i.e., when they attack the plant leaves. The performance and accuracy level were quite low in existing system. Modern developments in deep learning have made drastic improvements in accuracy of object recognition. The proposed study aims at classifying the defect in the leaves of different plants using the plant leaf image. This is achieved using Faster Region based convolution neural network. The proposed method has the potential to identify various diseases possible in the plants.

This paper introduces an effective and fast deep learning model for detecting and classifying vehicles in aerial images to address typical problems of small size, background clutter and unfavorable weather. The AeroYOLOv3-c2f-LSKA pipeline is suggested, which combines improved pre-processing, a combination of handcrafted and deep features, optimized feature selection, attention-guided object detection, and transformer-based classification. The most important innovations are the application of biologically inspired hybrid optimization Piranha Foraging Optimization Algorithm-Secretary Bird Optimization (PFOA-SBO), for feature selection, and the addition of c2f and Large Separable Kernel Attention (LSKA) to the YOLOv3 backbone to enhance localization performance in complicated scenes. Multi-class vehicle classification is further improved using a Vision Transformer. Experimental analysis on VisDrone dataset demonstrates that the proposed model has a better performance with an accuracy of 99.81%, F1-score of 99.80% and inference speed of 41 FPS compared to YOLOv8 and other state-of-the-art models. These findings prove the applicability of the framework in real-time applications of aerial surveillance.

Offshore and inland river ship detection has been studied on both synthetic aperture radar (SAR) and optical remote sensing imagery. However, the classic ship detection methods based on SAR images can cause a high false alarm ratio and be influenced by the sea surface model, especially on inland rivers and in offshore areas. The classic detection methods based on optical images do not perform well on small and gathering ships. This paper adopts the idea of deep networks and presents a fast regional-based convolutional neural network (R-CNN) method to detect ships from high-resolution remote sensing imagery. First, we choose GaoFen-2 optical remote sensing images with a resolution of 1 m and preprocess the images with a support vector machine (SVM) to divide the large detection area into small regions of interest (ROI) that may contain ships. Then, we apply ship detection algorithms based on a region-based convolutional neural network (R-CNN) on ROI images. To improve the detection result of small and gathering ships, we adopt an effective target detection framework, Faster-RCNN, and improve the structure of its original convolutional neural network (CNN), VGG16, by using multiresolution convolutional features and performing ROI pooling on a larger feature map in a region proposal network (RPN). Finally, we compare the most effective classic ship detection method, the deformable part model (DPM), another two widely used target detection frameworks, the single shot multibox detector (SSD) and YOLOv2, the original VGG16-based Faster-RCNN, and our improved Faster-RCNN. Experimental results show that our improved Faster-RCNN method achieves a higher recall and accuracy for small ships and gathering ships. Therefore, it provides a very effective method for offshore and inland river ship detection based on high-resolution remote sensing imagery.

Object detection in aerial images is more difficult than general object detection. The main reasons are as follows: (1) aerial image object pixels are small and difficult to detect; (2) aerial image objects are sparsely distributed, which increases the difficulty of detection. In this paper, we propose an adaptive Clustered Detection network based on Clustered Detection, which can better solve the above-mentioned problems. The whole process of image detection is as follows: (1) given an aerial image, the adaptive clustering sub-network will output the coordinates of several candidate clustering areas and the corresponding confidence, and also output a suggested candidate clustering area the number of N; (2) the fusion network merges and divides the candidate clustering area according to the coordinates, confidence and N of the candidate clustering area, and outputs N segmented pictures; (3) these segmented pictures are accurately detected, the original pictures are globally detected, and the results of the precise detection and the global detection are merged to obtain the final detection result. Compared with the Clustered Detection, our network can adaptively determine the number of clustering regions in the picture, thereby improving the detection efficiency and accuracy.

Knowledge-driven building extraction method exhibits a restricted adaptability scope and is vulnerable to external factors that affect its extraction accuracy. On the other hand, data-driven building extraction method lacks interpretability, heavily relies on extensive training data, and may result in extraction outcomes with building boundary blur issues. The integration of pre-existing knowledge with data-driven learning is essential for the intelligent identification and extraction of buildings from high-resolution aerial images. To overcome the limitations of current deep learning building extraction networks in effectively leveraging prior knowledge of aerial images, a geometric significance-aware deep mutual learning network (GSDMLNet) is proposed. Firstly, the GeoSay algorithm is utilized to derive building geometric significance feature maps as prior knowledge and integrate them into the deep learning network to enhance the targeted extraction of building features. Secondly, a bi-directional guidance attention module (BGAM) is developed to facilitate deep mutual learning between the building feature map and the building geometric significance feature map within the dual-branch network. Furthermore, the deployment of an enhanced flow alignment module (FAM++) is utilized to produce high-resolution, robust semantic feature maps with strong interpretability. Ultimately, a multi-objective loss function is crafted to refine the network’s performance. Experimental results demonstrate that the GSDMLNet excels in building extraction tasks within densely populated and diverse urban areas, reducing misidentification of shadow-obscured regions and color-similar terrains lacking building structural features. This approach effectively ensures the precise acquisition of urban building information in aerial images.

Recent development in object detection are greatly driven by the success of region proposal approaches and region-based convolutional neural networks (R-CNNs). In this paper, we designed and implemented an object detection system using a faster-CNN method that shares full-image convolutional features with a detection network, so as to enable nearly cost-free region proposals. Development of this system is based on the previous work on Faster R-CNN. Results shows that with this method, we could achieve high accuracy while detecting objects.

Rotated object detection in aerial images is still challenging due to arbitrary orientations, large scale and aspect ratio variations, and extreme density of objects. Existing state-of-the-art rotated object detection methods mainly rely on angle-based detectors. However, angle-based detectors can easily suffer from a long-standing boundary problem. To tackle this problem, we propose a purely angle-free framework for rotated object detection, called Point RCNN. Point RCNN is a two-stage detector including both PointRPN and PointReg which are angle-free. Given an input aerial image, first, the backbone-FPN extracts hierarchical features, then, the PointRPN module generates an accurate rotated region of interests (RRoIs) by converting the learned representative points of each rotated object using the MinAreaRect function of OpenCV. Motivated by RepPoints, we designed a coarse-to-fine process to regress and refine the representative points for more accurate RRoIs. Next, based on the learned RRoIs of PointRPN, the PointReg module learns to regress and refine the corner points of each RRoI to perform more accurate rotated object detection. Finally, the final rotated bounding box of each rotated object can be attained based on the learned four corner points. In addition, aerial images are often severely unbalanced in categories, and existing rotated object detection methods almost ignore this problem. To tackle the severely unbalanced dataset problem, we propose a balanced dataset strategy. We experimentally verified that re-sampling the images of the rare categories can stabilize the training procedure and further improve the detection performance. Specifically, the performance was improved from 80.37 mAP to 80.71 mAP in DOTA-v1.0. Without unnecessary elaboration, our Point RCNN method achieved new state-of-the-art detection performance on multiple large-scale aerial image datasets, including DOTA-v1.0, DOTA-v1.5, HRSC2016, and UCAS-AOD. Specifically, in DOTA-v1.0, our Point RCNN achieved better detection performance of 80.71 mAP. In DOTA-v1.5, Point RCNN achieved 79.31 mAP, which significantly improved the performance by 2.86 mAP (from ReDet’s 76.45 to our 79.31). In HRSC2016 and UCAS-AOD, our Point RCNN achieved higher performance of 90.53 mAP and 90.04 mAP, respectively.

It is interesting to develop effective fish sampling techniques using underwater videos and image processing to automatically estimate and consequently monitor the fish biomass and assemblage in water bodies. Such approaches should be robust against substantial variations in scenes due to poor luminosity, orientation of fish, seabed structures, movement of aquatic plants in the background and image diversity in the shape and texture among fish of different species. Keeping this challenge in mind, we propose a unified approach to detect freely moving fish in unconstrained underwater environments using a Region-Based Convolutional Neural Network, a state-of-the-art machine learning technique used to solve generic object detection and localization problems. To train the neural network, we employ a novel approach to utilize motion information of fish in videos via background subtraction and optical flow, and subsequently combine the outcomes with the raw image to generate fish-dependent candidate regions. We use two benchmark datasets extracted from a large Fish4Knowledge underwater video repository, Complex Scenes dataset and the LifeCLEF 2015 fish dataset to validate the effectiveness of our hybrid approach. We achieve a detection accuracy (F-Score) of 87.44% and 80.02% respectively on these datasets, which advocate the utilization of our approach for fish detection task.