Improved Object Detection in UAV Images using Deep Learning

Abstract. This paper looks into the latest achievement in the low-cost Unmanned Aerial Vehicle (UAV) technology in their capacity to map the semi-development areas. The objectives of this study are to establish a new methodology or a new algorithm in image registration during interior orientation process and to determine the accuracy of the photogrammetric products by using UAV images. Recently, UAV technology has been used in several applications such as mapping, agriculture and surveillance. The aim of this study is to scrutinize the usage of UAV to map the semi-development areas. The performance of the low cost UAV mapping study was established on a study area with two image processing methods so that the results could be comparable. A non-metric camera was attached at the bottom of UAV and it was used to capture images at both sites after it went through several calibration steps. Calibration processes were carried out to determine focal length, principal distance, radial lens distortion, tangential lens distortion and affinity. A new method in image registration for a non-metric camera is discussed in this paper as a part of new methodology of this study. This method used the UAV Global Positioning System (GPS) onboard to register the UAV image for interior orientation process. Check points were established randomly at both sites using rapid static Global Positioning System. Ground control points are used for exterior orientation process, and check point is used for accuracy assessment of photogrammetric product. All acquired images were processed in a photogrammetric software. Two methods of image registration were applied in this study, namely, GPS onboard registration and ground control point registration. Both registrations were processed by using photogrammetric software and the result is discussed. Two results were produced in this study, which are the digital orthophoto and the digital terrain model. These results were analyzed by using the root mean square errors and mean absolute error to determine the level of accuracy and the precision of photogrammetric products. It can be concluded that the new method of image registration by using the GPS onboard of the UAV produces medium accuracy result compared to the method that uses the ground control point. This new method can be used for the medium accuracy requirements. Unmanned Aerial Vehicle can be used for several applications, which requires a medium accuracy.

Investigating post-earthquake surface ruptures is important for understanding the tectonics of seismogenic faults. The use of unmanned aerial vehicle (UAV) images to identify post-earthquake surface ruptures has the advantages of low cost, fast data acquisition, and high data processing efficiency. With the rapid development of deep learning in recent years, researchers have begun using it for image crack detection. However, due to the complex background and diverse characteristics of the surface ruptures, it remains challenging to quickly train an effective automatic earthquake surface rupture recognition model on a limited number of samples. This study proposes a workflow that applies an image segmentation algorithm based on convolutional neural networks (CNNs) to extract cracks from post-earthquake UAV images. We selected the 16-layer visual geometry group (VGG16) network as the primary network architecture. Then, we improved the VGG16 network and deleted several convolutional layers to reduce computation and memory consumption. Moreover, we added dilated convolution and atrous spatial pyramid pooling (ASPP) to make the network perform well in the surface crack identification of post-earthquake UAV images. We trained the proposed method using the data of the MS 7.4 Maduo earthquake and obtained a model that could automatically identify and draw small and irregular surface ruptures from high-resolution UAV images.

Unmanned Aerial Vehicles (UAVs) have revolutionized data acquisition across various domains, presenting immense potential for image processing and semantic segmentation. This literature review encompasses a thorough exploration of advancements, techniques, challenges, and datasets pertaining to UAV image semantic segmentation. It begins by introducing the fundamental concepts of UAVs, highlighting their pivotal role in capturing high-resolution imagery that serves diverse applications. The integration of deep learning algorithms with UAVs is emphasized, unlocking new horizons in autonomous flight, security, and environmental monitoring. Delving into the core principles of semantic segmentation, the review elucidates the critical task of classifying every pixel in an image. Convolutional Neural Networks (CNNs) are presented as the cornerstone technology, tracing their evolution from traditional CNNs to the highly adaptable Fully Convolutional Networks (FCNs). A substantial portion of the review is dedicated to FCNs, underscoring their ability to process images of varying dimensions while maintaining spatial coherence in the output. Their pivotal role in semantic segmentation, encompassing both classification and localization, is articulated. The subsequent sections delve into a comprehensive survey of state-of-the-art models, including SegNet, PSPNet, DeepLabNet, EfficientNet, DenseNet-C, and LinkNet. Each model's unique strengths and applications contribute to the evolving landscape of semantic segmentation tasks. The versatility of the U-Net architecture takes center stage in the latter parts of the review. Its fundamental structure is elucidated, followed by a comprehensive examination of its manifold adaptations—3D-U-Net, ResU-Net, U-Net++, Adversarial U-Net, Cascaded U-Net, and Improved U-Net 3+. These modifications address intrinsic challenges such as limited receptive fields and class imbalances, propelling U-Net to the forefront of image segmentation techniques. The subsequent sections pivot toward the application of U-Net in UAV image segmentation, illustrating its efficacy in diverse tasks, including land cover and crop classification. Nevertheless, persisting challenges, such as the scarcity of annotated datasets and the need for model generalization across varied environmental conditions, remain key areas of concern. The review culminates by underlining the significance of large, authentic datasets and data augmentation techniques. Furthermore, a brief exploration of publicly available UAV image datasets is presented, enhancing our understanding of the resources accessible for training and evaluating models. This comprehensive literature review encapsulates the dynamism of UAV image processing and semantic segmentation, illuminating recent developments and avenues for future research in this burgeoning field.

Bridges are an important part of railway infrastructure and need regular inspection and maintenance. Using unmanned aerial vehicle (UAV) technology to inspect railway infrastructure is an active research issue. However, due to the large size of UAV images, flight distance, and height changes, the object scale changes dramatically. At the same time, the elements of interest in railway bridges, such as bolts and corrosion, are small and dense objects, and the sample data set is seriously unbalanced, posing great challenges to the accurate detection of defects. In this paper, an adaptive cropping shallow attention network (ACSANet) is proposed, which includes an adaptive cropping strategy for large UAV images and a shallow attention network for small object detection in limited samples. To enhance the accuracy and generalization of the model, the shallow attention network model integrates a coordinate attention (CA) mechanism module and an alpha intersection over union (α-IOU) loss function, and then carries out defect detection on the bolts, steel surfaces, and railings of railway bridges. The test results show that the ACSANet model outperforms the YOLOv5s model using adaptive cropping strategy in terms of the total mAP (an evaluation index) and missing bolt mAP by 5% and 30%, respectively. Also, compared with the YOLOv5s model that adopts the common cropping strategy, the total mAP and missing bolt mAP are improved by 10% and 60%, respectively. Compared with the YOLOv5s model without any cropping strategy, the total mAP and missing bolt mAP are improved by 40% and 67%, respectively.

Information acquisition and rescue operation management can be facilitated by the use of unmanned aerial vehicles (UAVs) equipped with cameras and sensors that are capable of aerial photography and acquire important information from disaster-stricken areas. UAV technology is a relatively low-cost, flexible, and safe solution in missions that are difficult for humans or other systems. UAV usage in pre-and post-disaster assessment, response, and management is an active area of research. There are two important technical aspects of UAV applications in disaster situations. The first is the limited communication range. Information acquired and photography produced by UAVs are only transmitted in real-time to the ground control station (GCS) within its communication range. Remote real-time monitoring of distant areas beyond the UAV's communication range is not possible using a single UAV. It is only possible through communication among several UAVs. The second is the relatively short flight time due to limited battery capacity. UAV replacement is required in order to carry out a persistent monitoring mission. In this paper, we propose a new remote real-time monitoring system of a distant disaster area using multiple UAVs. Wireless communications between multiple UAVs constitutes a UAV network in our proposed system. Moreover, the proposed system replaces the UAVs in-flight with standby UAVs according to the remaining energy levels. We compare the performance of the proposed system with an existing UAV system. Our simulation results show that the proposed system can achieve efficient remote real-time monitoring.

In recent years, the use of unmanned aerial vehicle (UAV) platforms in civil and military applications has surged, highlighting the critical role of artificial intelligence (AI) embedded UAV systems in the future. This study introduces the Autonomous Drone (Vechür-SIHA), a novel AI-embedded UAV system designed for real-time detection and tracking of other UAVs during flight sequences. Leveraging advanced object detection algorithms and an LSTM-based tracking mechanism, our system achieves an impressive 80% accuracy in drone detection, even in challenging conditions like varying backgrounds and adverse weather. Our system boasts the capability to simultaneously track multiple drones within its field of view, maintaining flight for up to 35 minutes, making it ideal for extended missions that require continuous UAV tracking. Moreover, it can lock onto and track other UAVs in mid-air for durations of 4-10 seconds without losing contact, a feature with significant potential for security applications. This research marks a substantial contribution to the development of AI-embedded UAV systems, with broad implications across diverse domains such as search and rescue operations, border security, and forest fire prevention. These results provide a solid foundation for future research, fostering the creation of similar systems tailored to different applications, ultimately enhancing the efficiency and safety of UAV operations. The novel approach to real-time UAV detection and tracking presented here holds promise for driving innovations in UAV technology and its diverse applications.

In order to ensure the safety of transmission lines, the use of unmanned aerial vehicle (UAV) images for automatic object detection has important application prospects, such as the detection of birds’ nests. The traditional bird’s nest detection methods mainly include the study of morphological characteristics of the bird’s nest. These methods have poor applicability and low accuracy. In this work, we propose a deep learning-based birds’ nests automatic detection framework—region of interest (ROI) mining faster region-based convolutional neural networks (RCNN). First, the prior dimensions of anchors are obtained by using k-means clustering to improve the accuracy of coordinate boxes generation. Second, in order to balance the number of foreground and background samples in the training process, the focal loss function is introduced in the region proposal network (RPN) classification stage. Finally, the ROI mining module is added to solve the class imbalance problem in the classification stage, combined with the characteristics of difficult-to-classify bird’s nest samples in the UAV images. After parameter optimization and experimental verification, the deep learning-based bird’s nest automatic detection framework proposed in this work achieves high detection accuracy. In addition, the mean average precision (mAP) and formula 1 (F1) score of the proposed method are higher than the original faster RCNN and cascade RCNN. Our comparative analysis verifies the effectiveness of the proposed method.

The use of unmanned aerial vehicles (UAVs) for natural resource applications has increased considerably in recent years due to their greater availability, the miniaturization of sensors, and the ability to deploy a UAV relatively quickly and repeatedly at low altitudes. We examine in this paper the potential of using a small UAV for rangeland inventory, assessment and monitoring. Imagery with a ground resolved distance of 8 cm was acquired over a 290 ha site in southwestern Idaho. We developed a semiautomated orthorectification procedure suitable for handling large numbers of small-footprint UAV images. The geometric accuracy of the orthorectified image mosaics ranged from 1.5 m to 2 m. We used object-based hierarchical image analysis to classify imagery of plots measured concurrently on the ground using standard rangeland monitoring procedures. Correlations between imageand ground-based estimates of percent cover resulted in r-squared values ranging from 0.86 to 0.98. Time estimates indicated a greater efficiency for the image-based method compared to ground measurements. The overall classification accuracies for the two image mosaics were 83 percent and 88 percent. Even under the current limitations of operating a UAV in the National Airspace, the results of this study show that UAVs can be used successfully to obtain imagery for rangeland monitoring, and that the remote sensing approach can either complement or replace some ground-based measurements. We discuss details of the UAV mission, image processing and analysis, and accuracy assessment.

With the growing use of unmanned aerial vehicles (UAVs), it poses a potential threat to public safety, so drone detection has become an increasingly important research direction. However, the irregular shooting angles, deformations, significant target scale variations, and complex backgrounds in UAV imagery pose challenges for existing object detection models. To address these issues, we propose ALDNet, a precise and lightweight UAV image object detection network built on an improved RT-DETR model. First, we introduce the position accurate continuous convolution (PAC), which dynamically adjusts convolution kernel coordinates, allowing flexibility in detecting objects of varied shapes. PAC improve the ability of backbone network to capture objects with different shapes. Second, we present an enhanced multi-scale feature fusion approach that integrates spatial information more effectively, reducing the loss of fine-grained details during fusion. Additionally, we propose the Inner-Generalized Intersection over Union loss function, which reduces interference and accelerates the convergence of bounding box optimisation, thus improving the overall detection performance. We conducted experiments on three UAV image datasets, Det-Fly, ARD-MAV and VisDrone, and compared the ALDNet results with the RT-DETR model. The experimental results show that ALDNet achieved 98.3%, 97.4% and 48.1% mAP50 values on Det-Fly, ARD-MAV and VisDrone, which are 1.5%, 0.4% and 1.9% higher than RT-DETR, respectively, and the number of parameters and computation amount of ALDNet are 25% and 14% less than the baseline, respectively. Experimental results show that ALDNet achieves greater efficiency and effectiveness with less computational power.

This paper presents a novel automated road damage detection approach using Unmanned Aerial Vehicle (UAV) images and deep learning techniques. Maintaining road infrastructure is critical for ensuring a safe and sustainable transportation system. However, the manual collection of road damage data can be labor-intensive and unsafe for humans. Therefore, we propose using UAVs and Artificial Intelligence (AI) technologies to improve road damage detection’s efficiency and accuracy significantly. Our proposed approach utilizes three algorithms, YOLOv4, YOLOv5, and YOLOv7, for object detection and localization in UAV images. We trained and tested these algorithms using a combination of the RDD2022 dataset from China and a Spanish road dataset. The experimental results demonstrate that our approach is efficient and achieves 59.9% mean average precision mAP@.5 for the YOLOv5 version, 65.70% mAP@.5 for a YOLOv5 model with a Transformer Prediction Head, and 73.20% mAP@.5 for the YOLOv7 version. These results demonstrate the potential of using UAVs and deep learning for automated road damage detection and pave the way for future research in this field.

This study investigates the performance of the YOLOv8 object detection algorithm for fuel tank detection using unmanned aerial vehicles (UAVs) under various conditions and perspectives. UAVs, with their agile mobility and ability to capture high-quality images from different angles, are proven to facilitate the detection of fuel tanks, which play a critical role in various industries. The integration of UAVs with YOLOv8 has led to significant advancements in automation and precision in inspection processes within the energy and industrial sectors. YOLOv8, a single-stage deep learning-based object detection model, demonstrates superior architecture compared to previous YOLO iterations, showcasing its strengths in speed and accuracy for object detection tasks. The model achieves a precision of 0.888, recall of 0.896, and mAP of 0.891, confirming its strong capabilities in detecting fuel tanks and supporting the sustainability of industrial and energy operations. With a processing time of 41 ms, YOLOv8 proves to be highly effective for real-time applications. This research highlights the importance of optimizing UAVs and deep learning models for reliable data collection in challenging environments and demonstrates their potential for use in fuel tank monitoring and infrastructure reliability tracking across industries. While accurate detection successes are noted, the study emphasizes the need for further optimization of the algorithm to address false positives and undetected objects in real-world applications. Future work will explore the adaptation of high-performance algorithms such as YOLOv8 for broader object detection scenarios and their testing under diverse environmental conditions.

In this study, the trend of unmanned aerial vehicle (UAV) technology and its applications in the mining industry were reviewed. To understand the trend of the UAV technology, the definition and classification of the UAVs, their major applications in each field, the characteristics of the sensors of the UAV were investigated. Some examples that used the UAV technology for planning, work management, facility management, safety management, and other works in the mining industry were reviewed. By introducing the UAV technology in the mining site, it is expected that the work efficiency related to mining operations would be increased. Therefore, the application of the UAV technology in the mining industry would be expanded further.

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

Recent technological advance in UAV (Unmanned Aerial Vehicle) technology offers new opportunities for assessing crop situation using UAV imagery. The objective of this study was to assess if reflectance and NDVI derived from consumer-grade cameras mounted on UAVs are useful for crop condition monitoring. This study was conducted using a fixed-wing UAV(Ebee) with Cannon S110 camera from March 2015 to March 2016 in the experiment field of National Institute of Agricultural Sciences. Results were compared with ground-based recordings obtained from consumer-grade cameras and ground multi-spectral sensors. The relationship between raw digital numbers (DNs) of UAV images and measured calibration tarp reflectance was quadratic. Surface (lawn grass, stairs, and soybean cultivation area) reflectance obtained from UAV images was not similar to reflectance measured by ground-based sensors. But NDVI based on UAV imagery was similar to NDVI calculated by ground-based sensors. Change of NDVI in lawn grass and soybean cultivation area depending on method of reflectance calculation using UAV images.

The paper discusses the ways to increase the effectiveness and efficiency of green health care by using Unmanned Aerial Vehicle (UAV) technology. Using this technology at the center of care can significantly reduce costs and increase the effectiveness of a modern hospital’s services while enhancing efficiency and emphasizing environmental sustainability. This paper introduces UAVs as the future efficient method of real-time data transmission and trains them according to the circumstances to work more efficiently than manpower with better security. This can be the most significant method in the case of emergency handling. Where manually can’t be reached UAVs can solve the problem of bandwidth and transmitting data from lower to higher bandwidth. UAVs can be pre-trained and show acceptable accuracy to ensure green health care. By using the MobileNet model, the paper has implemented a mask detection process. It suggests a method in which UAVs are used to transport biological material from smaller hospitals and clinics to larger hospitals without any physical movement. The paper has discussed the UAV workflow and the UAV training process and specifically aims to show how technology can reduce inefficiencies caused by the under-utilization of professional resources and assets, as well as transportation.