An improved UAV image object detection algorithm combining multi-scale feature fusion and receptive-field attention-based convolution

Object detection on unmanned aerial vehicle (UAV) images has been a hot research topic recently. However, object detection models for general scenarios struggle with UAV images due to the challenges of detecting small targets and handling complex image backgrounds. To solve the two issues, we proposed DPH-YOLOv8, an enhanced version of YOLOv8 tailored for UAV scenarios. First, we improved the prediction head from three prediction heads to double prediction heads (DPHs), reducing the model’s parameters by 42.9% while improving the mAP for small targets. Second, we designed a tiny-path feature fusion (TP-Fusion) module to fuse richer detailed information, enabling the detector to accurately match targets of different sizes and shapes. Third, we introduced a coordinate attention (CA) module to help the model reduce the interference of complex background information and focus on detecting foreground targets. Finally, bottleneck modules were added before the prediction heads to enhance the extraction of small target features. Extensive experimental results on both the VisDrone2021 and UAVDT benchmarks demonstrated that DPH-YOLOv8 not only improved the mAP by 4.5% on VisDrone2021 and 2.2% on UAVDT but also reduced localization error by 0.53%, confusion with objects by 0.69%, and confusion with background by 0.45%. These enhancements, along with a reduction in the model’s parameters, make DPH-YOLOv8 more suitable for UAV scenarios.

Less effective information is obtained by the object detection network, due to the small size of the detection object in the entire image, the complex background, and the dense object in unmanned aerial vehicle (UAV) images. In response to the difficulties encountered, a small object detection method in UAV images is proposed as an improved YOLOv5-based algorithm in this paper. First, the space-to-depth(SPD) conv module is introduced into the basic feature extraction network, to improve significant loss of image information during downsampling. Then, various attention mechanisms are added, to intensify the acquisition of regions of interest in UAV images. Finally, the multiscale detection module is improved, to enhance the network's ability to detect small objects in UAV images. By conducting experiments on the VisDrone-DET2019 dataset, the test results of the established model show. The improved algorithm achieved a Mean Average Precision (mAP) of 41.8%, which is 7.8% better than the baseline network. In addition, the detection performance is better than most current mainstream target detection algorithms and is of some practical value.

Unmanned aerial vehicles (UAVs) have been widely used in post-disaster search and rescue operations, object tracking, and other tasks. Therefore, the autonomous perception of UAVs based on computer vision has become a research hotspot in recent years. However, UAV images include dense objects, small objects, and arbitrary object directions, which bring about significant challenges to existing object detection methods. To alleviate these issues, we propose a global-local feature enhanced network (GLF-Net). Considering the difficulty of processing UAV images with complex scenes and dense objects, we designed a backbone based on an involution and self-attention that can extract effective features from complex objects. A multiscale feature fusion module is also proposed to address the presence of numerous small objects in UAV images through multiscale object detection and feature fusion. To accurately detect rotated objects, a rotated regional proposal network was designed based on the midpoint offset representation, which can apply a rotated box to determine the real direction and contour of an object. GLF-Net achieves a state-of-the-art detection accuracy (86.52% mAP) on our created RO-UAV dataset, while achieving 96.95% and 97% mAP on the public datasets HRSC2016 and UCAS-AOU, respectively. The experimental results demonstrate that our method achieves a high detection accuracy and generalization, which can meet the practical requirements of UAVs under various complex scenarios.

General deep learning methods achieve high-level semantic feature representation by aggregating hierarchical features, which performs well in object detection tasks. However, issues arise with general deep learning methods in UAV-based remote sensing image object detection tasks. Firstly, general feature aggregation methods such as stride convolution may lead to information loss in input samples. Secondly, common FPN methods introduce conflicting information by directly fusing feature maps from different levels. These shortcomings limit the model’s detection performance on small and weak targets in remote sensing images. In response to these concerns, we propose an unmanned aerial vehicle (UAV) object detection algorithm, IF-YOLO. Specifically, our algorithm leverages the Information-Preserving Feature Aggregation (IPFA) module to construct semantic feature representations while preserving the intrinsic features of small objects. Furthermore, to filter out irrelevant information introduced by direct fusion, we introduce the Conflict Information Suppression Feature Fusion Module (CSFM) to improve the feature fusion approach. Additionally, the Fine-Grained Aggregation Feature Pyramid Network (FGAFPN) facilitates interaction between feature maps at different levels, reducing the generation of conflicting information during multi-scale feature fusion. The experimental results on the VisDrone2019 dataset demonstrate that in contrast to the standard YOLOv8-s, our enhanced algorithm achieves a mean average precision (mAP) of 47.3%, with precision and recall rates enhanced by 6.3% and 5.6%, respectively.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

In recent years, research on Unmanned Aerial Vehicles (UAVs) has developed rapidly. Compared to traditional remote-sensing images, UAV images exhibit complex backgrounds, high resolution, and large differences in object scales. Therefore, UAV object detection is an essential yet challenging task. This paper proposes a multi-scale object detection network, namely YOLO-DroneMS (You Only Look Once for Drone Multi-Scale Object), for UAV images. Targeting the pivotal connection between the backbone and neck, the Large Separable Kernel Attention (LSKA) mechanism is adopted with the Spatial Pyramid Pooling Factor (SPPF), where weighted processing of multi-scale feature maps is performed to focus more on features. And Attentional Scale Sequence Fusion DySample (ASF-DySample) is introduced to perform attention scale sequence fusion and dynamic upsampling to conserve resources. Then, the faster cross-stage partial network bottleneck with two convolutions (named C2f) in the backbone is optimized using the Inverted Residual Mobile Block and Dilated Reparam Block (iRMB-DRB), which balances the advantages of dynamic global modeling and static local information fusion. This optimization effectively increases the model’s receptive field, enhancing its capability for downstream tasks. By replacing the original CIoU with WIoUv3, the model prioritizes anchoring boxes of superior quality, dynamically adjusting weights to enhance detection performance for small objects. Experimental findings on the VisDrone2019 dataset demonstrate that at an Intersection over Union (IoU) of 0.5, YOLO-DroneMS achieves a 3.6% increase in mAP@50 compared to the YOLOv8n model. Moreover, YOLO-DroneMS exhibits improved detection speed, increasing the number of frames per second (FPS) from 78.7 to 83.3. The enhanced model supports diverse target scales and achieves high recognition rates, making it well-suited for drone-based object detection tasks, particularly in scenarios involving multiple object clusters.

Small object detection is a challenging computer vision problem due to their low feature representation in the images and factors such as occlusions and noise. In images captured from a camera mounted on an unmanned aerial vehicle (UAV), objects are usually acquired in small sizes depending on the UAV flight altitude. The state-of-the-art object detectors often have lower detection accuracy with small objects. New approaches of combining features at multi-levels in the network helps in improving the object detection performance. In this paper, we propose a multi-scale approach of low-level feature combinations with deconvolutional modules on a single shot multibox detection (SSD) object detector to improve the small object detection in images acquired from a UAV. The proposed SSD based architecture is evaluated on UAV datasets to compare its performance with the state-of-the-art detectors.

Accurate object detection in unmanned aerial vehicle (UAV) images plays a crucial role in fields such as aviation, transportation, and agriculture. However, UAV images often contain a high proportion of small objects, and the limited resources of UAV platforms make it challenging for existing small object detection algorithms to balance detection performance and resource consumption. To address these issues, a lightweight small object detection algorithm called SRP-YOLO is proposed. Compared to the YOLOv8 network, the following improvements are made: first, a high-resolution detection head is designed; second, a receptive field attention convolution module tailored for small UAV objects is introduced to replace the standard convolution module, enhancing the detection capability for small objects and significantly improving their feature representation; Third, partial convolution is combined with C2f to replace some of the standard convolutions, fully leveraging the high-resolution information from shallow features. Finally, A normalized Gaussian Wasserstein Distance (NWD) metric is introduced to reduce the sensitivity of Intersection over Union (IoU) to minor positional deviations of small objects. On VisDrone-DET2019, SRP-YOLO reduces parameters while improving mAP 0.5 by 7.9%. Generalization tests on TinypersonV2 and RAIVD confirm significant gains in small object detection.

With special consideration for complex scenes and densely distributed small objects, this frequently leads to serious false and missed detections for unmanned aerial vehicle (UAV) images in small object detection scenarios. Consequently, we propose a UAV image small object detection algorithm, termed SMA-YOLO. Firstly, a parameter-free simple slicing convolution (SSC) module is integrated in the backbone network to slice the feature maps and enhance the features so as to effectively retain the features of small objects. Subsequently, to enhance the information exchange between upper and lower layers, we design a special multi-cross-scale feature pyramid network (M-FPN). The C2f-Hierarchical-Phantom Convolution (C2f-HPC) module in the network effectively reduces information loss by fine-grained multi-scale feature fusion. Ultimately, adaptive spatial feature fusion detection Head (ASFFDHead) introduces an additional P2 detection head to enhance the resolution of feature maps to better locate small objects. Moreover, the ASFF mechanism is employed to optimize the detection process by filtering out information conflicts during multi-scale feature fusion, thereby significantly optimizing small object detection capability. Using YOLOv8n as the baseline, SMA-YOLO is evaluated on the VisDrone2019 dataset, achieving a 7.4% improvement in mAP@0.5 and a 13.3% reduction in model parameters, and we also verified its generalization ability on VAUDT and RSOD datasets, which demonstrates the effectiveness of our approach.

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

With the development of unmanned aerial vehicle (UAV) remote sensing technology, target detection based on UAV images has increasingly become a hot spot for research. Aiming at the problems of many small target instances, complex backgrounds and difficult feature extraction in UAV images, we propose a UAV aerial image target detection algorithm called BLUR-YOLO. First, the h-swish activation function is used in the backbone network and the neck network to increase the expressiveness of the model. Second, an attention mechanism (CoordAttention) is added to the bottleneck layer of the backbone network, thereby increasing the weight of valid information and suppressing background noise interference. Finally, by removing redundant nodes of the path aggregation network (PANet), adding additional connections, and using BlurPool instead of the downsampling method, a feature pyramid network (Blur-PANet) is proposed to effectively fuse multilayer features. Experimental results on the VisDrone public dataset show that the proposed drone image object detection model (BLUR-YOLO) is 1.2% better than the YOLOv4 algorithm, which proves the effectiveness of the method.

With the growing use of unmanned aerial vehicles (UAVs), it poses a potential threat to public safety, so drone detection has become an increasingly important research direction. However, the irregular shooting angles, deformations, significant target scale variations, and complex backgrounds in UAV imagery pose challenges for existing object detection models. To address these issues, we propose ALDNet, a precise and lightweight UAV image object detection network built on an improved RT-DETR model. First, we introduce the position accurate continuous convolution (PAC), which dynamically adjusts convolution kernel coordinates, allowing flexibility in detecting objects of varied shapes. PAC improve the ability of backbone network to capture objects with different shapes. Second, we present an enhanced multi-scale feature fusion approach that integrates spatial information more effectively, reducing the loss of fine-grained details during fusion. Additionally, we propose the Inner-Generalized Intersection over Union loss function, which reduces interference and accelerates the convergence of bounding box optimisation, thus improving the overall detection performance. We conducted experiments on three UAV image datasets, Det-Fly, ARD-MAV and VisDrone, and compared the ALDNet results with the RT-DETR model. The experimental results show that ALDNet achieved 98.3%, 97.4% and 48.1% mAP50 values on Det-Fly, ARD-MAV and VisDrone, which are 1.5%, 0.4% and 1.9% higher than RT-DETR, respectively, and the number of parameters and computation amount of ALDNet are 25% and 14% less than the baseline, respectively. Experimental results show that ALDNet achieves greater efficiency and effectiveness with less computational power.

This study presents a modified YOLOv5 algorithm specifically designed to enhance small‐object detection in unmanned aerial vehicle (UAV) images. Traditional object detection in UAV images is particularly challenging due to the high altitude of the cameras, which results in small object sizes and varying viewing angles. To address these challenges, the algorithm incorporates an additional prediction head to detect objects across a wide range of scales, a channel feature fusion with involution (CFFI) block to minimize information loss, a convolutional block attention module (CBAM) to highlight the crucial spatial and channel features, and a C3 structure with a Transformer block (C3TR) to capture contextual information. The algorithm additionally employs soft non‐maximum suppression to enhance the bounding box scoring of overlapping objects in dense scenes. Extensive experiments were conducted on the VisDrone‐DET2019 dataset, which demonstrated the effectiveness of the proposed algorithm. The results showed improvements with precision scores of 55.0%, recall scores of 44.6%, mean average precision scores of mAP50 = 50.9% and mAP50:95 = 33.0% on the VisDrone‐DET2019 validation set, and precision of 50.8%, recall of 37.3%, mAP50 = 44.2%, and mAP50:95 = 27.3% on the VisDrone‐DET2019 testing set. The improved performance is due to the incorporation of attention mechanisms, which allow the proposed model to stay lightweight while still extracting the features needed to detect small objects.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Recent advances in structure from motion (SfM) and dense matching algorithms enable surface reconstruction from unmanned aerial vehicle (UAV) images with high spatial resolution, allowing for new insights into earth surface processes. However, accuracy issues are inherent in parallel‐axes UAV image configurations. In this study, the quality of digital elevation models (DEMs) is assessed using images from a simulated UAV flight. Five different SfM tools and three different cameras are compared. If ground control points (GCPs) are not integrated into the adjustment process with parallel‐axes image configurations, significant dome‐effect systematic errors are observed, which can be reduced based on calibration parameters retrieved from a testfield captured with convergent images immediately before or after the UAV flight. A comparison between DEMs of a soil surface generated from UAV images and terrestrial laser‐scanning data show that natural surfaces can be very accurately reconstructed from UAV images, even when GCPs are missing and simple geometric camera models are considered.