AMCF-NET: ADAPTIVE MULTI-SCALE CROSS-MODAL FUSION NETWORK FOR UAV-SATELLITE CROSS-VIEW LOCALIZATION

Multi-view scene matching refers to the establishment of a mapping relationship between images captured from different perspectives, such as those taken by unmanned aerial vehicles (UAVs) and satellites. This technology is crucial for the geo-localization of UAV views. However, the geometric discrepancies between images from different perspectives, combined with the inherent computational constraints of UAVs, present significant challenges for matching UAV and satellite images. Additionally, the imbalance of positive and negative samples between drone and satellite images during model training can lead to instability. To address these challenges, this study proposes a novel and efficient cross-view geo-localization framework called MSM-Transformer. The framework employs the Dual Attention Vision Transformer (DaViT) as the core architecture for feature extraction, which significantly enhances the modeling capacity for global features and the contextual relevance of adjacent regions. The weight-sharing mechanism in MSM-Transformer effectively reduces model complexity, making it highly suitable for deployment on embedded devices such as UAVs and satellites. Furthermore, the framework introduces a contrastive learning-based Symmetric Decoupled Contrastive Learning (DCL) loss function, which effectively mitigates the issue of sample imbalance between satellite and UAV images. Experimental validation on the University-1652 dataset demonstrates that MSM-Transformer achieves outstanding performance, delivering optimal matching results with a minimal number of parameters.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

Agricultural intensification and efficient use and targeting of fertilizer inputs on smallholder farms is key to sustainably improve food security. The objective of this paper is to demonstrate how high-resolution satellite and unmanned aerial vehicle (UAV) images can be used to assess the spatial variability of yield, and yield response to fertilizer. The study included 48 and 50 smallholder fields monitored during the 2014 and 2015 cropping seasons south-east of Koutiala (Mali), cropped with the five major crops grown in the area (cotton, maize, sorghum, millet and peanuts). Each field included up to five plots with different fertilizer applications and one plot with farmer practice. Fortnightly, in-situ in each field data were collected synchronous with UAV imaging using a Canon S110 NIR camera. A concurrent series of very high-resolution satellite images was procured and these images were used to mask out trees. For each plot, we calculated vegetation index means, medians and coefficients of variation. Cross-validated general linear models were used to assess the predictability of relative differences in crop yield and yield response to fertilizer, explicitly accounting for the effects of fertility treatments, between-field and within-field variabilities. Differences between fields accounted for a much larger component of variation than differences between fertilization treatments.Vegetation indices from UAV images strongly related to ground cover (R2 = 0.85), light interception (R2 = 0.79) and vegetation indices derived from satellite images (R2 values of about 0.8). Within-plot distributions of UAV-derived vegetation index values were negatively skewed, and within-plot variability of vegetation index values was negatively correlated with yield. Plots on shallow soils with poor growing conditions showed the largest within-plot variability. GLM models including UAV derived estimates of light interception explained up to 78% of the variation in crop yield and 74% of the variation in fertilizer response within a single field. These numbers dropped to about 45% of the variation in yield and about 48% of the variation in fertilizer response when lumping all fields of a given crop, with Q2 values of respectively 22 and 40% respectively when tested with a leave-field-out procedure. This indicates that remotely sensed imagery doesn’t fully capture the influence of crop stress and management. Assessment of crop fertilizer responses with vegetation indices therefore needs a reference under similar management. Spatial variability in UAV-derived vegetation index values at the plot scale was significantly related to differences in yields and fertilizer responses. The strong relationships between light interception and ground cover indicate that combining vertical photographs or high-resolution remotely sensed vegetation indices with crop growth models allows to explicitly account for the spatial variability and will improve the accuracy of yield and crop production assessments, especially in heterogeneous smallholder conditions.

Object detection on unmanned aerial vehicle (UAV) images has been a hot research topic recently. However, object detection models for general scenarios struggle with UAV images due to the challenges of detecting small targets and handling complex image backgrounds. To solve the two issues, we proposed DPH-YOLOv8, an enhanced version of YOLOv8 tailored for UAV scenarios. First, we improved the prediction head from three prediction heads to double prediction heads (DPHs), reducing the model’s parameters by 42.9% while improving the mAP for small targets. Second, we designed a tiny-path feature fusion (TP-Fusion) module to fuse richer detailed information, enabling the detector to accurately match targets of different sizes and shapes. Third, we introduced a coordinate attention (CA) module to help the model reduce the interference of complex background information and focus on detecting foreground targets. Finally, bottleneck modules were added before the prediction heads to enhance the extraction of small target features. Extensive experimental results on both the VisDrone2021 and UAVDT benchmarks demonstrated that DPH-YOLOv8 not only improved the mAP by 4.5% on VisDrone2021 and 2.2% on UAVDT but also reduced localization error by 0.53%, confusion with objects by 0.69%, and confusion with background by 0.45%. These enhancements, along with a reduction in the model’s parameters, make DPH-YOLOv8 more suitable for UAV scenarios.

In recent years, research on Unmanned Aerial Vehicles (UAVs) has developed rapidly. Compared to traditional remote-sensing images, UAV images exhibit complex backgrounds, high resolution, and large differences in object scales. Therefore, UAV object detection is an essential yet challenging task. This paper proposes a multi-scale object detection network, namely YOLO-DroneMS (You Only Look Once for Drone Multi-Scale Object), for UAV images. Targeting the pivotal connection between the backbone and neck, the Large Separable Kernel Attention (LSKA) mechanism is adopted with the Spatial Pyramid Pooling Factor (SPPF), where weighted processing of multi-scale feature maps is performed to focus more on features. And Attentional Scale Sequence Fusion DySample (ASF-DySample) is introduced to perform attention scale sequence fusion and dynamic upsampling to conserve resources. Then, the faster cross-stage partial network bottleneck with two convolutions (named C2f) in the backbone is optimized using the Inverted Residual Mobile Block and Dilated Reparam Block (iRMB-DRB), which balances the advantages of dynamic global modeling and static local information fusion. This optimization effectively increases the model’s receptive field, enhancing its capability for downstream tasks. By replacing the original CIoU with WIoUv3, the model prioritizes anchoring boxes of superior quality, dynamically adjusting weights to enhance detection performance for small objects. Experimental findings on the VisDrone2019 dataset demonstrate that at an Intersection over Union (IoU) of 0.5, YOLO-DroneMS achieves a 3.6% increase in mAP@50 compared to the YOLOv8n model. Moreover, YOLO-DroneMS exhibits improved detection speed, increasing the number of frames per second (FPS) from 78.7 to 83.3. The enhanced model supports diverse target scales and achieves high recognition rates, making it well-suited for drone-based object detection tasks, particularly in scenarios involving multiple object clusters.

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

In this paper, we develop an alternative navigation system for Unmanned Aerial Vehicle (UAV) in Global Positioning Systems (GPS)-denied environment. We use two image inputs, one is acquired with an on-board camera placed on the UAV (which is the large-area image) and the other is from satellite images (which is small known image) with GPS information. We use a convolutional neural network (CNN) architecture based on Oxford’s Visual Geometry Group network (VGG-16) and utilize normalized variant mutual information between these two images to obtain position of the UAV. Satellite images are labelled and given to the UAV. When GPS information is lost, our algorithm starts to function and images from UAV camera are searched whether satellite image is seen by cameras on UAV image or not. If the UAV is in that area, our algorithm finds the GPS information from satellite image data.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Recent advances in structure from motion (SfM) and dense matching algorithms enable surface reconstruction from unmanned aerial vehicle (UAV) images with high spatial resolution, allowing for new insights into earth surface processes. However, accuracy issues are inherent in parallel‐axes UAV image configurations. In this study, the quality of digital elevation models (DEMs) is assessed using images from a simulated UAV flight. Five different SfM tools and three different cameras are compared. If ground control points (GCPs) are not integrated into the adjustment process with parallel‐axes image configurations, significant dome‐effect systematic errors are observed, which can be reduced based on calibration parameters retrieved from a testfield captured with convergent images immediately before or after the UAV flight. A comparison between DEMs of a soil surface generated from UAV images and terrestrial laser‐scanning data show that natural surfaces can be very accurately reconstructed from UAV images, even when GCPs are missing and simple geometric camera models are considered.

This paper presents a novel approach to automatically detect and count cars in different aerial images, which can be satellite or unmanned aerial vehicle (UAV) images. Variations in satellite and/or UAV data make it particularly challenging to have a robust method that works properly on a variety of images. A solution based on the bag-of-words (BoW) model is explored in this paper due to its invariance characteristic and highly stable performance in object/scene categorization. Different from categorization tasks, vehicle detection needs to localize the positions of cars in images. To make BoW suitable for this purpose, we extensively improve the methodology in three aspects, namely, by introducing a recently proposed feature representation, i.e., the local steering kernel descriptor, adding spatial structure constraints, and developing an orientation aware scanning mechanism to produce detection with “one-window-one-car” results. Experiments are conducted on various aerial images with large variations, which consist of data from two public databases, e.g., the Overhead Imagery Research Data Set and Vehicle Detection in Aerial Imagery, as well as other satellite and UAV images. The results demonstrate the effectiveness and robustness of the proposed method. Compared with existing techniques, the proposed method is applicable to a wider range of aerial images.

Unmanned Aerial Vehicle (UAV) image object detection plays a crucial role in various fields. However, compared with natural images, UAV images are characterized by significant target scale variations, complex backgrounds, dense small targets, and clustered target distributions, which pose serious challenges to object detection tasks. To address these issues, this study proposes an improved object detection algorithm named MFRA-YOLO, which combines multi-scale feature fusion and receptive-field attention-based convolution, built upon the baseline YOLOv8n algorithm. First, Monte Carlo attention is integrated into receptive-field attention-based convolution to enhance cross-scale information interaction capability, thereby forming a novel convolutional module for downsampling operations. Second, a multi-scale selective fusion module is incorporated into the feature fusion network to enable adaptive cross-scale integration of features. When coupled with the scale sequence feature fusion module, this integration significantly enhances the detection performance for small targets. Finally, the Focaler-PIoUv2 loss function is designed to replace the CIoU in the baseline algorithm. This replacement allows the algorithm to better balance hard and easy samples and improve detection accuracy. Experimental results on the public dataset VisDrone2019 show that MFRA-YOLO achieves superior accuracy-efficiency tradeoffs compared to the baseline YOLOv8n and other YOLO variants. Compared to YOLOv8n, MFRA-YOLO improves mAP50 by 3.5% and mAP50:95 by 2.3%, respectively. These performance gains are accompanied by only modest increases in parameter count and computational cost. Notably, while maintaining performance at 143 FPS, thereby satisfying the real-time deployment requirements for UAVs. Furthermore, compared with several state-of-the-art algorithms designed for UAV scenarios, MFRA-YOLO also offers distinct advantages. To verify the generalization ability and stability of MFRA-YOLO, comparative experiments are carried out on the RSOD dataset, where our algorithm consistently demonstrates excellent detection performance. Overall, these results confirm that MFRA-YOLO not only improves the detection performance for UAV imagery substantially but also achieves an excellent balance between detection accuracy and efficiency.

Remote sensing images often encounter various challenges arising from differences in shooting time, location, equipment, sensors, and other factors. These disparities lead to image distortion and insufficient overlap between pairs of images captured at the same location. Consequently, the accuracy of agricultural remote sensing image registration is significantly compromised. This paper proposes a robust feature matching technique called GAN-based neighborhood representation (RFM-GAN) for the intricate registration of satellite images and unmanned aerial vehicle (UAV) images. The RFM-GAN method leverages a neighborhood representation approach based on a generative adversarial network (GAN) with two discriminators. This representation enhances the distinction between true matches (inliers) and false matches (outliers). Additionally, a dissimilarity measure network employing a self-supervised training approach, eliminating the need for manual labeling, is designed to handle the multi-view transformation of satellite and UAV images. The experimental results confirm that RFM-GAN outperforms seven other state-of-the-art methods in terms of satellite image and UAV image processing.

The Chuprene Reserve was created in 1973 to preserve the natural coniferous forests in the Western Balkan Range in Bulgaria. The first infestations by European spruce bark beetle (Ips typographus) were registered in Norway spruce (Picea abies) stands in the mid-1980s. The aim of this study is to assess the damages caused by I. typographus in the Chuprene Reserve using remote sensing techniques – unmanned aerial vehicle (UAV) images, airborne images, and satellite images of European Space Imaging (EUSI), combined with terrestrial verification. High-resolution images in four bands of the electromagnetic spectrum and in a standard RGB channel were taken in 2017 via a multispectral camera ‘Parrot Sequoia’, integrated with a specialized professional UAV system eBee ‘Flying Wing’. The health status of Norway spruce stands in the reserve was assessed with the normalized difference vegetation index, based on the digital mixing of imagery captured in the red and near infrared range. The dynamic of bark beetle attacks was studied in GIS on the basis of maps generated from photographic surveys, airborne images taken in 2011 and 2015, and satellite images from 2020. In the UAV-captured area (314.0 ha), the size of Norway spruce stands attacked by I. typographus increased from 7.6 ha (2.4%) in 2011 to 44.9 ha (14.3%) in 2020. The satellite images showed that on the entire territory of the Chuprene Reserve (1451.9 ha), I. typographus killed spruce trees on 137.4 ha, which is 9.6% of the total area.

Target detection based on unmanned aerial vehicle (UAV) images has increasingly become a hot topic with the rapid development of UAVs and related technologies. UAV aerial images often feature a large number of small targets and complex backgrounds due to the UAV’s flying height and shooting angle of view. These characteristics make the advanced YOLOv4 detection method lack outstanding performance in UAV aerial images. In light of the aforementioned problems, this study adjusted YOLOv4 to the image’s characteristics, making the improved method more suitable for target detection in UAV aerial images. Specifically, according to the characteristics of the activation function, different activation functions were used in the shallow network and the deep network, respectively. The loss for the bounding box regression was computed using the EIOU loss function. Improved Efficient Channel Attention (IECA) modules were added to the backbone. At the neck, the Spatial Pyramid Pooling (SPP) module was replaced with a pyramid pooling module. At the end of the model, Adaptive Spatial Feature Fusion (ASFF) modules were added. In addition, a dataset of forklifts based on UAV aerial imagery was also established. On the PASCAL VOC, VEDAI, and forklift datasets, we ran a series of experiments. The experimental results reveal that the proposed method (YOLO-DRONE, YOLOD) has better detection performance than YOLOv4 for the aforementioned three datasets, with the mean average precision (mAP) being improved by 3.06%, 3.75%, and 1.42%, respectively.

Current irrigation management zones (IMZs) for variable rate irrigation (VRI) systems are static. They are delineated in the beginning of the season and used thereafter. However, recent research has shown that IMZ boundaries are transient and change with time during the growing season. The primary goal of this study was to explore the potential of using vegetation indices (VIs) developed from unmanned aerial vehicle (UAV) and satellite images to predict cotton physiological parameters that can be used to delineate in-season boundaries of IMZs. A 2 year study was conducted in a 38 ha commercial cotton field in southwestern Georgia, USA. Throughout the two growing seasons, VIs were calculated from UAV, PlanetScope, and Sentinel-2 images. Predawn leaf water potential (LWPPD) and plant height were measured at 37 locations in the field on the same day as the flights and correlated with UAV and satellite based-VIs. GNDVI (Green normalized difference vegetation index) was the best predictor of plant height with correlation values of 0.72 (p < .0001) and 0.84 (p < .0001) for 2019 and 2020, respectively. A secondary goal was to compare the performance of dynamic VRI (DVRI) to conventional irrigation. The field was divided into alternating parallel conventional, and DVRI strips to compare the two scheduling methods. The conventional strips were irrigated using the farmer’s standard method and individual IMZs within the DVRI strips were irrigated based on soil water tension (SWT) measured with a wireless soil moisture sensor network. LWP and SWT measurements correlated well. IMZs were initially delineated using soil texture, apparent soil electrical conductivity (ECa), and yield maps and satellite images from previous years and were modified in-season to reflect patterns observed in the plant height maps. In 2020, the DVRI system prescribed an average irrigation amount of 50.8 mm while conventional irrigation applied an average of 58.4 mm. Average yields for DVRI and conventional were 1248 and 1191 kg ha−1, respectively. The DVRI system resulted in average yield 4.6% higher than conventional irrigation, while applying 14.0% less water. Despite the lower water application by the DRVI system, the performance comparison between the DRVI and the conventional irrigation was not conclusive.