Software design for image mapping and analytics for high throughput phenotyping

This paper presents a novel approach to automatically detect and count cars in different aerial images, which can be satellite or unmanned aerial vehicle (UAV) images. Variations in satellite and/or UAV data make it particularly challenging to have a robust method that works properly on a variety of images. A solution based on the bag-of-words (BoW) model is explored in this paper due to its invariance characteristic and highly stable performance in object/scene categorization. Different from categorization tasks, vehicle detection needs to localize the positions of cars in images. To make BoW suitable for this purpose, we extensively improve the methodology in three aspects, namely, by introducing a recently proposed feature representation, i.e., the local steering kernel descriptor, adding spatial structure constraints, and developing an orientation aware scanning mechanism to produce detection with “one-window-one-car” results. Experiments are conducted on various aerial images with large variations, which consist of data from two public databases, e.g., the Overhead Imagery Research Data Set and Vehicle Detection in Aerial Imagery, as well as other satellite and UAV images. The results demonstrate the effectiveness and robustness of the proposed method. Compared with existing techniques, the proposed method is applicable to a wider range of aerial images.

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

Aim. Determination of the elements of external spatial orientation of the surveying systems at the moment of image acquisition is the fundamental task in photogrammetry. Principally, this problem is solving in two ways. The first way is direct positioning and measuring of directions of camera optical axis in the geodetic space with the help of GNSS/INS equipment. The second way is the analytical solution of the problem using a set of reference information (often such information is a set of ground control points whose geodetic positions are known with sufficient accuracy and which are reliably recognised on aerial images of the photogrammetric block). The authors consider the task of providing reference and control information using the second approach, which has a number of advantages in terms of reliability and accuracy of determining the unknown image exterior orientation parameters. It is proposed to obtain additional images of ground control points by the method of their auxiliary aerial photography using an unmanned aerial vehicle (UAV) on a larger scale compared to the scale of the images of the photogrammetric block. The aim of the presented work is the implementation of the method of creating reference points and experimental confirmation of its effectiveness for photogrammetric processing. Methods and results. For the entire realization of the potential of the analytical way to determine the elements of external orientation of images, it is necessary to have a certain number of ground control points (GCP) and to keep the defined scheme of their location on the photogrammetric block. As the main source of input data authors use UAV aerial images of the terrain, which are obtained separately from the block of aerial survey, and have a better geometric resolution and which clearly depict the control reference points. Application of such auxiliary images gives the possibility of automated transferring of the position of ground control point into images of the main photogrammetric block. In our interpretation, these images of ground control points and their surroundings on the ground are called "control reference images". The basis of the work is to develop a method for obtaining the auxiliary control reference images and transferring of position of GCP depicted on them into aerial or space images of terrain by means of computer stereo matching. To achieve this goal, we have developed a processing method for the creation of control reference images of aerial image or a series of auxiliary multi-scale aerial images obtained by a drone from different heights above the reference point. The operator identifies and measures the GCP once on the auxiliary aerial image of the highest resolution. Then there is an automatic stereo matching of the control reference image in the whole series of auxiliary images in succession with a decrease in the resolution and, ultimately, directly with the aerial images of photogrammetric block. On this stage there are no recognition/cursor targeting by the human operator, and therefore there are no discrepancies, errors or mistakes related to it. In addition, if to apply fairly large size of control reference images, the proposed method can be used on a low-texture terrain, and therefore deal in many cases without the physical marking of points measured by GNSS method. And this is a way to simplify and reduce the cost of photogrammetric technology. The action of the developed method has been verified experimentally to provide the control reference information of the block of archival aerial images of the low-texture terrain. The results of the experimental approbation of the proposed method give grounds to assert that the method makes it possible to perform geodetic reference of photogrammetric projects more efficiently due to the refusal of the physical marking of the area before aerial survey. The proposed method can also be used to obtain the information for checking the quality of photogrammetric survey for provision of check points. The authors argue that the use of additional equipment - UAV of semi-professional class to obtain control reference images is economically feasible. Scientific novelty and practical relevance. The results of approbation of the "control reference image" method with obtaining stereo pairs of aerial images with vertical placement of the base are presented for the first time. There was implemented the study of the properties of such stereo pairs of aerial images to obtain images of reference points. The effectiveness of including reference images in the main block of the digital aerial triangulation network created on UAV’s images is shown.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Recent advances in structure from motion (SfM) and dense matching algorithms enable surface reconstruction from unmanned aerial vehicle (UAV) images with high spatial resolution, allowing for new insights into earth surface processes. However, accuracy issues are inherent in parallel‐axes UAV image configurations. In this study, the quality of digital elevation models (DEMs) is assessed using images from a simulated UAV flight. Five different SfM tools and three different cameras are compared. If ground control points (GCPs) are not integrated into the adjustment process with parallel‐axes image configurations, significant dome‐effect systematic errors are observed, which can be reduced based on calibration parameters retrieved from a testfield captured with convergent images immediately before or after the UAV flight. A comparison between DEMs of a soil surface generated from UAV images and terrestrial laser‐scanning data show that natural surfaces can be very accurately reconstructed from UAV images, even when GCPs are missing and simple geometric camera models are considered.

Ultra-reliable intelligent link scheduling based on DRL for manned/unmanned aerial vehicle cooperative scenarios

Climate change intensifies biotic and abiotic stresses, threatening global crop productivity. High-throughput phenotyping (HTP) technologies provide a non-destructive approach to monitor plant responses to environmental stresses, offering new opportunities for both crop stress resilience and breeding research. Innovations, such as hyperspectral imaging, unmanned aerial vehicles, and machine learning, enhance our ability to assess plant traits under various environmental stresses, including drought, salinity, extreme temperatures, and pest and disease infestations. These tools facilitate the identification of stress-tolerant genotypes within large segregating populations, improving selection efficiency for breeding programs. HTP can also play a vital role by accelerating genetic gain through precise trait evaluation for hybridization and genetic enhancement. However, challenges such as data standardization, phenotyping data management, high costs of HTP equipment, and the complexity of linking phenotypic observations to genetic improvements limit its broader application. Additionally, environmental variability and genotype-by-environment interactions complicate reliable trait selection. Despite these challenges, advancements in robotics, artificial intelligence, and automation are improving the precision and scalability of phenotypic data analyses. This review critically examines the dual role of HTP in assessment of plant stress tolerance and crop performance, highlighting both its transformative potential and existing limitations. By addressing key challenges and leveraging technological advancements, HTP can significantly enhance genetic research, including trait discovery, parental selection, and hybridization scheme optimization. While current methodologies still face constraints in fully translating phenotypic insights into practical breeding applications, continuous innovation in high-throughput precision phenotyping holds promise for revolutionizing crop resilience and ensuring sustainable agricultural production in a changing climate.

Highlights Measurement of canopy light interception data using a ground-based mobile system. Using UAV-captured aerial images and zenith angle to estimate canopy light interception at different times of the day. Identifying boundaries of individual trees using the maximum likelihood estimator and the watershed algorithm. Abstract. Photosynthetically Active Radiation (PAR) absorbed by the leaves is a key piece of information to study the crop response to environmental conditions that could be used to estimate crop production potential. Canopies in a commercial orchard present differences in their capability to intercept light mainly due to the spatial variability in canopy development. There is a need for developing tools that could capture spatial variability in PAR interception to predict potential yield. Unmanned Aerial Vehicles (UAV) present an interesting alternative to provide this information, as they cover a larger area than ground-based systems in a shorter period with high spatial resolution. The objective of this study was to determine the relationship between the shadow of a tree derived from a ground-based canopy light interception scan obtained using a lightbar mounted on a mobile platform and that acquired from UAV Red-Green-Blue (RGB) images. Information acquired by an UAV was classified to separate canopy from its shadow, grass and sunlit soil using maximum likelihood estimator. Boundaries of individual trees were identified based on their positions using watershed transform algorithm. The relationship between canopy PAR interception data, sun angle in the sky (zenith angle), and the information derived from aerial images was analyzed. Coefficient of determination (R2) values of 0.92 and 0.88 were found for the multiple linear regression between PAR, the shadow area and the cosine of zenith angle obtained for almond and walnut crops, respectively. Moreover, R2 values of 0.81 and 0.86 were found for the relationship between the shadow’s area obtained underneath the canopy and the shadow’s area obtained from the UAV images and the cosine of the zenith angle for almond and walnut crops, respectively. The results show that the PAR interception can be estimated using the zenith angle and the area of the shadow, which can be obtained from a RGB aerial image. Keywords: Almond, Canopy segmentation, Image classification, PAR interception, Shadow area, UAV, Walnut.

To develop an intelligent recognition model based on deep learning algorithms of unmanned aerial vehicle (UAV) images, and to preliminarily explore the value of this model for remote identification, monitoring and management of cattle, a source of Schistosoma japonicum infection. Oncomelania hupensis snail-infested marshlands around the Poyang Lake area were selected as the study area. Image datasets of the study area were captured by aerial photography with UAV and subjected to augmentation. Cattle in the sample database were annotated with the annotation software VGG Image Annotator to create the morphological recognition labels for cattle. A model was created for intelligent recognition of livestock based on deep learning-based Mask R-convolutional neural network (CNN) algorithms. The performance of the model for cattle recognition was evaluated with accuracy, precision, recall, F1 score and mean precision. A total of 200 original UAV images were obtained, and 410 images were yielded following data augmentation. A total of 2 860 training samples of cattle recognition were labeled. The created deep learning-based Mask R-CNN model converged following 200 iterations, with an accuracy of 88.01%, precision of 92.33%, recall of 94.06%, F1 score of 93.19%, and mean precision of 92.27%, and the model was effective to detect and segment the morphological features of cattle. The deep learning-based Mask R-CNN model is highly accurate for recognition of cattle based on UAV images, which is feasible for remote intelligent recognition, monitoring, and management of the source of S. japonicum infection.

Green infrastructure (GI), such as green roofs, is now widely used in sustainable urban development. An accurate mapping of GI is important to provide surface parameterization for model development. However, the accuracy and precision of mapping GI is still a challenge in identifying GI at the small catchment scale. We proposed a framework for blue-green-gray infrastructure classification using machine learning algorithms and unmanned aerial vehicle (UAV) images that contained digital surface model (DSM) information. We used the campus of the Southern University of Science and Technology in Shenzhen, China, as a study case for our classification method. The UAV was a DJI Phantom 4 Multispectral, which measures the blue, green, red, red-edge, and near-infrared bands and DSM information. Six machine learning algorithms, i.e., fuzzy classifier, k-nearest neighbor classifier, Bayes classifier, classification and regression tree, support vector machine (SVM), and random forest (RF), were used to classify blue (including water), green (including green roofs, grass, trees (shrubs), bare land), and gray (including buildings, roads) infrastructure. The highest kappa coefficient was observed for RF and the lowest was observed for SVM, with coefficients of 0.807 and 0.381, respectively. We optimized the sampling method based on a chessboard grid and got the optimal sampling interval of 11.6 m to increase the classification efficiency. We also analyzed the effects of weather conditions, seasons, and different image layers, and found that images in overcast days or winter days could improve the classification accuracy. In particular, the DSM layer was crucial for distinguishing green roofs and grass, and buildings and roads. Our study demonstrates the feasibility of using UAV images in urban blue-green-gray infrastructure classification, and our infrastructure classification framework based on machine learning algorithms is effective. Our results could provide the basis for the future urban stormwater management model development and aid sustainable urban planning.

Multi-view scene matching refers to the establishment of a mapping relationship between images captured from different perspectives, such as those taken by unmanned aerial vehicles (UAVs) and satellites. This technology is crucial for the geo-localization of UAV views. However, the geometric discrepancies between images from different perspectives, combined with the inherent computational constraints of UAVs, present significant challenges for matching UAV and satellite images. Additionally, the imbalance of positive and negative samples between drone and satellite images during model training can lead to instability. To address these challenges, this study proposes a novel and efficient cross-view geo-localization framework called MSM-Transformer. The framework employs the Dual Attention Vision Transformer (DaViT) as the core architecture for feature extraction, which significantly enhances the modeling capacity for global features and the contextual relevance of adjacent regions. The weight-sharing mechanism in MSM-Transformer effectively reduces model complexity, making it highly suitable for deployment on embedded devices such as UAVs and satellites. Furthermore, the framework introduces a contrastive learning-based Symmetric Decoupled Contrastive Learning (DCL) loss function, which effectively mitigates the issue of sample imbalance between satellite and UAV images. Experimental validation on the University-1652 dataset demonstrates that MSM-Transformer achieves outstanding performance, delivering optimal matching results with a minimal number of parameters.

Bacterial blight poses a threat to rice production and food security, which can be controlled through large-scale breeding efforts toward resistant cultivars. Unmanned aerial vehicle (UAV) remote sensing provides an alternative means for the infield phenotype evaluation of crop disease resistance to relatively time-consuming and laborious traditional methods. However, the quality of data acquired by UAV can be affected by several factors such as weather, crop growth period, and geographical location, which can limit their utility for the detection of crop disease and resistant phenotypes. Therefore, a more effective use of UAV data for crop disease phenotype analysis is required. In this paper, we used time series UAV remote sensing data together with accumulated temperature data to train the rice bacterial blight severity evaluation model. The best results obtained with the predictive model showed an R p 2 of 0.86 with an RMSEp of 0.65. Moreover, model updating strategy was used to explore the scalability of the established model in different geographical locations. Twenty percent of transferred data for model training was useful for the evaluation of disease severity over different sites. In addition, the method for phenotypic analysis of rice disease we built here was combined with quantitative trait loci (QTL) analysis to identify resistance QTL in genetic populations at different growth stages. Three new QTLs were identified, and QTLs identified at different growth stages were inconsistent. QTL analysis combined with UAV high-throughput phenotyping provides new ideas for accelerating disease resistance breeding.

Recent technological advance in UAV (Unmanned Aerial Vehicle) technology offers new opportunities for assessing crop situation using UAV imagery. The objective of this study was to assess if reflectance and NDVI derived from consumer-grade cameras mounted on UAVs are useful for crop condition monitoring. This study was conducted using a fixed-wing UAV(Ebee) with Cannon S110 camera from March 2015 to March 2016 in the experiment field of National Institute of Agricultural Sciences. Results were compared with ground-based recordings obtained from consumer-grade cameras and ground multi-spectral sensors. The relationship between raw digital numbers (DNs) of UAV images and measured calibration tarp reflectance was quadratic. Surface (lawn grass, stairs, and soybean cultivation area) reflectance obtained from UAV images was not similar to reflectance measured by ground-based sensors. But NDVI based on UAV imagery was similar to NDVI calculated by ground-based sensors. Change of NDVI in lawn grass and soybean cultivation area depending on method of reflectance calculation using UAV images.

Assessing the efficacy of machine learning techniques to characterize soybean defoliation from unmanned aerial vehicles

In 2019, Depok Municipality in Indonesia has participated for the second times in PROKLIM (Program Kampung Iklim-bahasa) which is focussed in RW 10 Baktijaya Village. In previous year, community activities in biopori holes, family herbal planting (TOGA) and rainwater infiltration well (RIW) was difficult to be updated. Therefore, in this study unmanned aerial vehicle (UAV) is applied to acquire location of PROKLIM in Baktijaya village with very high resolution image. Herein, DJI Phantom 4 Pro was flown at 100 m height to capture images of participated houses. The image from UAV then processed with Agisoft Photoscan to generate orthophoto image. With the guiandce from local community leader, biopori hole, TOGA and RIW locations were tagged using Avenza Maps and input into ArcGIS. Layers from UAV images and field survey then integrated to produce maps of biopori hole, TOGA and RIW activities. The map of PROKLIM activities was successfully generated from UAV image and Avenza Maps application. Resulted maps are very useful for updating submission to PROKLIM registration website in Indonesia’s Ministry of Environment and Forestry.