Fluvial land cover classification by using CSC deep learning method with UAV airborne images

AN ASSESSMENT OF SPATIAL VARIATION OF LAND SURFACE CHARACTERISTICS OF MINNA, NIGER STATE NIGERIA FOR SUSTAINABLE URBANIZATION USING GEOSPATIAL TECHNIQUES

In this study we evaluate the extent to which accurate topographic data can be obtained by applying Structure from Motion (SfM) photogrammetric methods to archival imagery. While SfM has proven valuable in photogrammetric applications using specially acquired imagery (e.g. from unmanned aerial vehicles), it also has the potential to improve the precision of topographic data and the ease with which can be produced from historical imagery. We evaluate the application of SfM to a relatively extreme case, one of low relative relief: a braided river–floodplain system. We compared the bundle adjustments of SfM and classical photogrammetric methods, applied to eight dates. The SfM approach resulted in data quality similar to the classical approach, although the lens parameter values (e.g. focal length) recovered in the SfM process were not necessarily the same as their calibrated equivalents. Analysis showed that image texture and image overlap/configuration were critical drivers in the tie‐point generation which impacted bundle adjustment quality. Working with archival imagery also illustrated the general need for the thorough understanding and careful application of (commercial) SfM software packages. As with classical methods, the propagation of (random) error in the estimation of lens and exterior orientation parameters using SfM methods may lead to inherent systematic error in the derived point clouds. We have shown that linear errors may be accounted for by point cloud registration based on a reference dataset, which is vital for the further application in quantitative morphological analyses when using archival imagery. Copyright © 2016 John Wiley & Sons, Ltd.

This paper proposes a new methodology to address the image quality problem encountered as the use of an unmanned aerial vehicle (UAV) in the field of bridge inspection increased. When inspecting a bridge, the image obtained from the UAV was degraded by various interference factors such as vibration, wind, and motion of UAV. Image quality degradation such as blur, noise, and low-resolution is a major obstacle in utilizing bridge inspection technology based on UAV. In particular, in the field of bridge inspection where damages must be accurately and quickly detected based on data obtained from UAV, these quality issues weaken the advantage of using UAVs by requiring re-take of images through re-flighting. Therefore, in this study, image quality assessment (IQA) based on local blur map (LBM) and image quality enhancement (IQE) using the variational Dirichlet (VD) kernel estimation were proposed as a solution to address the quality issues. First, image data was collected by setting different camera parameters for each bridge member. Second, a blur map was generated through discrete wavelet transform (DWT) and a new quality metric to measure the degree of blurriness was proposed. Third, for low-quality images with a large degree of blurriness, the blind kernel estimation and blind image deconvolution were performed to enhance the quality of images. In the validation tests, the proposed quality metric was applied to material image sets of bridge pier and deck taken from UAV, and its results were compared with those of other quality metrics based on singular value decomposition (SVD), sum of gray-intensity variance (SGV) and high-frequency multiscale fusion and sort transform (HiFST) methods. It was validated that the proposed IQA metric showed better classification performance on UAV images for bridge inspection through comparison with the classification results by human perception. In addition, by performing IQE, on average, 26% of blur was reduced, and the images with enhanced quality showed better damage detection performance through the deep learning model (i.e., mask and region-based convolutional neural network).

Ground control points (GCPs) are used in forest surveys employing unmanned aerial vehicle (UAV)-based structure from motion (SfM). In that context, the influence of the surrounding environment on GCP placement requires further analysis. This study investigated the effects of GCP placement and the surrounding environment on the estimation of forest information by UAV-SfM. Forest resource estimation was performed using UAV (Inspire2) aerial images and SfM analysis (via Pix4Dmapper) under varying environmental conditions around GCPs within the same forest stand. The results indicated that GCP placement had no significant effect on SfM processing, tree top extraction (the number of extracted target trees was 151 or 150), or tree crown area estimation (RMSEs ranged from approximately 5 to 6.5 m2). However, when GCPs were placed in open areas, the tree height estimation accuracy improved, without significant differences between estimated and measured values (patterns A, B, D and E, had RMSEs of 1.60 to 3.09 m; patterns C and D had RMSEs of 5.69 to 7.92 m). These findings suggest that in UAV-SfM-based forest resource surveys, particularly for tree height estimation, both the number and placement of GCPs, as well as the surrounding environment, are crucial in enhancing estimation accuracy.

In this paper, we develop an alternative navigation system for Unmanned Aerial Vehicle (UAV) in Global Positioning Systems (GPS)-denied environment. We use two image inputs, one is acquired with an on-board camera placed on the UAV (which is the large-area image) and the other is from satellite images (which is small known image) with GPS information. We use a convolutional neural network (CNN) architecture based on Oxford’s Visual Geometry Group network (VGG-16) and utilize normalized variant mutual information between these two images to obtain position of the UAV. Satellite images are labelled and given to the UAV. When GPS information is lost, our algorithm starts to function and images from UAV camera are searched whether satellite image is seen by cameras on UAV image or not. If the UAV is in that area, our algorithm finds the GPS information from satellite image data.

The study aimed to determine the efficacy and capabilities of using high-resolution aerial imagery and a convolutional neural network (CNN) to identify plant species and monitor land cover and land change in the context of remote sensing. The full capabilities of a CNN were examined, including testing whether the platform could be used for land cover and the evaluation of land change over time. An unmanned aerial vehicle (UAV) was used to collect the aerial data of the study area. The CNN was encoded and operated in RStudio, while digitised data from the input imagery were used by the programme as training and validation data. The object in this respect was to learn about the relevant features of the landscape, and thereafter to classify the Opuntia invasive plant species. Accuracy assessments were carried out on the results to test the efficacy of the aerial imagery in terms of its accuracy and reliability. The classification achieved an overall accuracy of 93%, while the kappa coefficient score was 0.86. CNN was also able to predict the land coverage area of Opuntia to be within four percent (4%) of the ground truthing data. A change in land cover over time was detected by the programme after the manual clearing of the plant had been undertaken. This research has determined that the use of a CNN in remote sensing is a very powerful tool for supervised image classifications. It can be used for monitoring land cover in that it is able to accurately estimate the spatial distribution of plant species and to monitor the growth or decline in the species over time. As such, it is an efficient methodology and its use in remote sensing could be extended.

ABSTRACTIn recent years, object-based segmentation methods and shallow-model classification algorithms have been widely integrated for remote sensing image supervised classification. However, as the image resolution increases, remote sensing images contain increasingly complex characteristics, leading to higher intraclass heterogeneity and interclass homogeneity and thus posing substantial challenges for the application of segmentation methods and shallow-model classification algorithms. As important methods of deep learning technology, convolutional neural networks (CNNs) can hierarchically extract higher-level spatial features from images, providing CNNs with a more powerful recognition ability for target detection and scene classification in high-resolution remote sensing images. However, the input of the traditional CNN is an image patch, the shape of which is scarcely consistent with a given segment. This inconsistency may lead to errors when directly using CNNs in object-based remote sensing classification: jagged errors may appear along the land cover boundaries, and some land cover areas may overexpand or shrink, leading to many obvious classification errors in the resulting image. To address the above problem, this paper proposes an object-based and heterogeneous segment filter convolutional neural network (OHSF-CNN) for high-resolution remote sensing image classification. Before the CNN processes an image patch, the OHSF-CNN includes a heterogeneous segment filter (HSF) to process the input image. For the segments in the image patch that are obviously different from the segment to be classified, the HSF can differentiate them and reduce their negative influence on the CNN training and decision-making processes. Experimental results show that the OHSF-CNN not only can take full advantage of the recognition capabilities of deep learning methods but also can effectively avoid the jagged errors along land cover boundaries and the expansion/shrinkage of land cover areas originating from traditional CNN structures. Moreover, compared with the traditional methods, the proposed OHSF-CNN can achieve higher classification accuracy. Furthermore, the OHSF-CNN algorithm can serve as a bridge between deep learning technology and object-based segmentation algorithms thereby enabling the application of object-based segmentation methods to more complex high-resolution remote sensing images.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The empowering unmanned aerial vehicles (UAVs) have been extensively used in providing intelligence such as target tracking. In our field experiments, a pretrained convolutional neural network (CNN) is deployed at UAV to identify a target (a vehicle) from the captured video frames and enable the UAV to keep tracking. However, this kind of visual target tracking demands a lot of computational resources due to the desired high inference accuracy and stringent delay requirement. This motivates us to consider offloading this type of deep learning (DL) tasks to a mobile-edge computing (MEC) server due to the limited computational resource and energy budget of the UAV and further improve the inference accuracy. Specifically, we propose a novel hierarchical DL tasks distribution framework, where the UAV is embedded with lower layers of the pretrained CNN model while the MEC server (MES) with rich computing resources will handle the higher layers of the CNN model. An optimization problem is formulated to minimize the weighted-sum cost, including the tracking delay and energy consumption introduced by communication and computing of UAVs while taking into account the quality of data (e.g., video frames) input to the DL model and the inference errors. Analytical results are obtained and insights are provided to understand the tradeoff between the weighted-sum cost and inference error rate in the proposed framework. Numerical results demonstrate the effectiveness of the proposed offloading framework.

Forest fires pose a serious threat to the environment with the potential of causing ecological harm, financial losses, and human casualties. While research suggests that climate change will increase the frequency and severity of these fires, recent developments in deep learning and convolutional neural networks (CNN) have greatly enhanced fire detection techniques and capability. These models can be leveraged by unmanned aerial vehicles (UAVs) to automatically monitor burning areas. However, drones can carry only limited computational and power resources; therefore, on-board computing capabilities are constrained by hardware limitations. This work focuses on the design of segmentation models to identify and localize active burning areas from aerial RGB images processed on limited computing resources. To achieve this goal, the research compares the performance of different variants of the DeepLabv3 neural network model for fire segmentation when trained and tested with the FLAME dataset using a k-fold cross validation approach. Experimental results are compared with U-Net, a benchmark model used with the FLAME dataset, by implementing this model in the same codebase as the DeepLabv3 model. This work demonstrates that a refined version of DeepLabv3, with a MobileNetv2 backbone using pretrained layers and a simplified atrous spatial pyramid pooling (ASPP) module, yields a similar performance to U-Net, with a precision of 87.8% and a recall of 83.2%, while only requiring 20% of the number of parameters involved with the U-Net topology. This significantly reduces memory and power consumption, enabling longer UAV flight duration and reducing the processing overhead associated with sensor input, making it more suitable for deployment on unmanned aerial vehicles. The model’s compact architecture, implemented using TensorFlow and Keras for model design and training, along with OpenCV for image preprocessing, makes it portable and easy to integrate with edge devices such as NVIDIA Jetson boards.

With the rapid proliferation of unmanned aerial vehicles (UAVs) in civilian and industrial applications, the risk of malicious or unauthorized UAV use has become a critical security concern. Existing machine learning (ML)-based UAV recognition methods offer a certain degree of interpretability, but their performance is often limited in complex environments and across diverse UAV types. In contrast, deep learning (DL)-based methods exhibit strong representation capability, yet they generally lack physical interpretability. To address this issue, we propose an interpretable UAV recognition framework, termed frequency-aware network for UAV recognition (FANet-UAV), which performs coarse-to-fine feature learning in the frequency domain. Specifically, a multiplication filter module (MFM) is first designed to capture coarse-grained spectral patterns by exploiting multi-mode and multi-scale frequency characteristics of UAV signals. Based on these coarse representations, a convolutional neural network (CNN) is further employed to extract fine-grained discriminative features for accurate classification. Experimental results on two public UAV datasets demonstrate the effectiveness of the proposed method. In particular, FANet-UAV improves the recognition accuracy from 90.45% to 96.82% on DroneRFa and from 94.15% to 98.83% on DroneRF. Moreover, visualization results and channel-wise SHAP analysis provide both pre-hoc and post-hoc interpretability, revealing that FANet-UAV mainly relies on flight control signal (FCS) features for decision-making, while video transmission signal (VTS) features contribute less to the final recognition results.

Farmland quality assessment using deep learning and UAVs

Conventional geographic object-based image analysis (GEOBIA) land cover classification methods by using very high resolution images are hardly applicable due to their complex ground truth and manually selected features, while convolutional neural networks (CNNs) with many hidden layers provide the possibility of extracting deep features from very high resolution images. Compared with pixel-based CNNs, superpixel-based CNN classification, carrying on the idea of GEOBIA, is more efficient. However, superpixel-based CNNs are still problematic in terms of their processing units and accuracies. Firstly, the limitations of salt and pepper errors and low boundary adherence caused by superpixel segmentation still exist; secondly, this method uses the central point of the superpixel as the classification benchmark in identifying the category of the superpixel, which does not allow classification accuracy to be ensured. To solve such problems, this paper proposes a region-based majority voting CNN which combines the idea of GEOBIA and the deep learning technique. Firstly, training data was manually labeled and trained; secondly, images were segmented under multiresolution and the segmented regions were taken as basic processing units; then, point voters were generated within each segmented region and the perceptive fields of points voters were put into the multi-scale CNN to determine their categories. Eventually, the final category of each region was determined in the region majority voting system. The experiments and analyses indicate the following: 1. region-based majority voting CNNs can fully utilize their exclusive nature to extract abstract deep features from images; 2. compared with the pixel-based CNN and superpixel-based CNN, the region-based majority voting CNN is not only efficient but also capable of keeping better segmentation accuracy and boundary fit; 3. to a certain extent, region-based majority voting CNNs reduce the impact of the scale effect upon large objects; and 4. multi-scales containing small scales are more applicable for very high resolution image classification than the single scale.

<p>With consumer-grade unmanned aerial vehicles (UAVs) on the rise, which enable easy, time-flexible, and cost-effective acquisition of very high-resolution RGB data, the mapping of forest tree species using solely RGB imagery is of high interest, as it does not rely on sophisticated sensors, does not require extensive calibration and preprocessing and, therefore, enables the application by a wide audience. In combination with convolutional neural networks (CNNs), which particularly exploit spatial patterns and, therefore, highly benfit from very high-resolution remote sensing, this offers great potential for accurately mapping forest tree species.</p><p>Here, we present the findings of our recent study, in which we used very high-resolution RGB imagery from UAVs in combination with CNNs for the mapping of forest tree species. In this study, we used multicopter UAVs to obtain very high-resolution (<2 cm) RGB imagery over 51 ha of temperate forests in the Southern Black Forest region, and the Hainich National Park in Germany. To fully harness the end-to-end learning capabilities of CNNs, we used a semantic segmentation approach (U-net) that concurrently segments and classifies tree species from imagery. With a diverse dataset in terms of study areas, site conditions, illumination properties, and phenology, we accurately mapped nine tree species, three genus-level classes, deadwood, and forest floor (mean F1-score 0.73). We found that a coarser spatial resolution substantially reduced the model accuracy (mean F1-score of 0.26 at 32 cm resolution) and that larger tile sizes during CNN training negatively affected the model accuracies for underrepresented classes. Additional height information from normalized digital surface models slightly increased the model accuracy but simultaneously increased computational complexity and data requirements. Our results highlight the key role that UAVs can play in the mapping of forest tree species, given that air- and spaceborne remote sensing currently does not provide comparable spatial resolutions. Given the end-to-end learning capabilites of CNNs extensive preprocessing becomes partly obsolete, whereas the use of a large and diverse dataset facilitates a high degree of generalization of the CNN, thus fostering transferability. The synergy of high-resolution UAV imagery and CNN provide a fast and flexible yet accurate means of mapping forest tree species.</p><p>In this contribution, we will give an outlook on how the combination of UAV imagery and CNNs can be integrated with multitemporal satellite imagery (Sentinel-1 and Sentinel-2) in order to extrapolate the UAV-based tree species maps to larger areas.</p>

Abstract. Mud volcanoes are geological formations resulting from the expulsion of mud, gases, and fluids from deep underground. Monitoring these formations provides critical insights into subsurface processes and geological hazards. This study focuses on detecting recent mud extrusions in mud volcano environments using high-resolution aerial imagery acquired by unmanned aerial vehicles (UAVs). Using UAV-based surveys instead of satellite imagery, we obtain finer spatial detail suitable for identifying subtle textural and chromatic variations in relatively small sites. A binary image classification pipeline was developed to distinguish recent mud from non-mud areas. Traditional machine learning algorithms, including Support Vector Machine (SVM), Random Forest, and Extreme Gradient Boosting (XGBoost), were compared with deep learning architectures such as Convolutional Neural Networks (CNNs), both leveraging transfer learning and custom models. Traditional algorithms rely on handcrafted features, while CNNs learn hierarchical representations directly from raw data. Feature extraction methods were selected based on their ability to distinguish between the two designated classes effectively. To enhance model robustness and generalization, a designed augmentation pipeline was applied before each training epoch or cross-validation fold. This strategy introduced controlled and random variations to simulate real-world imaging conditions, such as varying viewpoints and lighting, ensuring the models generalization, moreover it also minimized data leakage by presenting distinct image variations throughout training. CNNs achieved the highest accuracy, outperforming traditional algorithms and demonstrating the advantages of combining deep learning with effective data augmentation. These findings underscore the potential of CNNs for accurate and efficient monitoring of dynamic geological environments.