Accurate Building Extraction from Fused DSM and UAV Images Using a Chain Fully Convolutional Neural Network

In recent years, research on Unmanned Aerial Vehicles (UAVs) has developed rapidly. Compared to traditional remote-sensing images, UAV images exhibit complex backgrounds, high resolution, and large differences in object scales. Therefore, UAV object detection is an essential yet challenging task. This paper proposes a multi-scale object detection network, namely YOLO-DroneMS (You Only Look Once for Drone Multi-Scale Object), for UAV images. Targeting the pivotal connection between the backbone and neck, the Large Separable Kernel Attention (LSKA) mechanism is adopted with the Spatial Pyramid Pooling Factor (SPPF), where weighted processing of multi-scale feature maps is performed to focus more on features. And Attentional Scale Sequence Fusion DySample (ASF-DySample) is introduced to perform attention scale sequence fusion and dynamic upsampling to conserve resources. Then, the faster cross-stage partial network bottleneck with two convolutions (named C2f) in the backbone is optimized using the Inverted Residual Mobile Block and Dilated Reparam Block (iRMB-DRB), which balances the advantages of dynamic global modeling and static local information fusion. This optimization effectively increases the model’s receptive field, enhancing its capability for downstream tasks. By replacing the original CIoU with WIoUv3, the model prioritizes anchoring boxes of superior quality, dynamically adjusting weights to enhance detection performance for small objects. Experimental findings on the VisDrone2019 dataset demonstrate that at an Intersection over Union (IoU) of 0.5, YOLO-DroneMS achieves a 3.6% increase in mAP@50 compared to the YOLOv8n model. Moreover, YOLO-DroneMS exhibits improved detection speed, increasing the number of frames per second (FPS) from 78.7 to 83.3. The enhanced model supports diverse target scales and achieves high recognition rates, making it well-suited for drone-based object detection tasks, particularly in scenarios involving multiple object clusters.

Accurate object detection in unmanned aerial vehicle (UAV) images plays a crucial role in fields such as aviation, transportation, and agriculture. However, UAV images often contain a high proportion of small objects, and the limited resources of UAV platforms make it challenging for existing small object detection algorithms to balance detection performance and resource consumption. To address these issues, a lightweight small object detection algorithm called SRP-YOLO is proposed. Compared to the YOLOv8 network, the following improvements are made: first, a high-resolution detection head is designed; second, a receptive field attention convolution module tailored for small UAV objects is introduced to replace the standard convolution module, enhancing the detection capability for small objects and significantly improving their feature representation; Third, partial convolution is combined with C2f to replace some of the standard convolutions, fully leveraging the high-resolution information from shallow features. Finally, A normalized Gaussian Wasserstein Distance (NWD) metric is introduced to reduce the sensitivity of Intersection over Union (IoU) to minor positional deviations of small objects. On VisDrone-DET2019, SRP-YOLO reduces parameters while improving mAP 0.5 by 7.9%. Generalization tests on TinypersonV2 and RAIVD confirm significant gains in small object detection.

Recent advances in structure from motion (SfM) and dense matching algorithms enable surface reconstruction from unmanned aerial vehicle (UAV) images with high spatial resolution, allowing for new insights into earth surface processes. However, accuracy issues are inherent in parallel‐axes UAV image configurations. In this study, the quality of digital elevation models (DEMs) is assessed using images from a simulated UAV flight. Five different SfM tools and three different cameras are compared. If ground control points (GCPs) are not integrated into the adjustment process with parallel‐axes image configurations, significant dome‐effect systematic errors are observed, which can be reduced based on calibration parameters retrieved from a testfield captured with convergent images immediately before or after the UAV flight. A comparison between DEMs of a soil surface generated from UAV images and terrestrial laser‐scanning data show that natural surfaces can be very accurately reconstructed from UAV images, even when GCPs are missing and simple geometric camera models are considered.

With the development of deep learning technology, an enormous number of convolutional neural network (CNN) models have been proposed to address the challenging building extraction task from very high-resolution (VHR) remote sensing images. However, searching for better CNN architectures is time-consuming, and the robustness of a new CNN model cannot be guaranteed. In this paper, an improved boundary-aware perceptual (BP) loss is proposed to enhance the building extraction ability of CNN models. The proposed BP loss consists of a loss network and transfer loss functions. The usage of the boundary-aware perceptual loss has two stages. In the training stage, the loss network learns the structural information from circularly transferring between the building mask and the corresponding building boundary. In the refining stage, the learned structural information is embedded into the building extraction models via the transfer loss functions without additional parameters or postprocessing. We verify the effectiveness and efficiency of the proposed BP loss both on the challenging WHU aerial dataset and the INRIA dataset. Substantial performance improvements are observed within two representative CNN architectures: PSPNet and UNet, which are widely used on pixel-wise labelling tasks. With BP loss, UNet with ResNet101 achieves 90.78% and 76.62% on IoU (intersection over union) scores on the WHU aerial dataset and the INRIA dataset, respectively, which are 1.47% and 1.04% higher than those simply trained with the cross-entropy loss function. Additionally, similar improvements (0.64% on the WHU aerial dataset and 1.69% on the INRIA dataset) are also observed on PSPNet, which strongly supports the robustness of the proposed BP loss.

Green infrastructure (GI), such as green roofs, is now widely used in sustainable urban development. An accurate mapping of GI is important to provide surface parameterization for model development. However, the accuracy and precision of mapping GI is still a challenge in identifying GI at the small catchment scale. We proposed a framework for blue-green-gray infrastructure classification using machine learning algorithms and unmanned aerial vehicle (UAV) images that contained digital surface model (DSM) information. We used the campus of the Southern University of Science and Technology in Shenzhen, China, as a study case for our classification method. The UAV was a DJI Phantom 4 Multispectral, which measures the blue, green, red, red-edge, and near-infrared bands and DSM information. Six machine learning algorithms, i.e., fuzzy classifier, k-nearest neighbor classifier, Bayes classifier, classification and regression tree, support vector machine (SVM), and random forest (RF), were used to classify blue (including water), green (including green roofs, grass, trees (shrubs), bare land), and gray (including buildings, roads) infrastructure. The highest kappa coefficient was observed for RF and the lowest was observed for SVM, with coefficients of 0.807 and 0.381, respectively. We optimized the sampling method based on a chessboard grid and got the optimal sampling interval of 11.6 m to increase the classification efficiency. We also analyzed the effects of weather conditions, seasons, and different image layers, and found that images in overcast days or winter days could improve the classification accuracy. In particular, the DSM layer was crucial for distinguishing green roofs and grass, and buildings and roads. Our study demonstrates the feasibility of using UAV images in urban blue-green-gray infrastructure classification, and our infrastructure classification framework based on machine learning algorithms is effective. Our results could provide the basis for the future urban stormwater management model development and aid sustainable urban planning.

Mining developments in alpine coal mining areas result in slow or rapid ground subsidence, which can lead to melting and collapse of permafrost. This paper integrated unmanned aerial vehicle (UAV) images and satellite-based SAR interferometry images to monitor intensive surface mining subsidence during reclamation. Digital Surface Model (DSM) acquired from UAV images was first used to evaluate the changes of the reclamation scheme on the microtopography carried out by slope and the Digital Elevation Model (DEM) of difference (DoD). The monitoring results showed that the slope had been reduced from over 30 degrees to under 15 degrees after the terrain had been reshaped. The DoD map revealed the distribution of main extraction areas and landfill areas. To further monitor the surface subsidence after local terrain adjustment, the Permanent Scatterer Interferometry (PS-InSAR) method was used to reveal the surface subsidence characteristics of the mine site before and after reclamation. The maximum cumulative subsidence ranged from −772.3 to 1183 mm based on 21 Sentinel-1A images in three years. Within a year of terrain reshaping, uplift and subsidence still occurred at hills and pit side slopes, following the nearly equal subsidence rate. The experimental results showed that the slope reshaping and vegetation recovery had a limited impact on the reduction of the ground subsidence in a short period. Therefore, on this basis, a combination of UAV and PS-InSAR methods can be used to continue monitoring time series subsidence in alpine mines.

Automatic extraction of yardangs using Landsat 8 and UAV images: A case study in the Qaidam Basin, China

Accurate building extraction from very high-resolution (VHR) remote sensing images plays an important role in urban dynamic monitoring, planning, and management. However, it is still a challenging task to achieve building extraction with high accuracy and integrity due to diverse building appearances and more complex ground background in VHR remote sensing images. Recently, Unity Networking (UNet) has been proven to be capable of feature extraction and semantic segmentation of remote sensing images. However, UNet cannot achieve sufficient multi-scale and multi-level features with larger receptive fields. To address these problems, an improved network based on UNet structure (Refine-UNet) is proposed for extracting buildings from the VHR images. The proposed Refine-UNet mainly consists of encoder module, decoder module, and refine skip connection scheme. The refine skip connection scheme is composed of an atrous spatial convolutional pyramid pooling (ASPP) module and several improved depthwise separable convolution (IDSC) modules. Experimental results on the Jilin-1 VHR datasets with a spatial resolution of 0.75 m demonstrate that compared with UNet, PSPNet, DeepLabV3+, and SegNet, the proposed Refine-UNet can obtain more accurate building extraction results and achieve the best Precision of 95.1% and intersection over union (IoU) of 87.0%, respectively, indicating the great practical potential.

Investigating post-earthquake surface ruptures is important for understanding the tectonics of seismogenic faults. The use of unmanned aerial vehicle (UAV) images to identify post-earthquake surface ruptures has the advantages of low cost, fast data acquisition, and high data processing efficiency. With the rapid development of deep learning in recent years, researchers have begun using it for image crack detection. However, due to the complex background and diverse characteristics of the surface ruptures, it remains challenging to quickly train an effective automatic earthquake surface rupture recognition model on a limited number of samples. This study proposes a workflow that applies an image segmentation algorithm based on convolutional neural networks (CNNs) to extract cracks from post-earthquake UAV images. We selected the 16-layer visual geometry group (VGG16) network as the primary network architecture. Then, we improved the VGG16 network and deleted several convolutional layers to reduce computation and memory consumption. Moreover, we added dilated convolution and atrous spatial pyramid pooling (ASPP) to make the network perform well in the surface crack identification of post-earthquake UAV images. We trained the proposed method using the data of the MS 7.4 Maduo earthquake and obtained a model that could automatically identify and draw small and irregular surface ruptures from high-resolution UAV images.

Building extraction from very high-resolution remote sensing images is a fundamental task and is widely used in applications such as change detection, disaster assessment, and real-time update of geographic information databases. However, due to the complexity of the geographical environment and the diversity of target features, accurate automatic building extraction remains very challenging. With the fast development of deep learning techniques, convolutional neural networks(CNN) have been widely used in remote sensing research and have achieved considerable results. But for large urban area-based building detection tasks, the CNN-based method usually gets into local optima and generates many false positive detections around building boundaries. To avoid the local optima and be aware of non-local information, this paper proposes a hybrid feature extraction model based on the combination of the CNN and Transformer to realize the automatic building detection from very high-resolution remote sensing images. Meanwhile, a multi-constraint weighting mechanism is proposed to enhance the ability of the model to recognize the regular geometric boundaries of buildings. Comprehensive experiments are conducted on the three different datasets. The proposed MC-TRANSU achieves the best F1-score, IoU, compared to the state-of-the-art methods such as SegNet, TransUnet, Swin-Unet, the detection accuracy improved around 5%. Quantitative and qualitative results verify the superiority and effectiveness of our model.

Quantitative structural analysis is a useful approach for studying geologic structures. It is particularly important in remote and complex fold-thrust belts where outcrop data and high-quality seismic reflection images are challenging to obtain. In this study, we integrated terrestrial light detection and ranging (LiDAR), unmanned aerial vehicle (UAV), and Landsat 8 (L8) data to extract high-resolution topographic and surface geologic information and constrain interpretations of three-dimensional (3-D) seismic reflection data in the Hero Range of the Qaidam Basin (QB) in northwestern China. UAV images were used to obtain a digital elevation model (DEM) and to measure the orientation of sedimentary bedding. Terrestrial LiDAR data were used to generate high-resolution digital outcrops and to evaluate the accuracy of the UAV-based DEM. L8 images were used to distinguish different stratigraphic units. The random sample consensus (RANSAC) algorithm was adopted to ascertain the best-fit plane of bedding. The results show that UAV images can be used to construct a DEM with <;1 m resolution and orthophotos with 0.15-m resolution. Collectively, these data improve the ability to identify and measure small exposures of bedding surfaces. The RANSAC algorithm improves the accuracy of measuring bedding orientations by removing erroneous selection points and facilitating the recognition of second-order variations in bedding orientation. The integrated analysis of remotely sensed and 3-D seismic data indicates that, of the three anticlines within the Hero Range, two are fault-propagation folds (the Shizigou and Youshashan anticlines) and one is associated with a pop-up structure (Ganchaigou anticline).

Automatic building extraction is an important topic for many applications such as urban planning, disaster management, 3D building modeling and updating GIS databases. Its approaches mainly depend on two data sources: light detection and ranging (LiDAR) point cloud and aerial imagery both of which have advantages and disadvantages of their own. In this study, in order to benefit from the advantages of each data sources, LiDAR and image data combined together. And then, the building boundaries were extracted with the automated active contour algorithm implemented in MATLAB. Active contour algorithm uses initial contour positions to segment an object in the image. Initial contour positions were detected without user interaction by a series of image enhancements, band ratio and morphological operations. Four test areas with varying building and background levels of detail were selected from ISPRS’s benchmark Vaihingen and Istanbul datasets. Vegetation and shadows were removed from all the datasets by band ratio to improve segmentation quality. Subsequently, LiDAR point cloud data was converted to raster format and added to the aerial imagery as an extra band. Resulting merged image and initial contour positions were given to the active contour algorithm to extract building boundaries. In order to compare the contribution of LiDAR to the proposed method, the boundaries of the buildings were extracted from the input image before and after adding LiDAR data to the image as a layer. Finally extracted building boundaries were smoothed by the Awrangjeb (Int J Remote Sen 37(3): 551–579. https://doi.org/10.1080/01431161.2015.1131868 , 2016) boundary regularization algorithm. Correctness (Corr), completeness (Comp) and accuracy (Q) metrics were used to assess accuracy of segmented building boundaries by comparing extracted building boundaries with manually digitized building boundaries. Proposed approach shows the promising results with over 93% correctness, 92% completeness and 89% quality.

Building extraction has attracted much attentions for decades as a prerequisite for many applications and is still a challenging topic in the field of photogrammetry and remote sensing. Due to the lack of spectral information, massive data processing, and approach universality, building extraction from point clouds is still a thorny and challenging problem. In this paper, a novel deep-learning-based framework is proposed for building extraction from point cloud data. Specifically, first, a sample generation method is proposed to split the raw preprocessed multi-spectral light detection and ranging (LiDAR) data into numerous samples, which are directly fed into convolutional neural networks and completely cover the original inputs. Then, a graph geometric moments (GGM) convolution is proposed to encode the local geometric structure of point sets. In addition, a hierarchical architecture equipped with GGM convolution, called GGM convolutional neural networks, is proposed to train and recognize building points. Finally, the test scenes with varying sizes can be fed into the framework and obtain a point-wise extraction result. We evaluate the proposed framework and methods on the airborne multi-spectral LiDAR point clouds collected by an Optech Titan system. Compared with previous state-of-the-art networks, which are designed for point cloud segmentation, our method achieves the best performance with a correctness of 95.1%, a completeness of 93.7%, an F-measure of 94.4%, and an intersection over union (IoU) of 89.5% on two test areas. The experimental results confirm the effectiveness and efficiency of the proposed framework and methods.

Traditional methods for building extraction from remote sensing images rely on feature classification techniques, which often suffer from high usage thresholds, cumbersome data processing, slow recognition speeds, and poor adaptability. With the rapid advancement of artificial intelligence, particularly machine learning and deep learning, significant progress has been achieved in the intelligent extraction of remote sensing images. Building extraction plays a crucial role in geographic information applications, such as urban planning, resource management, and ecological protection. This study proposes an efficient and accurate building extraction method based on the SegFormer model, a state-of-the-art Transformer-based architecture for semantic segmentation. The workflow includes data preparation, model construction, model deployment, and application. The SegFormer model is selected for its hierarchical Transformer encoder and lightweight MLP decoder, which enable high-precision binary classification of buildings in remote sensing images. Additionally, post-processing techniques, such as noise filtering, boundary cleanup, and building regularization, are applied to refine the inference results, significantly improving both the visual presentation and accuracy of the extracted buildings. Experimental validation is conducted using the publicly available WHU building dataset, demonstrating the effectiveness of the proposed method in urban, rural, and mountainous areas. The results show that the SegFormer model achieves high accuracy, with the MiT-B5 backbone network reaching 94.13% Intersection over Union (IoU) after 100 training epochs. The study highlights the robustness and scalability of the method, providing a solid technical foundation for remote sensing image analysis and practical applications in geographic information systems.

Deep convolutional neural networks have promoted significant progress in building extraction from high-resolution remote sensing imagery. Although most of such work focuses on modifying existing image segmentation networks in computer vision, we propose a new network in this paper, Deep Encoding Network (DE-Net), that is designed for the very problem based on many lately introduced techniques in image segmentation. Four modules are used to construct DE-Net: the inception-style downsampling modules combining a striding convolution layer and a max-pooling layer, the encoding modules comprising six linear residual blocks with a scaled exponential linear unit (SELU) activation function, the compressing modules reducing the feature channels, and a densely upsampling module that enables the network to encode spatial information inside feature maps. Thus, DE-Net achieves state-of-the-art performance on the WHU Building Dataset in recall, F1-Score, and intersection over union (IoU) metrics without pre-training. It also outperformed several segmentation networks in our self-built Suzhou Satellite Building Dataset. The experimental results validate the effectiveness of DE-Net on building extraction from aerial imagery and satellite imagery. It also suggests that given enough training data, designing and training a network from scratch may excel fine-tuning models pre-trained on datasets unrelated to building extraction.