CSVM Architectures for Pixel-Wise Object Detection in High-Resolution Remote Sensing Images

Oil palm trees detection and counting on Microsoft Bing Maps Very High Resolution (VHR) satellite imagery and Unmanned Aerial Vehicles (UAV) data using image processing thresholding approach

Convolutional Neural Network (CNN) is widely used for semantic segmentation and land-use and land-cover (LULC) mapping of very high-resolution (VHR) remote sensing images. The convolution operation is a powerful method for VHR classification, but the loss of high-frequency detail information caused during its operation decreases the classification accuracy, particularly in the boundary. Thus, it is necessary to supply additional boundary information to the CNN for alleviating this situation. In the classification task (and in LULC mapping), providing more effective information generates a better classification result. Current methods regard the boundary of images as the same category object and process it uniformly, which loses a notable amount of useful information because of the different properties, such as ambiguity and transition, between remote sensing images and their boundaries. Thus, a semantic segmentation method with category boundary for LULC mapping is proposed in this paper. First, a multi-task CNN called the category boundary detection network (CBDN) is designed to extract the boundary information of different category objects. Second, this category boundary and VHR images are used for initial semantic segmentation. Finally, the category boundary and the initial semantic segmentation result (ISSR) are fused to obtain the final LULC map by a two-step strategy, including the explicit fusion and the boundary attention loss function. To verify whether category boundary improved the classification accuracy, a set of comparative experiments were conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) Vaihingen and Potsdam datasets. The method in this paper was compared with a semantic segmentation method with no boundary information and a semantic segmentation method with global boundary. The results showed that the proposed method in this paper achieved good performances in the Vaihingen (overall accuracy (OA) = 0.924, Kappa coefficient (K) = 0.898, mean F1 score (mF1) = 0.896 and mean Intersection over Union (mIoU) = 0.817) and Potsdam datasets (OA = 0.890, K = 0.857, mF1 = 0.923, and mIoU = 0.860) based on the eroded labels.

Accurate building extraction from very high-resolution (VHR) remote sensing images plays an important role in urban dynamic monitoring, planning, and management. However, it is still a challenging task to achieve building extraction with high accuracy and integrity due to diverse building appearances and more complex ground background in VHR remote sensing images. Recently, Unity Networking (UNet) has been proven to be capable of feature extraction and semantic segmentation of remote sensing images. However, UNet cannot achieve sufficient multi-scale and multi-level features with larger receptive fields. To address these problems, an improved network based on UNet structure (Refine-UNet) is proposed for extracting buildings from the VHR images. The proposed Refine-UNet mainly consists of encoder module, decoder module, and refine skip connection scheme. The refine skip connection scheme is composed of an atrous spatial convolutional pyramid pooling (ASPP) module and several improved depthwise separable convolution (IDSC) modules. Experimental results on the Jilin-1 VHR datasets with a spatial resolution of 0.75 m demonstrate that compared with UNet, PSPNet, DeepLabV3+, and SegNet, the proposed Refine-UNet can obtain more accurate building extraction results and achieve the best Precision of 95.1% and intersection over union (IoU) of 87.0%, respectively, indicating the great practical potential.

Timely and accurate extraction of crop planting units plays a critical role in crop yield estimation, soil management, food supplies, and disaster warnings. However, precisely mapping crop types is challenging in smallholder farming systems due to heterogeneous and mixed pixels, where field sizes are small, and crop types are very diverse. In this paper, the crop type system of Huocheng County in northwest China as an example area, crop classification feature variables are constructed using Sentinel-2 remote sensing images, and combining the ReliefF algorithm, three feature selection classification models are established by 3092 crop type sampling points data. The crop planting units are extracted using pixel-based and object-based classification methods, respectively, and unmanned aerial vehicle (UAV) data assess the accuracy as ground truth. The Sentinel-2 image based smallholder crops classification results indicate that: The effective combination of optimal input feature variables selection and classification models significantly improves crop classification accuracy. This is particularly evident when integrating this approach into pixel-based classification, addressing challenges such as low pixel classification accuracy in regions of this type. The validation results from UAV data also confirm these findings. At validation points 2, 3, and 4, pixel-based classification demonstrates significantly higher accuracy compared to object-based classification, with pixel-based classification accuracy improving by 9.16%, 7.83%, and 31.83%, respectively. Although the UAV validation accuracy does not reach a high level, pixel-based classification remains the optimal choice for smallholder crop classification. This method offers new insights and references for research related to the classification of smallholder crops with complex planting structures.

As the acquisition of very high resolution (VHR) images becomes easier, the complex characteristics of VHR images pose new challenges to traditional machine learning semantic segmentation methods. As an excellent convolutional neural network (CNN) structure, U-Net does not require manual intervention, and its high-precision features are widely used in image interpretation. However, as an end-to-end fully convolutional network, U-Net has not explored enough information from the full scale, and there is still room for improvement. In this study, we constructed an effective network module: residual module under a multisensory field (RMMF) to extract multiscale features of target and an attention mechanism to optimize feature information. RMMF uses parallel convolutional layers to learn features of different scales in the network and adds shortcut connections between stacked layers to construct residual blocks, combining low-level detailed information with high-level semantic information. RMMF is universal and extensible. The convolutional layer in the U-Net network is replaced with RMMF to improve the network structure. Additionally, the multiscale convolutional network was tested using RMMF on the Gaofen-2 data set and Potsdam data sets. Experiments show that compared to other technologies, this method has better performance in airborne and spaceborne images.

Accurate leaf area index estimation in sorghum using high-resolution UAV data and machine learning models

Open-source data-driven urban land-use mapping integrating point-line-polygon semantic objects: A case study of Chinese cities

The paper focuses on the application of the Cross-Correlation Analysis (CCA) technique for quantifying changes of semi-natural grasslands to artificial structures at different spatial resolutions (grain) on a Mediterranean Natura 2000 site as a further test case of the results reported in [1]. In that work the CCA was applied to detect specific changes associated with agricultural intensification and fires. A semi-natural grasslands layer extracted from an existing Land Cover/Land Use map (1:5000, time T1) was considered as input to the CCA jointly with a Very High Resolution (VHR) WorldView-2 satellite image (2 meters spatial resolution, time T2) and with a High Resolution Landsat 8 OLS satellite image (30 meters spatial resolution, time T2), with T2 > T1, respectively, for the fine and the coarse scale analysis. The results were compared to those obtained by applying a traditional Post Classification Comparison technique to the same reference at time T1 map and an updated at time T2 map obtained by a knowledge driven classification of four multi-seasonal Worldview-2 input images, according to [1]. Also in this case the CCA technique results encouraging offering the possibility to reduce the costs of change detection when the acquisition of multi-seasonal VHR images at time T2 is too expensive or when no archive VHR image is available in the past for comparison between at time T1 and T2 images. The areas of change detected at VHR and HR were quite similar for the specific transition analyzed with larger error values in HR change images.

In this article, we develop a novel method for the detection of vineyard parcels in agricultural landscapes based on very high resolution (VHR) optical remote sensing images. Our objective is to perform texture-based image retrieval and supervised classification algorithms. To do that, the local textural and structural features inside each image are taken into account to measure its similarity to other images. In fact, VHR images usually involve a variety of local textures and structures that may verify a weak stationarity hypothesis. Hence, an approach only based on characteristic points, not on all pixels of the image, is supposed to be relevant. This work proposes to construct the local extrema-based descriptor (LED) by using the local maximum and local minimum pixels extracted from the image. The LED descriptor is formed based on the radiometric, geometric and gradient features from these local extrema. We first exploit the proposed LED descriptor for the retrieval task to evaluate its performance on texture discrimination. Then, it is embedded into a supervised classification framework to detect vine parcels using VHR satellite images. Experiments performed on VHR panchromatic PLEIADES image data prove the effectiveness of the proposed strategy. Compared to state-of-the-art methods, an enhancement of about 7% in retrieval rate is achieved. For the detection task, about 90% of vineyards are correctly detected.

With the development of deep learning technology, an enormous number of convolutional neural network (CNN) models have been proposed to address the challenging building extraction task from very high-resolution (VHR) remote sensing images. However, searching for better CNN architectures is time-consuming, and the robustness of a new CNN model cannot be guaranteed. In this paper, an improved boundary-aware perceptual (BP) loss is proposed to enhance the building extraction ability of CNN models. The proposed BP loss consists of a loss network and transfer loss functions. The usage of the boundary-aware perceptual loss has two stages. In the training stage, the loss network learns the structural information from circularly transferring between the building mask and the corresponding building boundary. In the refining stage, the learned structural information is embedded into the building extraction models via the transfer loss functions without additional parameters or postprocessing. We verify the effectiveness and efficiency of the proposed BP loss both on the challenging WHU aerial dataset and the INRIA dataset. Substantial performance improvements are observed within two representative CNN architectures: PSPNet and UNet, which are widely used on pixel-wise labelling tasks. With BP loss, UNet with ResNet101 achieves 90.78% and 76.62% on IoU (intersection over union) scores on the WHU aerial dataset and the INRIA dataset, respectively, which are 1.47% and 1.04% higher than those simply trained with the cross-entropy loss function. Additionally, similar improvements (0.64% on the WHU aerial dataset and 1.69% on the INRIA dataset) are also observed on PSPNet, which strongly supports the robustness of the proposed BP loss.

This work explores the suitability of various deep convolutional neural network (CNN) architectures for semantic segmentation of agricultural crops such as cotton, maize etc. from multi-spectral UAV (unmanned aerial vehicle) data. Initially, the UAV data were preprocessed and training samples for each crop type were manually annotated from multiple UAV scenes. Different CNN architectures such as U-Net, SegNet and PSPNet (Pyramid Scene Parsing Network) were trained with various combinations of input spectral bands along with select band derived indices such as NDVI (Normalized Difference Vegetation Index) and EVI (Enhanced Vegetation Index) as additional features. The experimental results indicated that inclusion of NIR (near-infrared) band and NDVI in the input data yielded high segmentation accuracy of more than 90%. U-Net proved to be the best among the three architectures with 97% overall accuracy while dealing with three classes separation problem. This study demonstrated the scope of deep neural network based semantic segmentation techniques in crop classification from multi-spectral UAV data.KeywordsCrop classificationUAV (unmanned aerial vehicle)Multi-spectral dataDeep learningU-NetSegNetPSPNetImage analysis

Abstract. Very high resolution (VHR) remote sensing imagery can reveal the ground object in greater detail, depicting their color, shape, size and structure. However, VHR also leads much original noise in spectra, and this original noise may reduce the reliability of the classification’s result. This paper presents an Adaptive Morphological Mean Filter (AMMF) for smoothing the original noise of VHR imagery and improving the classification’s performance. AMMF is a shape-adaptive filter which is constructed by detecting gradually the spectral similarity between a kernel-anchored pixel and its contextual pixels through an extension-detector with 8-neighbouring pixels, and the spectral value of the kernel-anchored pixel is instead by the mean of group pixels within the adaptive region. The classification maps based on the AMMF are compared with the classification of VHR images based on the homologous filter processing, such as Mean Filter (MF) and Median Filter(MedF). The experimental results suggest the following: 1) VHR image processed using AMMF can not only preserve the detail information among inter-classes but also smooth the noise within intra-class; 2) The proposed AMMF processing can improve the classification’s performance of VHR image, and it obtains a better visual performance and accuracy while comparing with MF and MedF.

This paper focuses on the application of a semi-automatic unsupervised change detection algorithm called Cross Correlation Analysis (CCA) to the detection of (semi-) natural grasslands changes at Very High Resolution (VHR). A reference validated Land Cover/Land Use map at time T1 and only one satellite image at time T2, with T2>T1, are required to detect changes occurred at T2 in the selected target class. This approach offers the possibility to reduce the costs of change detection when the acquisition of multi-seasonal VHR images at time T2 for supervised change detection is too expensive or when no archive VHR image is available in the past for unsupervised comparison between T1 and T2 images. A summer Worldview-2 image for a Natura 2000 test site was considered and the results appear encouraging.

Building detection in very high resolution (VHR) remote sensing images is crucial for many urban planning and management applications. Since buildings are elevated objects, the incorporation of elevation data provides a mean to reliable detection. However, almost all existing methods of elevation-based building detection must first generate a normalized Digital Surface Model (nDSM). This model is generated by processes of extracting and subtracting terrain elevations from the DSM data. The generation of accurate nDSM is still a challenging task to some extent. This paper introduces a segment-based terrain filtering (SegTF) technique to filter out the terrain elevations directly using DSM elevations. This technique has four steps: elevation co-registration, image segmentation, slope calculation, and building detection. These steps of the developed technique were applied to a dataset that consisted of a VHR image and a corresponding DSM for detecting buildings. The result of the building detection was evaluated and found to be 100% correct with an overall detection quality of 93%. These values indicate a highly reliable and promising technique for mapping buildings in VHR images.

Precise and timely leaf area index (LAI) estimation for winter wheat is crucial for precision agriculture. The emergence of high-resolution unmanned aerial vehicle (UAV) data and machine learning techniques offers a revolutionary approach for fine-scale estimation of wheat LAI at the low cost. While machine learning has proven valuable for LAI estimation, there are still model limitations and variations that impede accurate and efficient LAI inversion. This study explores the potential of classical machine learning models and deep learning model for estimating winter wheat LAI using multispectral images acquired by drones. Initially, the texture features and vegetation indices served as inputs for the partial least squares regression (PLSR) model and random forest (RF) model. Then, the ground-measured LAI data were combined to invert winter wheat LAI. In contrast, this study also employed a convolutional neural network (CNN) model that solely utilizes the cropped original image for LAI estimation. The results show that vegetation indices outperform the texture features in terms of correlation analysis with LAI and estimation accuracy. However, the highest accuracy is achieved by combining both vegetation indices and texture features to invert LAI in both conventional machine learning methods. Among the three models, the CNN approach yielded the highest LAI estimation accuracy (R 2 = 0.83), followed by the RF model (R 2 = 0.82), with the PLSR model exhibited the lowest accuracy (R 2 = 0.78). The spatial distribution and values of the estimated results for the RF and CNN models are similar, whereas the PLSR model differs significantly from the first two models. This study achieves rapid and accurate winter wheat LAI estimation using classical machine learning and deep learning methods. The findings can serve as a reference for real-time wheat growth monitoring and field management practices.