ME-Net: A Multi-Scale Erosion Network for Crisp Building Edge Detection from Very High Resolution Remote Sensing Imagery

Offshore and inland river ship detection has been studied on both synthetic aperture radar (SAR) and optical remote sensing imagery. However, the classic ship detection methods based on SAR images can cause a high false alarm ratio and be influenced by the sea surface model, especially on inland rivers and in offshore areas. The classic detection methods based on optical images do not perform well on small and gathering ships. This paper adopts the idea of deep networks and presents a fast regional-based convolutional neural network (R-CNN) method to detect ships from high-resolution remote sensing imagery. First, we choose GaoFen-2 optical remote sensing images with a resolution of 1 m and preprocess the images with a support vector machine (SVM) to divide the large detection area into small regions of interest (ROI) that may contain ships. Then, we apply ship detection algorithms based on a region-based convolutional neural network (R-CNN) on ROI images. To improve the detection result of small and gathering ships, we adopt an effective target detection framework, Faster-RCNN, and improve the structure of its original convolutional neural network (CNN), VGG16, by using multiresolution convolutional features and performing ROI pooling on a larger feature map in a region proposal network (RPN). Finally, we compare the most effective classic ship detection method, the deformable part model (DPM), another two widely used target detection frameworks, the single shot multibox detector (SSD) and YOLOv2, the original VGG16-based Faster-RCNN, and our improved Faster-RCNN. Experimental results show that our improved Faster-RCNN method achieves a higher recall and accuracy for small ships and gathering ships. Therefore, it provides a very effective method for offshore and inland river ship detection based on high-resolution remote sensing imagery.

Urban neighborhood classification using very high resolution (VHR) remote sensing imagery is a challenging and {\em emerging} application. A semi-supervised learning approach for identifying neighborhoods is presented which employs superpixel tessellation representations of VHR imagery. The image representation utilizes homogeneous and irregularly shaped regions termed superpixels and derives novel features based on intensity histograms, geometry, corner and superpixel density and scale of tessellation. The semi-supervised learning approach uses a support vector machine (SVM) to obtain a preliminary classification which is then subsequently refined using graph Laplacian propagation. Several intermediate stages in the pipeline are presented to showcase the important features of this approach. We evaluated this approach on four different geographic settings with varying neighborhood types and compared it with the recent Gaussian Multiple Learning algorithm. This evaluation shows several advantages, including model building, accuracy, and efficiency which makes it a great choice for deployment in large scale applications like global human settlement mapping and population distribution (e.g., LandScan), and change detection.

Building extraction based on high-resolution remote sensing imagery has been widely used in automatic surveying and mapping. Recently, the instance segmentation algorithm has been introduced to the building extraction, which can calculate the number and area of buildings simultaneously. However, there are some challenges: 1) multi-scale buildings; 2) occlusion by other adjacent buildings. In this paper, to solve these problems, we propose a multi-scale building instance extraction framework based on feature pyramid object-aware convolution neural network (CNN). In order to solve the multi-scale problem, a feature pyramid CNN is proposed, which combines features from both the bottom-up and top-down architectures. In order to solve the occlusion problem, a multi-scale object-aware instance proposal network is proposed, which introduces the multiscale attention mechanism to aware objects. The experiments conducted on two public datasets and a self-constructed dataset of Changzhou show that the proposed method can achieve an excellent performance.

Greenhouses are densely distributed across the cultivated land in high-resolution remote sensing imagery, resulting in the problem of dense object extraction. On the one hand, objects tend to be wrongly merged into one object since objects are connected; on the other hand, the existed random sampling scheme does not make good use of the object distribution density to improve the training effect. To meet the demand for dense greenhouse extraction, this paper proposes a novel deep learning-based greenhouse extraction algorithm. To solve the problem that dense objects tend to be wrongly merged, this paper proposes a dual-task learning module, which uses the discriminant object boundary of the pixel-based branch to distinguish the objects of the object-based branch; to take advantage of the object distribution density for effective training, a high-density biased sampler is proposed. Moreover, this paper provides a dataset of manually labeled imagery to train and develop the proposed method. Results on greenhouse extraction in six regions in China achieve peak mIoU and mAP value, surpassing the state-of-the-art methods. Finally, a product of a greenhouse map in China is provided for analysis.

Deep-learning-based methods for building-edge-detection have been widely researched and applied in the field of image processing. However, these methods often emphasis the analysis of deep features, which may result in neglecting crucial shallow information representation. Furthermore, abstract features in the deep layers can potentially interfere with the accuracy of edge extraction. To address these challenges, we propose a densely connected edge-detection enhancement network (DCEFE-Net) for building-edge-detection in high-resolution remote sensing images. Firstly, by introducing spatial land channel attention modules, we effectively captured low-level spatial information and high-level semantic information from the input image. Secondly, the proposed edge-aware feature enhancement (EAFE) module emphasis the representation of informative edge features. By alliteratively generating multiple layers of edge-detection maps, it addresses the issue of edge detail loss and enhances edge-detection accuracy. Finally, the dense connectivity blocks strengthen the connections between the convolutional layers, thereby preventing the loss of edge features. Experimental results on the WHU and the Inria Aerial Image Labeling datasets validate the effectiveness of DCEFE-Net, as it consistently produces clear and reliable building-edge results.

Semantic segmentation of remote sensing images enables in particular land-cover map generation for a given set of classes. Very recent literature has shown the superior performance of deep convolutional neural networks (DCNN) for many tasks, from object recognition to semantic labelling, including the classification of Very High Resolution (VHR) satellite images. However, while plethora of works aim at improving object delineation on geographically restricted areas, few tend to solve this classification task at very large scales. New issues occur such as intra-class class variability, diachrony between surveys, and the appearance of new classes in a specific area, that do not exist in the predefined set of labels. Therefore, this work intends to (i) perform large scale classification and to (ii) expand a set of land-cover classes, using the off-the-shelf model learnt in a specific area of interest and adapting it to unseen areas.

As the acquisition of very high resolution (VHR) images becomes easier, the complex characteristics of VHR images pose new challenges to traditional machine learning semantic segmentation methods. As an excellent convolutional neural network (CNN) structure, U-Net does not require manual intervention, and its high-precision features are widely used in image interpretation. However, as an end-to-end fully convolutional network, U-Net has not explored enough information from the full scale, and there is still room for improvement. In this study, we constructed an effective network module: residual module under a multisensory field (RMMF) to extract multiscale features of target and an attention mechanism to optimize feature information. RMMF uses parallel convolutional layers to learn features of different scales in the network and adds shortcut connections between stacked layers to construct residual blocks, combining low-level detailed information with high-level semantic information. RMMF is universal and extensible. The convolutional layer in the U-Net network is replaced with RMMF to improve the network structure. Additionally, the multiscale convolutional network was tested using RMMF on the Gaofen-2 data set and Potsdam data sets. Experiments show that compared to other technologies, this method has better performance in airborne and spaceborne images.

The goal of this research was to explore the utility of very high spatial resolution, digital remotely sensed imagery for monitoring land‐cover changes in habitat preserves within southern California coastal shrublands. Changes were assessed for Los Penasquitos Canyon Preserve, a large open space in San Diego County, over the 1996 to 1999 period for which imagery was available. Multispectral, digital camera imagery from two summer dates, three years apart, was acquired using the Airborne Data Acquisition and Registration (ADAR) digital‐camera system. These very high resolution (VHR) image data (1m), composed of three visible and one near‐infrared wavebands (V/NIR), were the primary image input for assessing land cover change. Image‐derived datasets generated from georeferenced and registered ADAR imagery included multitemporal overlays and multitemporal band differencing with threshold selection. Two different multitemporal image classifications were generated from these datasets and compared. Single‐date imagery was analyzed interactively with image‐derived datasets and with information from field observations in an effort to discern change types. A ground sampling survey conducted soon after the 1999 image acquisition provided concurrent ground reference data. Most changes occurring within the three‐year interval were associated with transitional phenological states and differential precipitation effects on herbaceous cover. Variations in air temperatures and timing of rainfall contributed to differences that the seven‐week image acquisition offset may have caused. Disturbance factors of mechanical clearing, erosion, potentially invasive plants, and fire were evident and their influence on the presence, absence, and type of vegetation cover were likely sources of change signals. The multitemporal VHR, V/NIR image data enabled relatively fine‐scale land cover changes to be detected and identified. Band differencing followed by multitemporal classification provided an effective means for detecting vegetation increase or decrease. Detailed information on short‐term disturbance effects and long‐term vegetation type conversions can be extracted if image acquisitions are carefully planned and geometric and radiometric processing steps are implemented.

With the emergence of large scale Very High Resolution (VHR) remotely sensed imageries, more accurate urban building segmentation now become possible. Meanwhile, various appearances along with complicated background of the urban VHR imageries make accurate segmentation of urban buildings yet a very challenging task. In this study, we proposed DeepAttentionUnet, which is an end-to-end attention-based model following the basic structure of U-Net and being integrated with attention mechanism so that the model can automatically learn to focus on target building structure of different sizes and shapes. This model adopts attention mechanism to implicitly highlight useful features of buildings while suppressing irrelevant areas of input images and combines with residual learning method to ease training process by alleviating the degradation problem in the training process that often occurs in such applications. Our proposed DeepAttentionUnet was tested with a set of 0.075m resolution aerial images and was compared for performance under the exact same conditions with other two state-of-the-art segmentation networks, i. e. SegNet and U-Net, respectively. The experiment results suggested that in both quantitative and visual evaluation, our proposed model outperformed other two models in urban building semantic segmentation tasks in terms of F1 score, Kappa coefficient and overall accuracy, moreover, it also takes the advantage of having much less parameters compared with other two state-of-art models in comparison.

Many objects in the naturalenvironment are generated from the background and even transformed by nature or human beings. Thus, they do not have closed and well-defined boundaries in remote sensing imagery. Recently, convolutional neural network (CNN) based object detection achieved great success in the remote sensing field. However, there is no investigation in the literature about the detection of objects with ambiguous boundaries. In this article, taking the case of the potential loess landslide detection, we designed massive experiments to evaluate convolutional neural networks for detecting objects with ambiguous boundaries in remote sensing imagery. We analyzed the evaluated methods comprehensively by comparing the performance on objects with ambiguous boundaries in remote sensing imagery with the performance on ordinary objects in visual imagery. Furthermore, drawing from these analyses, we provided a fundamental principle of object representation and a meaningful suggestion of information learning to detect objects with ambiguous boundaries. We finished this article by presenting several promising directions for detecting objects with ambiguous boundaries to facilitate and spur future research. This article would provide a significant reference and guidance to develop detectors for objects with ambiguous boundaries in remote sensing imagery.

Deep convolutional neural networks have promoted significant progress in building extraction from high-resolution remote sensing imagery. Although most of such work focuses on modifying existing image segmentation networks in computer vision, we propose a new network in this paper, Deep Encoding Network (DE-Net), that is designed for the very problem based on many lately introduced techniques in image segmentation. Four modules are used to construct DE-Net: the inception-style downsampling modules combining a striding convolution layer and a max-pooling layer, the encoding modules comprising six linear residual blocks with a scaled exponential linear unit (SELU) activation function, the compressing modules reducing the feature channels, and a densely upsampling module that enables the network to encode spatial information inside feature maps. Thus, DE-Net achieves state-of-the-art performance on the WHU Building Dataset in recall, F1-Score, and intersection over union (IoU) metrics without pre-training. It also outperformed several segmentation networks in our self-built Suzhou Satellite Building Dataset. The experimental results validate the effectiveness of DE-Net on building extraction from aerial imagery and satellite imagery. It also suggests that given enough training data, designing and training a network from scratch may excel fine-tuning models pre-trained on datasets unrelated to building extraction.

Detecting objects becomes an increasingly important task in very high resolution (VHR) remote sensing imagery analysis. With the development of GPU-computing capability, a growing number of deep convolutional neural networks (CNNs) have been designed to address the object detection challenge. However, compared with CPU, GPU is much more costly. Therefore, GPU-based methods are less attractive in practical applications. In this article, we propose a CPU-based method that is based on convolutional support vector machines (CSVMs) to address the object detection challenge in VHR images. Experiments are conducted on three VHR and two unmanned aerial vehicle (UAV) data sets with very limited training data. Results show that the proposed CSVM achieves competitive performance compared to U-Net which is an efficient CNN-based model designed for small training data sets.

Compared with traditional High Efficiency Video Coding (HEVC), 3D-HEVC introduces multi-view coding and depth map coding, which leads to significant increase in coding complexity. In this paper, we propose a low complexity intra coding algorithm for depth map based on end-to-end edge detection network. Firstly, we use Holistically Nested Edge Detection (HED) network to determine the edge location of the depth map. Secondly, we use Ostu method to divide the output of the HED into foreground region and background region. Finally, the CU size and the candidate list of intra mode are determined according to the region of coding tree unit (CTU). Experimental results demonstrate that the proposed algorithm can reduce the encoding time by 39.56% on average under negligible degradation of coding performance.

Urban land cover and land use mapping plays an important role in urban planning and management. In this paper, novel multi-scale deep learning models, namely ASPP-Unet and ResASPP-Unet are proposed for urban land cover classification based on very high resolution (VHR) satellite imagery. The proposed ASPP-Unet model consists of a contracting path which extracts the high-level features, and an expansive path, which up-samples the features to create a high-resolution output. The atrous spatial pyramid pooling (ASPP) technique is utilized in the bottom layer in order to incorporate multi-scale deep features into a discriminative feature. The ResASPP-Unet model further improves the architecture by replacing each layer with residual unit. The models were trained and tested based on WorldView-2 (WV2) and WorldView-3 (WV3) imageries over the city of Beijing. Model parameters including layer depth and the number of initial feature maps (IFMs) as well as the input image bands were evaluated in terms of their impact on the model performances. It is shown that the ResASPP-Unet model with 11 layers and 64 IFMs based on 8-band WV2 imagery produced the highest classification accuracy (87.1% for WV2 imagery and 84.0% for WV3 imagery). The ASPP-Unet model with the same parameter setting produced slightly lower accuracy, with overall accuracy of 85.2% for WV2 imagery and 83.2% for WV3 imagery. Overall, the proposed models outperformed the state-of-the-art models, e.g., U-Net, convolutional neural network (CNN) and Support Vector Machine (SVM) model over both WV2 and WV3 images, and yielded robust and efficient urban land cover classification results.

As the acquisition of very high resolution (VHR) satellite images becomes easier owing to technological advancements, ever more stringent requirements are being imposed on automatic image interpretation. Moreover, per-pixel classification has become the focus of research interests in this regard. However, the efficient and effective processing and the interpretation of VHR satellite images remain a critical task. Convolutional neural networks (CNNs) have recently been applied to VHR satellite images with considerable success. However, the prevalent CNN models accept input data of fixed sizes and train the classifier using features extracted directly from the convolutional stages or the fully connected layers, which cannot yield pixel-to-pixel classifications. Moreover, training a CNN model requires large amounts of labeled reference data. These are challenging to obtain because per-pixel labeled VHR satellite images are not open access. In this paper, we propose a framework called the unsupervised-restricted deconvolutional neural network (URDNN). It can solve these problems by learning an end-to-end and pixel-to-pixel classification and handling a VHR classification using a fully convolutional network and a small number of labeled pixels. In URDNN, supervised learning is always under the restriction of unsupervised learning, which serves to constrain and aid supervised training in learning more generalized and abstract feature. To some degree, it will try to reduce the problems of overfitting and undertraining, which arise from the scarcity of labeled training data, and to gain better classification results using fewer training samples. It improves the generality of the classification model. We tested the proposed URDNN on images from the Geoeye and Quickbird sensors and obtained satisfactory results with the highest overall accuracy (OA) achieved as 0.977 and 0.989, respectively. Experiments showed that the combined effects of additional kernels and stages may have produced better results, and two-stage URDNN consistently produced a more stable result. We compared URDNN with four methods and found that with a small ratio of selected labeled data items, it yielded the highest and most stable results, whereas the accuracy values of the other methods quickly decreased. For some categories with fewer training pixels, accuracy for categories from other methods was considerably worse than that in URDNN, with the largest difference reaching almost 10%. Hence, the proposed URDNN can successfully handle the VHR image classification using a small number of labeled pixels. Furthermore, it is more effective than state-of-the-art methods.