Two-level active learning method for debris detection using VHR satellite imagery and local aerial surveys

Sea fog detection (SFD) presents a significant challenge in the field of intelligent Earth observation, particularly in analyzing meteorological satellite imagery. Akin to various vision tasks, ImageNet pre-training is commonly used for pre-training SFD. However, in the context of multi-spectral meteorological satellite imagery, the initial step of deep learning has received limited attention. Recently, pre-training with Very High-Resolution (VHR) satellite imagery has gained increased popularity in remote-sensing vision tasks, showing the potential to replace ImageNet pre-training. However, it is worth noting that the meteorological satellite imagery applied in SFD, despite being an application of computer vision in remote sensing, differs greatly from VHR satellite imagery. To address the limitation of pre-training for SFD, this paper introduces a novel deep-learning paradigm to the meteorological domain driven by Masked Image Modeling (MIM). Our research reveals two key insights: (1) Pre-training with meteorological satellite imagery yields superior SFD performance compared to pre-training with nature imagery and VHR satellite imagery. (2) Incorporating the architectural characteristics of SFD models into a vanilla masked autoencoder (MAE) can augment the effectiveness of meteorological pre-training. To facilitate this research, we curate a pre-training dataset comprising 514,655 temporal multi-spectral meteorological satellite images, covering the Bohai Sea and Yellow Sea regions, which have the most sea fog occurrence. The longitude ranges from 115.00E to 128.75E, and the latitude ranges from 27.60N to 41.35N. Moreover, we introduce SeaMAE, a novel MAE that utilizes a Vision Transformer as the encoder and a convolutional hierarchical decoder, to learn meteorological representations. SeaMAE is pre-trained on this dataset and fine-tuned for SFD, resulting in state-of-the-art performance. For instance, using the ViT-Base as the backbone, SeaMAE pre-training which achieves 64.18% surpasses from-scratch learning, natural imagery pre-training, and VRH satellite imagery pre-training by 5.53%, 2.49%, and 2.21%, respectively, in terms of Intersection over Union of SFD.

Detecting sparse and multi-sized objects in very high resolution (VHR) remote sensing images remains a significant challenge in satellite imagery applications and analytics. Difficulties include broad geographical scene distributions and high pixel counts in each image: a large-scale satellite image contains tens to hundreds of millions of pixels and dozens of complex backgrounds. Furthermore, the scale of the same category object can vary widely (e.g., ships can measure from several to thousands of pixels). To address these issues, here we propose the Big Map R-CNN method to improve object detection in VHR satellite imagery. Big Map R-CNN introduces mean shift clustering for quadric detecting based on the existing Mask R-CNN architecture. Big Map R-CNN considers four main aspects: 1) big map cropping to generate small size sub-images; 2) detecting these sub-images using the typical Mask R-CNN network; 3) screening out fragmented low-confidence targets and collecting uncertain image regions by clustering; 4) quadric detecting to generate prediction boxes. We also introduce a new large-scale and VHR remote sensing imagery dataset containing two categories (RSI LS-VHR-2) for detection performance verification. Comprehensive evaluations on RSI LS-VHR-2 dataset demonstrate the effectiveness of the proposed Big Map R-CNN algorithm for object detection in large-scale remote sensing images.

Satellite imagery has been used to improve scientific research worldwide. In this study, the southern right whale (Eubalaena australis) was chosen to test the use of medium, high, and very high resolution (VHR) satellite images, on the Brazilian breeding ground. These images were used to identify the whales and were compared to aerial survey data collected in the same area. The VHR satellite images from the Pleiades‐1A satellite, available on Google Earth, displayed the best results when compared to those from Sentinel 2, Landsat 8, Rapid Eye, and Planet Scope. No significant differences were observed (Mann‐Whitney U test) between the possible whales recorded in the satellite images and of real whales recorded in situ by aerial surveys, while considering either the number of groups (p = .841, n = 5) or the total number of animals (p = .222, n = 5). Further, when using VHR images, the geographical positions of the whales recorded in situ had a positive correlation with the positions generated from the satellite images (Mantel test: r = 0.52, p = .001, n = 13). This technique may represent an important tool for detecting right whales, especially in countries where research funding is scarce.

Pléiades satellite images for deriving forest metrics in the Alpine region

Abstract. The Very High Resolution (VHR) satellite imageries such us Pleiades, WorldView-2, GeoEye-1 used for precise mapping purpose must be corrected from any distortion to achieve the expected accuracy. Orthorectification is performed to eliminate geometric errors of the VHR satellite imageries. Orthorectification requires main input data such as Digital Elevation Model (DEM) and Ground Control Point (GCP). The VHR systematic-ortho imageries were generated using SRTM 30m DEM without using any GCP data. The accuracy value differences of VHR systematic-ortho imageries and VHR orthorectified imageries using GCP currently is not exactly defined. This study aimed to identified the accuracy comparison of VHR systematic-ortho imageries against orthorectified imageries using GCP. Orthorectified imageries using GCP created by using Rigorous model. Accuracy evaluation is calculated by using several independent check points.

Case instance segmentation of small farmland based on Mask R-CNN of feature pyramid network with double attention mechanism in high resolution satellite images

Monitoring the variation of urban expansion is crucial for sustainable urban planning and cultural heritage management. This paper proposes an approach for the semantic segmentation of very-high-resolution (VHR) satellite imagery to detect the changes in urban sprawl in the surroundings of Chan Chan, a UNESCO World Heritage Site in Peru. This study explores the effectiveness of combining Mix Transformer encoders with U-Net architectures to improve feature extraction and spatial context understanding in VHR satellite imagery. The integration of active contour loss functions further enhances the model’s ability to delineate complex urban boundaries, addressing the challenges posed by the heterogeneous landscape surrounding the archaeological complex of Chan Chan. The results demonstrate that the proposed approach achieves accurate semantic segmentation on images of the study area from different years. Quantitative results showed that the U-Net-scse model with an MiTB5 encoder achieved the best performance with respect to SegFormer and FT-UNet-Former, with IoU scores of 0.8288 on OpenEarthMap and 0.6743 on Chan Chan images. Qualitative analysis revealed the model’s effectiveness in segmenting buildings across diverse urban and rural environments in Peru. Utilizing this approach for monitoring urban expansion over time can enable managers to make informed decisions aimed at preserving cultural heritage and promoting sustainable urban development.

Semantic segmentation of remote sensing images enables in particular land-cover map generation for a given set of classes. Very recent literature has shown the superior performance of deep convolutional neural networks (DCNN) for many tasks, from object recognition to semantic labelling, including the classification of Very High Resolution (VHR) satellite images. However, while plethora of works aim at improving object delineation on geographically restricted areas, few tend to solve this classification task at very large scales. New issues occur such as intra-class class variability, diachrony between surveys, and the appearance of new classes in a specific area, that do not exist in the predefined set of labels. Therefore, this work intends to (i) perform large scale classification and to (ii) expand a set of land-cover classes, using the off-the-shelf model learnt in a specific area of interest and adapting it to unseen areas.

Reliable building delineation in very high resolution (VHR) satellite imagery can be achieved by precise disparity information extracted from stereo pairs. However, off-nadir VHR images over urban areas contain many occlusions due to building leaning that creates gaps in the extracted disparity maps. The typical approach to fill these gaps is by interpolation. However, it inevitably degrades the quality of the disparity map and reduces the accuracy of building detection. Thus, this research proposes a registration-based technique for mapping the disparity of off-terrain objects to avoid the need for disparity interpolation and normalization. The generated disparity by the proposed technique is then used to support building detection in off-nadir VHR satellite images. Experiments in a high-rise building area confirmed that 75 percent of the detected building roofs overlap precisely the reference data, with almost 100 percent correct detection. These accuracies are substantially higher than those achieved by other published research.

Superpixels, as a state-of-the-art segmentation paradigm, have recently been widely used in computer vision and pattern recognition. Despite the effectiveness of these algorithms, there are still many limitations and challenges dealing with Very High-Resolution (VHR) satellite images especially in complex urban scenes. In this paper, we develop a superpixel algorithm as a modified edge-based version of Simple Linear Iterative Clustering (SLIC), which is here called ESLIC, compatible with VHR satellite images. Then, based on the modified properties of generated superpixels, a heuristic multi-scale approach for building extraction is proposed, based on the stereo satellite imagery along with the corresponding Digital Surface Model (DSM). First, to generate the modified superpixels, an edge-preserving term is applied to retain the main building boundaries and edges. The resulting superpixels are then used to initially refine the stereo-extracted DSM. After shadow and vegetation removal, a rough building mask is obtained from the normalized DSM, which highlights the appropriate regions in the image, to be used as the input of a multi-scale superpixel segmentation of the proper areas to determine the superpixels inside the building. Finally, these building superpixels with different scales are integrated and the output is a unified building mask. We have tested our methods on building samples from a WorldView-2 dataset. The results are promising, and the experiments show that superpixels generated with the proposed ESLIC algorithm are more adherent to the building boundaries, and the resulting building mask retains urban object shape better than those generated with the original SLIC algorithm.

As the acquisition of very high resolution (VHR) satellite images becomes easier owing to technological advancements, ever more stringent requirements are being imposed on automatic image interpretation. Moreover, per-pixel classification has become the focus of research interests in this regard. However, the efficient and effective processing and the interpretation of VHR satellite images remain a critical task. Convolutional neural networks (CNNs) have recently been applied to VHR satellite images with considerable success. However, the prevalent CNN models accept input data of fixed sizes and train the classifier using features extracted directly from the convolutional stages or the fully connected layers, which cannot yield pixel-to-pixel classifications. Moreover, training a CNN model requires large amounts of labeled reference data. These are challenging to obtain because per-pixel labeled VHR satellite images are not open access. In this paper, we propose a framework called the unsupervised-restricted deconvolutional neural network (URDNN). It can solve these problems by learning an end-to-end and pixel-to-pixel classification and handling a VHR classification using a fully convolutional network and a small number of labeled pixels. In URDNN, supervised learning is always under the restriction of unsupervised learning, which serves to constrain and aid supervised training in learning more generalized and abstract feature. To some degree, it will try to reduce the problems of overfitting and undertraining, which arise from the scarcity of labeled training data, and to gain better classification results using fewer training samples. It improves the generality of the classification model. We tested the proposed URDNN on images from the Geoeye and Quickbird sensors and obtained satisfactory results with the highest overall accuracy (OA) achieved as 0.977 and 0.989, respectively. Experiments showed that the combined effects of additional kernels and stages may have produced better results, and two-stage URDNN consistently produced a more stable result. We compared URDNN with four methods and found that with a small ratio of selected labeled data items, it yielded the highest and most stable results, whereas the accuracy values of the other methods quickly decreased. For some categories with fewer training pixels, accuracy for categories from other methods was considerably worse than that in URDNN, with the largest difference reaching almost 10%. Hence, the proposed URDNN can successfully handle the VHR image classification using a small number of labeled pixels. Furthermore, it is more effective than state-of-the-art methods.

Road extraction from very high-resolution (VHR) satellite images plays an important role in the remote-sensing applications. Although tracing road features in satellite images has received much attention over the years, fully automated methods appear to be in their infancy. To tackle these limitations to some extent, this paper presents a novel object-based automatic road extraction method. The proposed method consists of five main steps. First, satellite images are segmented to generate objects. Then, two object-based filters are applied to compute object features to select road candidates. After that, the road class is extracted using the support vector machine (SVM) based on the extracted feature set. Finally, tensor voting (TV), active contour, and the geometrical information are integrated to eliminate road gaps and improve road smoothness. Experiments are conducted on nine test sites. It is experimentally demonstrated that the proposed method produces an excellent accuracy for the automatic road extraction from VHR satellite images.

Pine wilt disease has caused severe damage to China’s forest ecosystems. Utilizing the rich information from very-high-resolution (VHR) satellite imagery for large-scale and accurate monitoring of pine wilt disease is a crucial approach to curbing its spread. However, current research on identifying infected trees using VHR satellite imagery and deep learning remains extremely limited. This study introduces several advanced self-attention algorithms into the task of satellite-based monitoring of pine wilt disease to enhance detection performance. We constructed a dataset of discolored pine trees affected by pine wilt disease using imagery from the Gaofen-2 and Gaofen-7 satellites. Within the unified semantic segmentation framework MMSegmentation, we implemented four single-head attention models—NLNet, CCNet, DANet, and GCNet—and two multi-head attention models—Swin Transformer and SegFormer—for the accurate semantic segmentation of infected trees. The model predictions were further analyzed through visualization. The results demonstrate that introducing appropriate self-attention algorithms significantly improves detection accuracy for pine wilt disease. Among the single-head attention models, DANet achieved the highest accuracy, reaching 73.35%. The multi-head attention models exhibited an excellent performance, with SegFormer-b2 achieving an accuracy of 76.39%, learning the features of discolored pine trees at the earliest stage and converging faster. The visualization of model inference results indicates that DANet, which integrates convolutional neural networks (CNNs) with self-attention mechanisms, achieved the highest overall accuracy at 94.43%. The use of self-attention algorithms enables models to extract more precise morphological features of discolored pine trees, enhancing user accuracy while potentially reducing production accuracy.

Spatio-temporal analysis of Kilim River using Very High-Resolution (VHR) satellite image between the years of 2005 and 2012 are vital to assess river morphological changes and erosion detection over the specific time duration. Therefore, this study utilizes remote sensing and Geographical Information System (GIS) capabilities to identify the channel migration and shifting as well as to calculate the rate of erosion and accretion. Classification of the river from Quickbird and WorldView-2 images is carry out using Maximum Likelihood Classification (MLC) technique whereas feature selection tool performs to extract river feature from the raw image. Subsequently, river feature been overlaid to identify the changes of riverbank pattern and to calculate the rate of riverbank erosion in terms of area. The result shows the total area of erosion is 27252.959 m2 and the total area of accretion is 6079.999 m2. A maximum erosion of 6993.102 m2 detected at Section E while maximum accretion of 681.026 m2 spotted at Section D. The rates of riverbank erosion are 3406.62 m2/year while the rates for accretion are 759.999 m2/years between 2005 and 2012. Hence, the output of this study enables Langkawi Development Authority (LADA) and other stakeholders to recognize the specific location, which severely affected by erosion and accretion as well as to spot the area experienced huge river channel shifting.

Third-dimension information is of a great importance for several remote sensing applications, such as building detection. The main data-source for these applications is very high resolution (VHR) satellite images which allow detailed mapping of complex environments. Stereo VHR satellite images allow the extraction of two correlated types of third-dimension information: disparity and elevation information. While the disparity is measured directly, the elevation information is derived computationally. To measure the disparity information, two overlapped images are matched. However, for the backward and forward off-nadir VHR stereo images, building facades occlude areas and hence create many data gaps. When the disparity is required to represent only the off-terrain objects, interpolation and normalization techniques are typically used. However, in dense urban environments, these techniques destroy the quality of the generated data. Therefore, this paper proposes a registration-based technique to measure the disparity of the above-ground objects. The technique includes constructing epipolar images and registering them using common terrain- level features to allow direct disparity mapping for the off-terrain objects. After the implementation, the negative effects of occlusion in the off-nadir VHR stereo images are mitigated through direct disparity mapping of the above-ground objects and bypassing the interpolation and normalization steps.