Aircraft segmentation in remote sensing images based on multi-scale residual U-Net with attention

Segmenting objects from cluttered backgrounds in single-channel images, such as marine radar echoes, medical images, and remote sensing images, poses significant challenges due to limited texture, color information, and diverse target types. This paper proposes a novel solution: the Onet, an O-shaped assembly of twin U-Net deep neural networks, designed for unsupervised binary semantic segmentation. The Onet, trained with an intensity-complementary image pair and without the need for annotated labels, maximizes the Jensen-Shannon divergence (JSD) between the densely localized features and the class probability maps. By leveraging the symmetry of U-Net, Onet subtly strengthens the dependence between dense local features, global features, and class probability maps during the training process. The design of the complementary input pair aligns with the theoretical requirement that optimizing JSD needs the class probability of negative samples to accurately estimate the marginal distribution. Compared to the current leading unsupervised segmentation methods, the Onet demonstrates superior performance in target segmentation in marine radar frames and cloud segmentation in remote sensing images. Notably, we found that Onet's foreground prediction significantly enhances the signal-to-noise ratio (SNR) of targets amidst marine radar clutter. Onet's source code is publicly accessible at https://github.com/joeyee/Onet.

Semantic segmentation in remote sensing images is a very challenging task. Each pixel in a remote sensing image has a semantic meaning to it and automatic annotation of each pixel remains as an open challenge for the research community due to its high spatial resolution. To address this issue deep learning based encoder-decoder architectures like SegNet and ResNet that is widely used for computer vision dataset is adopted for remote sensing images and its performance is analyzed based on the pixel wise classification accuracy. From the experiment conducted it is inferred that SegNet suffers from degradation problem when the depth of the network is increased with an overall accuracy of about 86.086% whereas the Residual network manages to overcome the degradation effect with an overall accuracy of about 87.747%.

Accurate evaluation of orchard areas from remote sensing images is of great importance in economic and ecological aspects. In practice, the differences in distributions between remote sensing images and the lack of data labels make the semantic segmentation model impossible to use in new data. Unsupervised domain adaptation (UDA) methods can improve the performance of the model in the target domain by aligning the source domain and the target domain. However, due to the class mismatch problem and the interference of high-dimensional feature complexity, most UDA methods cannot achieve satisfactory results in orchard areas segmentation task. To address these issues, we propose an UDA model for orchard areas segmentation by developing a class feature aggregate discriminator. The class feature aggregate discriminator is designed to distinguish intra-domain classes and align inter-domain classes, and class feature aggregate can represent class information of different domains, which helps the model to avoid the interference of complex information. In addition, adversarial loss reweighting is introduced to the novel model, which makes the segmentation model pay more attention to the orchard areas. To verify the effectiveness of the proposed method, we conducted extensive experiments in three different remote sensing images around Yichang City. Compared to the baseline model, the proposed approach improves IoU by 27.68%, and we achieve high gains of 6.07% in IoU over other UDA methods. The larger gain indicates that our proposed method has great potential in cross-domain orchard areas segmentation.

Remote sensing images are usually characterized by complex backgrounds, scale and orientation variations, and large intraclass variance. General semantic segmentation methods usually fail to fully investigate the above issues, and thus their performances on remote sensing image segmentation are limited. In this article, we propose our LOGCAN++, a semantic segmentation model customized for remote sensing images, which is made up of a global class-aware (GCA) module and several local class-aware (LCA) modules. The GCA module captures global representations for class-level context modeling to reduce the interference of background noise. The LCA module generates local class representations as intermediate perceptual elements to indirectly associate pixels with the global class representations, targeting dealing with the large intraclass variance problem. In particular, we introduce affine transformations in the LCA module for adaptive extraction of local class representations to effectively tolerate scale and orientation variations in remote sensing images. Extensive experiments on three benchmark datasets show that our LOGCAN++ outperforms current mainstream general and remote sensing semantic segmentation methods and achieves a better trade-off between speed and accuracy.

Airport runways, as the core part of airports, belong to vital national infrastructure, and the target detection and segmentation of airport runways in remote sensing images using deep learning methods have significant research value. Most of the existing airport target detection methods based on deep learning rely on horizontal bounding boxes for localization, which often contain irrelevant background information. Moreover, when detecting multiple intersecting airport runways in a single remote sensing image, issues such as false positives and false negatives are apt to occur. To address these challenges, this study proposes an end-to-end remote sensing image airport runway detection and segmentation method based on an improved Mask RCNN (CR-Mask RCNN). The proposed method uses a rotated region generation network instead of a non-rotated region generation network, allowing it to generate rotated bounding boxes that fit the shape of the airport runway more closely, thus avoiding the interference of a large amount of invalid background information brought about by horizontal bounding boxes. Furthermore, the method incorporates an attention mechanism into the backbone feature extraction network to allocate attention to different airport runway feature map scales, which enhances the extraction of local feature information, captures detailed information more effectively, and reduces issues of false positives and false negatives when detecting airport runway targets. The results indicate that, when comparing horizontal bounding boxes with rotated bounding boxes for detecting and segmenting airport runways, the latter are more precise for complex backgrounds. Furthermore, incorporating an attention mechanism enhances the accuracy of airport runway recognition, making it highly effective and practical.

Cloud is always the weak and even uninformative area inevitably existing in the remote sensing images, and greatly limits the development of remote sensing application. Accurate and automatic detection and segmentation of clouds in satellite scenes is a key problem for the application of remote sensing images. In this paper, we propose a fully convolutional network (FCN) based on deep learning framework for cloud segmentation from remote sensing images. First, the obtained remote sensing image is pre-processed by using band merging and size cropping. Second the image is input into the FCN to achieve the cloud segmentation. To demonstrate the effectiveness of the method, we conduct experiments on Landsat-8 satellite dataset. The overall accuracy of our proposed method for cloud segmentation is higher than 88%. Experimental results demonstrate that both thin and thick cloud can be well segmented with higher accuracy and robustness by using our method.

In recent years, advancements in remote sensing image observation technology have significantly enriched the surface feature information captured in remote sensing images, posing greater challenges for semantic information extraction from remote sensing imagery. While convolutional neural networks (CNNs) excel at understanding relationships between adjacent image regions, processing multidimensional data requires reliance on attention mechanisms. However, due to the inherent complexity of remote sensing images, most attention mechanisms designed for natural images underperform when applied to remote sensing data. To address these challenges in remote sensing image semantic segmentation, we propose a highly generalizable multi-branch attention fusion method based on shallow and deep features. This approach applies pixel-wise, spatial, and channel attention mechanisms to feature maps fused with shallow and deep features, thereby enhancing the network’s semantic information extraction capability. Through evaluations on the Cityscapes, LoveDA, and WHDLD datasets, we validate the performance of our method in processing remote sensing data. The results demonstrate consistent improvements in segmentation accuracy across most categories, highlighting its strong generalization capability. Specifically, compared to baseline methods, our approach achieves average mIoU improvements of 0.42% and 0.54% on the WHDLD and LoveDA datasets, respectively, significantly enhancing network performance in complex remote sensing scenarios.

The imaging process of optical remote sensing images are easily affected by external conditions. Therefore, remote sensing images under different imaging conditions often show color differences, resulting in feature distribution differences between the source and target domain, hindering the migration of semantic segmentation models between domains. Currently, most domain adaptation methods are for single-source and single-target domains. Here, we proposed a novel and concise method, coined MDANet, for the adaptation of patch images of multi-source and multi-target domains and for reducing the distribution differences of different patch images by projecting them onto the virtual center of a mixed-domain. MDANet is a lightweight and self-supervised network that can be grafted with any semantic segmentation model. Our method significantly improved the segmentation accuracy of semantic segmentation models and showed higher stability and competitiveness than existing methods.

With the proliferation of remote sensing images, how to segment buildings more accurately in remote sensing images is a critical challenge. First, the high resolution leads to blurred boundaries in the extracted building maps. Second, the similarity between buildings and background results in intraclass inconsistency. To address these two problems, we propose an UNet-based network named Context-Transfer-UNet (CT-UNet). Specifically, we design Dense Boundary Block (DBB). Dense Block utilizes reuse mechanism to refine features and increase recognition capabilities. Boundary Block introduces the low-level spatial information to solve the fuzzy boundary problem. Then,to handle intra-class inconsistency, we construct Spatial Channel Attention Block (SCAB). Finally, we propose a novel loss function to enhance the purpose of loss by adding evaluation indicator.

Semantic segmentation methods based on deep neural networks have achieved great success in recent years. However, training such deep neural networks relies heavily on a large number of images with accurate pixel-level labels, which requires a huge amount of human effort, especially for large-scale remote sensing images. In this paper, we propose a point-based weakly supervised learning framework called the deep bilateral filtering network (DBFNet) for the semantic segmentation of remote sensing images. Compared with pixel-level labels, point annotations are usually sparse and cannot reveal the complete structure of the objects; they also lack boundary information, thus resulting in incomplete prediction within the object and the loss of object boundaries. To address these problems, we incorporate the bilateral filtering technique into deeply learned representations in two respects. First, since a target object contains smooth regions that always belong to the same category, we perform deep bilateral filtering (DBF) to filter the deep features by a nonlinear combination of nearby feature values, which encourages the nearby and similar features to become closer, thus achieving a consistent prediction in the smooth region. In addition, the DBF can distinguish the boundary by enlarging the distance between the features on different sides of the edge, thus preserving the boundary information well. Experimental results on two widely used datasets, the ISPRS 2-D semantic labeling Potsdam and Vaihingen datasets, demonstrate that our proposed DBFNet can achieve a highly competitive performance compared with state-of-the-art fully-supervised methods. Code is available at https://github.com/Luffy03/DBFNet.

There is a large amount of out-of-distribution data (OOD) in remote sensing, which hinders high-accuracy segmentation models under the assumption of independent identical distribution (i.i.d.) from stable and reliable performance in real-world remote sensing applications. And Domain Adaptation (DA) is presented to seamlessly extend classifiers to the label-scarce target domain in the presence of the label-sufficient source domain with different data distributions. However, given that the domain shift, i.e. the distribution difference between the two domains, is more serious in remote sensing images, the current DA methods for image segmentation in Computer Vision (CV) typically perform unsatisfactorily in remote sensing, even suffering from the negative domain alignment. To this end, this paper proposes the Self-Training Adversarial Domain Adaptation (STADA) method for remote sensing image segmentation, which not only performs adversarial learning to extract domain-invariant features, but also implements Self-Training using pseudo-labels in the target domain denoised by the conditional adversarial loss for classifier adaptation. The ISPRS and WHU datasets are employed to conduct extensive experiments to investigate the effectiveness of STADA and the specific effect of its each DA component. And the experimental results demonstrate that STADA outperforms other state-of-the-art DA methods in the remote sensing image segmentation task.

Semantic segmentation of high-resolution remote sensing images is an important and challenging task. Aiming at the difficulty of accurate segmentation of confusing objects in high-resolution remote sensing images, we propose a full-scale feature extraction network (FSFN et). The proposed architecture follows the encoder-decoder paradigm. The encoder adopts ResNeXt50 to fully extract features, and inputs the four-scale output of the encoder to the global feature extraction module (GFM) as part of the decoder input. The decoder uses group convolution residual module combined with attention mechanism (GRM) to refine features, and employs dense connection (DC) to strengthen feature propagation. In addition, the third and fourth decoding blocks are added with deep supervision (DS) to speed up the convergence of the network. We evaluate our proposed architecture on the Vaihingen and Potsdam datasets provided by the International Society for Photogrammetry and Remote Sensing (ISPRS). The experimental results show that the proposed approach can significantly improve the segmentation performance, reaching 89.8% and 89.3% overall accuracy (OA) on the two datasets, respectively.

Few-Shot Semantic Segmentation (FSS) has drawn massive attention recently due to its remarkable ability to segment novel-class objects given only a handful of support samples. However, current FSS methods mainly focus on natural images and pay little attention to more practical and challenging scenarios, e.g., remote sensing image segmentation. In the field of remote sensing image analysis, the characteristics of remote sensing images, like complex backgrounds and tiny foreground objects, make novel-class segmentation challenging. To cope with these obstacles, we propose a Class-Aware Self- and Cross-Attention Network (CSCANet) for FSS in remote sensing imagery, consisting of a lightweight self-attention module and a supervised prior-guided cross-attention module. Concretely, the self-attention module abstracts robust unseen-class information from support features, while the cross-attention module generates a superior quality query attention map for directing the network to focus on novel objects. Experiments demonstrate that our CSCANet achieves outstanding performance on the standard remote sensing FSS benchmark iSAID-5i, surpassing the existing state-of-the-art FSS models across all combinations of backbone networks and K-shot settings.

In weakly supervised semantic segmentation (WSSS) tasks on remote sensing images, it is a common practice to train a classification network from scratch using a large batch of images with a limited number of classes. Subsequently, class activation maps are extracted from the model based on predefined class indices, and these maps are then optimized to obtain pseudolabels. To make this strategy effective when introducing a new class, a substantial amount of data needs to be provided to the model. In this article, we present an innovative framework, RS-TextWS-Seg, designed to efficiently generate high-quality segmentation results for a wide range of remote sensing objects using concise descriptions. Our proposed framework comprises three sequential stages: initially, we undertake parameter fine-tuning of the contrastive language-image pretraining (CLIP) model to swiftly strengthen its capacity for zero-shot detection of a limited number of remote sensing features. Subsequently, we introduce a text-driven background suppression mechanism aimed at deriving class activation maps from the refined CLIP model based on textual cues, while concurrently mitigating background noises. Finally, we use the segment anything model (SAM) to refine the edges of the extracted class activation map. We widely researched the leading-edge methodologies in WSSS and conducted a range of comparative experiments and ablation studies to prove the efficacy of our proposed framework. The research findings underscore that RS-TextWS-Seg outperforms other state-of-the-art methods on renowned datasets such as DLRSD and Potsdam, as well as on bespoke datasets specifically curated for overground petroleum pipelines and oil well fields.

The image is a composition of many pixels. These pixels include two pieces of information: coordinate or position and intensity value. The image includes several objects; extracting the crucial objects from the image is critical. Based on the similarity of patterns, classes, groups, and segments of contained objects in the image can be created. Assigning the labels to the pixels is necessary to make the image more informative for analyzing features and decision-making. This study addresses segmentation techniques and classifying images pertaining to remote sensing images. Thereafter, Land Use Land Cover (LULC) mapping is discussed, which classifies the remote sensing images.