Dynamic Informative Proposal-Based Iterative Training Procedure for Weakly Supervised Object Detection in Remote Sensing Images

Weakly supervised object detection (WSOD) in remote sensing images (RSI) only require image-level labels to detect various objects. Most of the WSOD methods incline to capture the most discriminative parts of object rather than the entire object, and the number of easy and hard samples is imbalanced. To address the first problem, a novel metric named objectness score (OS) is proposed and incorporated into the training loss of our WSOD model. The OS is consisted of the traditional class confidence score (CCS) and the object completeness prior score (OCPS). The CCS can provide the probability that a proposal belongs to a certain class, and the OCPS can quantify the completeness that a proposal covers the entire object. Therefore, the samples which cover the entire object with high class confidences will be assigned large weight in the training loss through OS. To handle the second problem, a novel metric named difficulty evaluation score (DES) is proposed and also incorporated into the training loss. The DES is calculated by using the entropy of confidence score vector of each proposal and is used to quantify how difficult a proposal can be identified correctly, consequently, the hard samples will also be assigned large weight in the training loss through DES. The ablation experiments on two RSI datasets verify the effectiveness of the proposed OS and DES. The comprehensive quantitative and subjective evaluations demonstrate that our method inclines to detect the entire object accurately, and surpasses seven state-of-the-art WSOD methods.

Weakly supervised object detection (WSOD) in remote sensing images (RSIs) has attracted lots of attention because it solely employs image-level labels to drive the model training. Most of the WSOD methods incline to mine salient object as positive instance, and the less salient objects are considered as negative instances, which will cause the problem of missing instances. In addition, the quantity of hard and easy instances is usually imbalanced, consequently, the cumulative loss of a large amount of easy instances dominates the training loss, which limits the upper bound of WSOD performance. To handle the first problem, a complementary detection network (CDN) is proposed, which consists of a complementary multiple instance detection network (CMIDN) and a complementary feature learning (CFL) module. The CDN can capture robust complementary information from two basic multiple instance detection networks (MIDNs) and mine more object instances. To handle the second problem, an instance difficulty evaluation metric named instance difficulty score (IDS) is proposed, which is employed as the weight of each instance in the training loss. Consequently, the hard instances will be assigned larger weights according to the IDS, which can improve the upper bound of WSOD performance. The ablation experiments demonstrate that our method significantly increases the baseline method by large margins, i.e. 23.6% (10.2%) mAP and 32.4% (13.1%) CorLoc gains on the NWPU VHR-10.v2 (DIOR) dataset. Our method obtains 58.1% (26.7%) mAP and 72.4% (47.9%) CorLoc on the NWPU VHR-10.v2 (DIOR) dataset, which achieves better performance compared with seven advanced WSOD methods.

Weakly supervised object detection (WSOD) in remote sensing images (RSIs) aims to detect high-value targets by solely utilizing image-level category labels; however, two problems have not been well addressed by existing methods. Firstly, the seed instances (SIs) are mined solely relying on the category score (CS) of each proposal, which is inclined to concentrate on the most salient parts of the object; furthermore, they are unreliable because the robustness of the CS is not sufficient due to the fact that the inter-category similarity and intra-category diversity are more serious in RSIs. Secondly, the localization accuracy is limited by the proposals generated by the selective search or edge box algorithm. To address the first problem, a segment anything model (SAM)-induced seed instance-mining (SSIM) module is proposed, which mines the SIs according to the object quality score, which indicates the comprehensive characteristic of the category and the completeness of the object. To handle the second problem, a SAM-based pseudo-ground truth-mining (SPGTM) module is proposed to mine the pseudo-ground truth (PGT) instances, for which the localization is more accurate than traditional proposals by fully making use of the advantages of SAM, and the object-detection heads are trained by the PGT instances in a fully supervised manner. The ablation studies show the effectiveness of the SSIM and SPGTM modules. Comprehensive comparisons with 15 WSOD methods demonstrate the superiority of our method on two RSI datasets.

A weakly supervised object detection (WSOD) task is to train a detector with only image-level labels provided. Except for the training difficulty introduced by weaker annotations, the inherent complexity of the remote-sensing images (RSIs) also adds to the challenge. To boost the detector’s localization accuracy, we aim to exploit more semantic information contained in images and help improve the general robustness of the model. Noticing previous methods tend to focus on the most discriminative part of an object, we design a self-training-based network that leverages local semantic features. To this end, we develop a semantic-balanced localization module (SBLM) that distinguishes foreground from background and accurate proposals from incomplete ones, by leveraging a balance of region of interest (ROI) and its context information. Moreover, we find that the self-training strategy highly relies on the quality of pseudo-ground-truth boxes. Motivated by this possible lack of robustness, we design a comprehensive clustering module (CCM) and saliency-based proposal filtering (SPF) module that select pseudo-ground truth more comprehensively under supervision. To be more specific, CCM aims to reduce the arbitrariness during assigning pseudo-labels by considering multiple categorical vectors simultaneously. Salient object detection (SOD) is applied in the SPF module to help evaluate the quality of the chosen pseudo-ground-truth boxes. The detection performance is significantly boosted with the proposed method. Extensive experiments conducted on the NWPU VHR-10.v2 dataset and the DIOR dataset validate that the proposed model outperforms the previous state-of-the-art methods favorably with an mAP of 64.9% and 28.1%, respectively.

The presence of shadows in high-resolution (HR) remote-sensing images reduces object detection accuracy. To address this problem, in this paper, we proposed a deep neural network algorithm for shadow detection by using the AISD and SSAD remote-sensing shadow image datasets. To improve the ability to extract spatial information from feature maps, we developed a cross-spatial attention module that focuses on semantic information in the horizontal and vertical directions at each position point on the remote-sensing image. This module overcomes the limitations of existing technologies in accurately judging small areas and suspected shadow areas and in missing or incorrectly detected shadow areas. In addition, to improve the ability to extract shadow features and the accuracy of shadow detection in remote-sensing images, we developed a channel attention module that assigns more attention to channels that conform to the shadow color characteristics. The network architecture comprises an encoder – decoder structure, with ResNeXt50 used as the backbone for the encoder and a multi resolution parallel fusion (MRPF) designed for the decoder; cross-spatial and channel attention were incorporated into the decoder unit. Experimental results demonstrated the superior performance of the proposed algorithm, with an F1 score of 92.6% for the shadow category on the test set, thus, outperforming other algorithms and making the proposed method an effective solution for shadow detection in HR remote-sensing images.

ABSTRACTIn this article, we propose a novel difference image (DI) creation method for unsupervised change detection in multi-temporal multi-spectral remote-sensing images based on deep learning theory. First, we apply deep belief network to learn local and high-level features from the local neighbour of a given pixel in an unsupervised manner. Second, a back propagation algorithm is improved to build a DI based on selected training samples, which can highlight the difference on changed regions and suppress the false changes on unchanged regions. Finally, we get the change trajectory map using simple clustering analysis. The proposed scheme is tested on three remote-sensing data sets. Qualitative and quantitative evaluations show its superior performance compared to the traditional pixel-level and texture-level-based approaches.

Withcontinuous advancements in remote sensing technology, object detection in remote sensing images has emerged as a critical research direction in maritime surveillance, port management, and national defense. Among these applications, ship detection is a key task. Due to the fact that ships in images typically exhibit arbitrary rotations, multi-scale distributions, and complex backgrounds, conventional detection methods based on horizontal or rotated bounding boxes often fail to adequately capture the fine-grained information of the targets, thereby compromising detection accuracy. This paper proposes the Dense Representative Points-Guided Rotated-Ship Detection (DenseRRSD) method. The proposed approach represents ship objects using dense representative points (RepPoints) to effectively capture local semantic information, thereby avoiding the background noise issues associated with traditional rectangular bounding box representations. To further enhance detection accuracy, an edge region sampling strategy is devised to uniformly sample RepPoints from critical ship parts, and a Weighted Residual Feature Pyramid Network (WRFPN) is introduced to efficiently fuse the multi-scale features through residual connections and learnable weights. In addition, a Weighted Chamfer Loss (WCL) combined with a staged localization loss strategy is employed to progressively refine localization from coarse to fine stages. Experimental results on both the HRSC2016 dataset and the newly constructed DOTA-SHIP dataset demonstrate that DenseRRSD achieves state-of-the-art detection accuracy, with mean Average Precision (mAP) scores of 91.2% and 83.2%, respectively, significantly outperforming existing methods. These results verify the effectiveness and robustness of the proposed approach in rotated-ship detection under diverse conditions.

Object detection plays a crucial role in scene understanding and has extensive practical applications. In the field of remote sensing object detection, both detection accuracy and robustness are of significant concern. Existing methods heavily rely on sophisticated adversarial training strategies that tend to improve robustness at the expense of accuracy. However, detection robustness is not always indicative of improved accuracy. Therefore, in this paper, we research how to enhance robustness, while still preserving high accuracy, or even improve both simultaneously, with simple vanilla adversarial training or even in the absence thereof. In pursuit of a solution, we first conduct an exploratory investigation by shifting our attention from adversarial training, referred to as adversarial fine-tuning, to adversarial pretraining. Specifically, we propose a novel pretraining paradigm, namely structured adversarial self-supervised (SASS) pretraining, to strengthen both clean accuracy and adversarial robustness for object detection in remote sensing images. At a high level, SASS pretraining aims to unify adversarial learning and self-supervised learning into pretraining and encode structured knowledge into pretrained representations for powerful transferability to downstream detection. Moreover, to fully explore the inherent robustness of vision Transformers and facilitate their pretraining efficiency, by leveraging the recent masked image modeling (MIM) as the pretext task, we further instantiate SASS pretraining into a concise end-to-end framework, named structured adversarial MIM (SA-MIM). SA-MIM consists of two pivotal components, structured adversarial attack and structured MIM (S-MIM). The former establishes structured adversaries for the context of adversarial pretraining, while the latter introduces a structured local-sampling global-masking strategy to adapt to hierarchical encoder architectures. Comprehensive experiments on three different datasets have demonstrated the significant superiority of the proposed pretraining paradigm over previous counterparts for remote sensing object detection. More importantly, regardless of with or without adversarial fine-tuning, it enables simultaneous improvements on detection accuracy and robustness as expected, promisingly alleviating the dependence on complicated adversarial fine-tuning.

Illegal waste dumping not only encroaches on land resources but also threatens the health of the surrounding residents. The traditional artificial waste monitoring solution requires professional workers to conduct field investigations. This solution not only requires high labor resources and economic costs but also demands a prolonged cycle for updating the monitoring status. Therefore, some scholars use deep learning to achieve automatic waste detection from satellite imagery. However, relevant models cannot effectively capture multi-scale features and enhance key information. To further bolster the monitoring efficiency of urban solid waste, we propose a novel multi-scale context fusion network for solid waste detection in remote sensing images, which can quickly collect waste distribution information in a large-scale range. Specifically, it introduces a new guidance fusion module that leverages spatial attention mechanisms alongside the use of large kernel convolutions. This module helps guide shallow features to retain useful details and adaptively adjust multi-scale spatial receptive fields. Meanwhile, it proposes a novel context awareness module based on heterogeneous convolutions and gating mechanisms. This module can effectively capture richer context information and provide anisotropic features for waste localization. In addition, it also designs an effective multi-scale interaction module based on cross-guidance and coordinate perception. This module not only enhances critical information but also fuses multi-scale semantic features. To substantiate the effectiveness of our approach, we conducted a series of comprehensive experiments on two representative urban waste detection datasets. The outcomes of relevant experiments indicate that our methodology surpasses other deep learning models. As plug-and-play components, these modules can be flexibly integrated into existing object detection frameworks, thereby delivering consistent enhancements in performance. Overall, we provide an efficient solution for monitoring illegal waste dumping, which contributes to promoting eco-friendly development.

With the continuous growth of maritime activities and the shipping trade, the application of maritime target detection in remote sensing images has become increasingly important. However, existing detection methods face numerous challenges, such as small target localization, recognition of targets with large aspect ratios, and high computational demands. In this paper, we propose an improved target detection model, named YOLOv5-ASC, to address the challenges in maritime target detection. The proposed YOLOv5-ASC integrates three core components: an Attention-based Receptive Field Enhancement Module (ARFEM), an optimized SIoU loss function, and a Deformable Convolution Module (C3DCN). These components work together to enhance the model’s performance in detecting complex maritime targets by improving its ability to capture multi-scale features, optimize the localization process, and adapt to the large aspect ratios typical of maritime objects. Experimental results show that, compared to the original YOLOv5 model, YOLOv5-ASC achieves a 4.36 percentage point increase in mAP@0.5 and a 9.87 percentage point improvement in precision, while maintaining computational complexity within a reasonable range. The proposed method not only achieves significant performance improvements on the ShipRSImageNet dataset but also demonstrates strong potential for application in complex maritime remote sensing scenarios.

With the wide applications of remote sensing technology in engineering, the demands of efficient object detection algorithms for remote sensing images have also been significantly increased in recent years. Traditional detection methods have the shortcomings of low accuracy and poor robustness, which is difficult to be applied to remote sensing images with complex background and varieties of objects. Recently, the deep convolutional neural networks have already shown great advances in object detection and outperformed many traditional methods. In this work, we study the performance of a region proposal-based method, Mask R-CNN, for detecting airplane and ship in remote sensing images. Specifically, we add an FPN module to improve the accuracy of small objects detection, and a mask branch is used to describe the shape of objects. In addition, we used a series of data augmentation strategies during training for meeting the CNN’s requirement of training samples diversity. The experimental results show that our model has superior performance in object detection of remote sensing images.

Aiming at the problem of low target detection accuracy caused by the close arrangement of targets, complex feature information, and numerous small targets in remote sensing images, this paper proposes a target detection algorithm based on the improved Yolov5 algorithm for remote sensing images. Firstly, this paper designs a remote sensing image target detection algorithm C²-YOLO based on multi-level feature fusion. Combined with the attention mechanism module to focus on the spatial location information of the target, which improves the extraction effect of the features of the target of interest. The network uses the CARAFE upsampling operator with a larger sensing field to better utilize the surrounding information. The proposal is to incorporate a predictive header hl that is generated from a low-level, high-resolution feature map to improve the recognition of tiny objects. The problem of large target scales in remote sensing images and poor detection of small targets has been improved. Angular loss is also added to the loss function to improve the detection of rotating targets. Finally, the accuracy comparison and experimental effect visualization are carried out on the dataset, and the experimental results show that the improvement work in this paper achieves better results no matter comparing the original YOLOv5 detection algorithm or other types of detection algorithms. Specifically, the accuracy of C²-YOLO on the NWPU dataset 1-shot recognition task is 78.45%, which is much higher than that of other detection algorithms, and is able to reach a superior performance in the remote sensing image target detection task.

Aggregating transformers and CNNs for salient object detection in optical remote sensing images

Oriented object detection in remote sensing images remains a challenging task due to arbitrary target rotations, extreme scale variations, and complex backgrounds. However, current rotated detectors still face several limitations: insufficient orientation-sensitive feature representation, feature misalignment for rotated proposals, and unstable optimization of rotation parameters. To address these issues, this paper proposes an enhanced Rotation-Sensitive Feature Pyramid Network (RSFPN) framework. Building upon the effective Oriented R-CNN paradigm, we introduce three novel core components: (1) a Dynamic Adaptive Feature Pyramid Network (DAFPN) that enables bidirectional multi-scale feature fusion through semantic-guided upsampling and structure-enhanced downsampling paths; (2) an Angle-Aware Collaborative Attention (AACA) module that incorporates orientation priors to guide feature refinement; (3) a Geometrically Consistent Multi-Task Loss (GC-MTL) that unifies the regression of rotation parameters with periodic smoothing and adaptive weight mechanisms. Comprehensive experiments on the DOTA-v1.0 and HRSC2016 benchmarks show that our RSFPN achieves superior performance. It attains a state-of-the-art mAP of 77.42% on DOTA-v1.0 and 91.85% on HRSC2016, while maintaining efficient inference at 14.5 FPS, demonstrating a favorable accuracy-efficiency trade-off. Visual analysis confirms that our method produces concentrated, rotation-aware feature responses and effectively suppresses background interference. The proposed approach provides a robust solution for detecting multi-oriented objects in high-resolution remote sensing imagery, with significant practical value for urban planning, environmental monitoring, and security applications.

Object detection in remote sensing images faces the challenges of a complex background, large object size variations, and high inter-class similarity. To address these problems, we propose an adaptive adjacent layer feature fusion (AALFF) method, which is developed on the basis of RTMDet. Specifically, the AALFF method incorporates an adjacent layer feature fusion enhancement (ALFFE) module, designed to capture high-level semantic information and accurately locate object spatial positions. ALFFE also effectively preserves small objects by fusing adjacent layer features and employs involution to aggregate contextual information in a wide spatial range for object essential features extraction in complex backgrounds. Additionally, the adaptive spatial feature fusion (ASFF) module is introduced to guide the network to select and fuse the crucial features to improve the adaptability to objects with different sizes. The proposed method achieves mean average precision (mAP) values of 77.1%, 88.9%, and 95.7% on the DIOR, HRRSD, and NWPU VHR-10 datasets, respectively. Notably, our approach achieves mAP75 values of 60.8% and 79.0% on the DIOR and HRRSD datasets, respectively, surpassing the state-of-the-art performance on the DIOR dataset.