A multiscale object detection approach for remote sensing images based on MSE-DenseNet and the dynamic anchor assignment

With the wide application of remote sensing images (RSIs) in military and civil fields, remote sensing object detection (RSOD) has gradually become a hot research direction. However, we observe two main challenges for remote sensing object detection, namely the complicated background and the small objects issues. Given the different appearances of generic objects and remote sensing objects, the detection algorithms designed for the former usually cannot perform well for the latter. We propose a novel global to local scale-aware detection network (GLSANet) for remote sensing object detection, aiming to solve the above mentioned two challenges. Firstly, we design a global semantic information interaction module (GSIIM) to excavate and reinforce the high-level semantic information in the deep feature map, which alleviates the obstacles of complex background on foreground objects. Secondly, we optimize the feature pyramid network to improve the performance of multiscale object detection in RSIs. Finally, a local attention pyramid (LAP) is introduced to highlight the feature representation of small objects gradually while suppressing the background and noise in the shallower feature maps. Extensive experiments on three public datasets demonstrate that the proposed method achieves superior performance compared with the state-of-the-art detectors, especially on small object detection dataset. Specifically, our algorithm reaches 94.57% mAP on NWPU VHR-10 dataset, 95.93% mAP on RSOD dataset and 77.9% mAP on DIOR dataset, respectively.

How to represent the object more appropriately in oriented object detection is an essential problem to be solved, there are many solutions for the object represented. It is a relatively novel approach to represent objects as a number of sample points useful for both localization and recognition. However, the current point-set-based representation methods do not effectively supervise all points for learning, and the internal information of the convex hull in the point set cannot be effectively learned. Therefore, this letter proposes point set distance (PSD) loss, which learns set-to-set supervision of objects to effectively represent objects. Besides, most of the current sample selection strategies are based on the Intersection over Union (IoU), but these methods cannot comprehensively measure candidate samples quality. To select high-quality point sets, we propose to use the probability distribution of point sets to select the positive samples. Our probabilistic point set sample selection (PPSS) scheme effectively exploits the classification information, regression information, and the distribution characteristics of the point set. Experimental results on remote sensing image datasets including DOTA, DIOR-R, and HRSC2016, demonstrate the proposed method for arbitrary-oriented object detection achieves consistent and substantial improvements.

Recently, object detection has gained significant progress in remote sensing images. Nevertheless, we conclude two defects in remote sensing image object detection. At first, most methods rely on feature pyramid, but the features of different levels would influence each other when we use top-down operation. Second, the traditional label assignment strategy cannot assign suitable labels, as it adopts the fixed intersection over union (IoU) threshold to divide positive samples and negative samples during training. According to the problems we pointed out, a simple yet effective framework is employed to eliminate these two limitations. It integrates two novel components: aware feature pyramid network (AFPN) and group assignment strategy (GAS). AFPN is to mitigate the adverse effects caused by the first problem. Specifically, it learns a vector for the higher level features in the feature pyramid to obtain clean features. As for the second limitation, we recommend a new label assignment strategy named GAS to address this problem. Samples will be grouped according to their overlaps with ground truth, and then, they are assigned to positive or negative labels in each group. Extensive experiments are conducted on the large-scale object detection dataset DIOR and DOTA. With the newly introduced two key components, our model significantly improves the detection accuracy. Without bells and whistles, our proposed method achieves 2.0% and 1.9% higher mean average precision (mAP) than Faster R-CNN with FPN when using ResNet50 and ResNet101 as the backbones, respectively. Finally, we obtain 73.3% mAP on the DIOR dataset without any tricks. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hm-better/dior_detect</uri> .

The study of remote sensing image object detection has excellent research value in environmental protection and public safety. However, the performance of the detectors is unsatisfactory due to the large variability of object size and complex background noise in remote sensing images. Therefore, it is essential to improve the detection performance of the detectors. Inspired by the idea of "divide and conquer", we proposed a Multiple Receptive Field Attention(MRFA) to solve these problems and which is a plug-and-play attention method. First, we use the method of multiple receptive field feature map generation to convert the input feature map into four feature maps with different receptive fields. In this way, the small, medium, large, and immense objects in the input feature maps are "seen" in these feature maps, respectively. Then, we used the multiple attention map fusion method to focus objects of different sizes separately, which can effectively suppress noise in the background of remote sensing images. Experiments on remote sensing object detection datasets DIOR and HRRSD demonstrate that the performance of our method is better than other state-of-the-art attention modules. At the same time, the experiments on remote sensing image semantic segmentation dataset WHDLD and classification dataset AID prove the generalization and superiority of our method.

A weakly supervised object detection (WSOD) task is to train a detector with only image-level labels provided. Except for the training difficulty introduced by weaker annotations, the inherent complexity of the remote-sensing images (RSIs) also adds to the challenge. To boost the detector’s localization accuracy, we aim to exploit more semantic information contained in images and help improve the general robustness of the model. Noticing previous methods tend to focus on the most discriminative part of an object, we design a self-training-based network that leverages local semantic features. To this end, we develop a semantic-balanced localization module (SBLM) that distinguishes foreground from background and accurate proposals from incomplete ones, by leveraging a balance of region of interest (ROI) and its context information. Moreover, we find that the self-training strategy highly relies on the quality of pseudo-ground-truth boxes. Motivated by this possible lack of robustness, we design a comprehensive clustering module (CCM) and saliency-based proposal filtering (SPF) module that select pseudo-ground truth more comprehensively under supervision. To be more specific, CCM aims to reduce the arbitrariness during assigning pseudo-labels by considering multiple categorical vectors simultaneously. Salient object detection (SOD) is applied in the SPF module to help evaluate the quality of the chosen pseudo-ground-truth boxes. The detection performance is significantly boosted with the proposed method. Extensive experiments conducted on the NWPU VHR-10.v2 dataset and the DIOR dataset validate that the proposed model outperforms the previous state-of-the-art methods favorably with an mAP of 64.9% and 28.1%, respectively.

The presence of shadows in high-resolution (HR) remote-sensing images reduces object detection accuracy. To address this problem, in this paper, we proposed a deep neural network algorithm for shadow detection by using the AISD and SSAD remote-sensing shadow image datasets. To improve the ability to extract spatial information from feature maps, we developed a cross-spatial attention module that focuses on semantic information in the horizontal and vertical directions at each position point on the remote-sensing image. This module overcomes the limitations of existing technologies in accurately judging small areas and suspected shadow areas and in missing or incorrectly detected shadow areas. In addition, to improve the ability to extract shadow features and the accuracy of shadow detection in remote-sensing images, we developed a channel attention module that assigns more attention to channels that conform to the shadow color characteristics. The network architecture comprises an encoder – decoder structure, with ResNeXt50 used as the backbone for the encoder and a multi resolution parallel fusion (MRPF) designed for the decoder; cross-spatial and channel attention were incorporated into the decoder unit. Experimental results demonstrated the superior performance of the proposed algorithm, with an F1 score of 92.6% for the shadow category on the test set, thus, outperforming other algorithms and making the proposed method an effective solution for shadow detection in HR remote-sensing images.

Aggregating transformers and CNNs for salient object detection in optical remote sensing images

Object detection has a great significance to remote sensing image recognition. Conventional object detection methods by using horizontal bounding box have shown good performance in general images. However, because of looking down perspective of remote sensing images, rotation bounding box is more suitable and precise for object detection. Although rotation bounding box has been researched by some works for remote sensing images, most of them are still two stages. In this paper, by improvement of YOLO architecture, we propose an one-stage object detection method to predict rotation bounding box for remote sensing images. Firstly, in the data preprocessing stage, an image cropping operation is utilized instead of zooming operation to generate suitable input data for network. Meanwhile, in order to avoid an object from being divided, an overlapping band is set during cropping. Secondly, a novel rotation bounding box representation method is introduced, and a corresponding loss function is designed in training process. Experiments on the Dota dataset demonstrate that our method outperforms state-of-the-art rotation objects detection methods, in terms of mAP, our method achieves 73.50%mAP, which is higher than methods based on rotation bounding box by 2.0%.

For remorte sensing image object detection tasks in the small object feature, extraction ability is insufficient and difficult to locate, and other problems. This paper proposes an improved algorithm for small object detection in remote sensing images based on a window self-attention mechanism. On the basis of You Only Look Once (YOLO)v5s, a shallow feature extraction layer with four times downsampling is added to the feature fusion pyramid and the window self-attention mechanism is added to the Path Aggregation Network. Experiments show that the improved model obtained the Mean Average Precision (mAP) of 78.3% and 91.8% on the DIOR and Remote Sensing Object Detection public data sets with frames per second of 65 and 51, respectively. Compared with the basal YOLOv5s network, the mAP has improved by 5.8% and 3.3%, respectively. Compared with other object detection methods, the detection accuracy and real-time performance have been improved.

Unmanned aerial vehicle (UAV) remote-sensing images present unique challenges to the object-detection task due to uneven object densities, low resolution, and drastic scale variations. Downsampling is an important component of deep networks that expands the receptive field, reduces computational overhead, and aggregates features. However, object detectors using multi-layer downsampling result in varying degrees of texture feature loss for various scales in remote-sensing images, degrading the performance of multi-scale object detection. To alleviate this problem, we propose a lightweight texture reconstructive downsampling module called TRD. TRD models part of the texture features lost as residual information during downsampling. After modeling, cascading downsampling and upsampling operators provide residual feedback to guide the reconstruction of the desired feature map for each downsampling stage. TRD structurally optimizes the feature-extraction capability of downsampling to provide sufficiently discriminative features for subsequent vision tasks. We replace the downsampling module of the existing backbone network with the TRD module and conduct a large number of experiments and ablation studies on a variety of remote-sensing image datasets. Specifically, the proposed TRD module improves 3.1% AP over the baseline on the NWPU VHR-10 dataset. On the VisDrone-DET dataset, the TRD improves 3.2% AP over the baseline with little additional cost, especially the APS, APM, and APL by 3.1%, 8.8%, and 13.9%, respectively. The results show that TRD enriches the feature information after downsampling and effectively improves the multi-scale object-detection accuracy of UAV remote-sensing images.

High-precision remote sensing image object detection has broad application prospects in military defense, disaster emergency, urban planning, and other fields. However, the arbitrary orientation, dense arrangement, and small size of objects in remote sensing images lead to poor detection accuracy of existing methods. To achieve accurate detection, this paper proposes an arbitrary directional remote sensing object detection method, called FANet, based on feature fusion and angle classification. Initially, the angle prediction branch is introduced, and the circular smooth label method is used to transform the angle regression problem into a classification problem, which solves the difficult problem of abrupt changes in the boundaries of the rotating frame while realizing the object frame rotation. Subsequently, to extract robust remote sensing objects, innovative introduce pure convolutional model as a backbone network, while Conv is replaced by GSConv to reduce the number of parameters in the model along with ensuring detection accuracy. Finally, the strengthen connection feature pyramid network (SC-FPN) is proposed to redesign the lateral connection part for deep and shallow layer feature fusion, and add jump connections between the input and output of the same level feature map to enrich the feature semantic information. In addition, add a variable parameter to the original localization loss function to satisfy the bounding box regression accuracy under different IoU thresholds, and thus obtain more accurate object detection. The comprehensive experimental results on two public datasets for rotated object detection DOTA and HRSC2016 demonstrate the effectiveness of our method.

Oriented object detection in remote sensing images remains a challenging task due to arbitrary target rotations, extreme scale variations, and complex backgrounds. However, current rotated detectors still face several limitations: insufficient orientation-sensitive feature representation, feature misalignment for rotated proposals, and unstable optimization of rotation parameters. To address these issues, this paper proposes an enhanced Rotation-Sensitive Feature Pyramid Network (RSFPN) framework. Building upon the effective Oriented R-CNN paradigm, we introduce three novel core components: (1) a Dynamic Adaptive Feature Pyramid Network (DAFPN) that enables bidirectional multi-scale feature fusion through semantic-guided upsampling and structure-enhanced downsampling paths; (2) an Angle-Aware Collaborative Attention (AACA) module that incorporates orientation priors to guide feature refinement; (3) a Geometrically Consistent Multi-Task Loss (GC-MTL) that unifies the regression of rotation parameters with periodic smoothing and adaptive weight mechanisms. Comprehensive experiments on the DOTA-v1.0 and HRSC2016 benchmarks show that our RSFPN achieves superior performance. It attains a state-of-the-art mAP of 77.42% on DOTA-v1.0 and 91.85% on HRSC2016, while maintaining efficient inference at 14.5 FPS, demonstrating a favorable accuracy-efficiency trade-off. Visual analysis confirms that our method produces concentrated, rotation-aware feature responses and effectively suppresses background interference. The proposed approach provides a robust solution for detecting multi-oriented objects in high-resolution remote sensing imagery, with significant practical value for urban planning, environmental monitoring, and security applications.

Object detection plays a crucial role in scene understanding and has extensive practical applications. In the field of remote sensing object detection, both detection accuracy and robustness are of significant concern. Existing methods heavily rely on sophisticated adversarial training strategies that tend to improve robustness at the expense of accuracy. However, detection robustness is not always indicative of improved accuracy. Therefore, in this paper, we research how to enhance robustness, while still preserving high accuracy, or even improve both simultaneously, with simple vanilla adversarial training or even in the absence thereof. In pursuit of a solution, we first conduct an exploratory investigation by shifting our attention from adversarial training, referred to as adversarial fine-tuning, to adversarial pretraining. Specifically, we propose a novel pretraining paradigm, namely structured adversarial self-supervised (SASS) pretraining, to strengthen both clean accuracy and adversarial robustness for object detection in remote sensing images. At a high level, SASS pretraining aims to unify adversarial learning and self-supervised learning into pretraining and encode structured knowledge into pretrained representations for powerful transferability to downstream detection. Moreover, to fully explore the inherent robustness of vision Transformers and facilitate their pretraining efficiency, by leveraging the recent masked image modeling (MIM) as the pretext task, we further instantiate SASS pretraining into a concise end-to-end framework, named structured adversarial MIM (SA-MIM). SA-MIM consists of two pivotal components, structured adversarial attack and structured MIM (S-MIM). The former establishes structured adversaries for the context of adversarial pretraining, while the latter introduces a structured local-sampling global-masking strategy to adapt to hierarchical encoder architectures. Comprehensive experiments on three different datasets have demonstrated the significant superiority of the proposed pretraining paradigm over previous counterparts for remote sensing object detection. More importantly, regardless of with or without adversarial fine-tuning, it enables simultaneous improvements on detection accuracy and robustness as expected, promisingly alleviating the dependence on complicated adversarial fine-tuning.

Withcontinuous advancements in remote sensing technology, object detection in remote sensing images has emerged as a critical research direction in maritime surveillance, port management, and national defense. Among these applications, ship detection is a key task. Due to the fact that ships in images typically exhibit arbitrary rotations, multi-scale distributions, and complex backgrounds, conventional detection methods based on horizontal or rotated bounding boxes often fail to adequately capture the fine-grained information of the targets, thereby compromising detection accuracy. This paper proposes the Dense Representative Points-Guided Rotated-Ship Detection (DenseRRSD) method. The proposed approach represents ship objects using dense representative points (RepPoints) to effectively capture local semantic information, thereby avoiding the background noise issues associated with traditional rectangular bounding box representations. To further enhance detection accuracy, an edge region sampling strategy is devised to uniformly sample RepPoints from critical ship parts, and a Weighted Residual Feature Pyramid Network (WRFPN) is introduced to efficiently fuse the multi-scale features through residual connections and learnable weights. In addition, a Weighted Chamfer Loss (WCL) combined with a staged localization loss strategy is employed to progressively refine localization from coarse to fine stages. Experimental results on both the HRSC2016 dataset and the newly constructed DOTA-SHIP dataset demonstrate that DenseRRSD achieves state-of-the-art detection accuracy, with mean Average Precision (mAP) scores of 91.2% and 83.2%, respectively, significantly outperforming existing methods. These results verify the effectiveness and robustness of the proposed approach in rotated-ship detection under diverse conditions.

Object detection is an important part of remote sensing image analysis. With the development of the earth observation technology and convolutional neural network, remote sensing image object detection technology based on deep learning has received more and more attention and research. At present, many excellent object detection algorithms have been proposed and applied in the field of remote sensing. In this paper, the object detection algorithms of remote sensing image is systematically summarized, the main contents include the traditional remote sensing image object detection method and the method based on deep learning, emphasis on summarize the remote sensing image object detection algorithm based on deep learning and its development course, then we introduced the rule of performance evaluation of object detection and datasets that commonly used. Finally, the future development trend is analyzed and prospected. It is hoped that this summary and analysis can provide some reference for future research on object detection technology in remote sensing field.