Adaptive Adjacent Layer Feature Fusion for Object Detection in Remote Sensing Images

Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples

With the wide application of remote sensing images (RSIs) in military and civil fields, remote sensing object detection (RSOD) has gradually become a hot research direction. However, we observe two main challenges for remote sensing object detection, namely the complicated background and the small objects issues. Given the different appearances of generic objects and remote sensing objects, the detection algorithms designed for the former usually cannot perform well for the latter. We propose a novel global to local scale-aware detection network (GLSANet) for remote sensing object detection, aiming to solve the above mentioned two challenges. Firstly, we design a global semantic information interaction module (GSIIM) to excavate and reinforce the high-level semantic information in the deep feature map, which alleviates the obstacles of complex background on foreground objects. Secondly, we optimize the feature pyramid network to improve the performance of multiscale object detection in RSIs. Finally, a local attention pyramid (LAP) is introduced to highlight the feature representation of small objects gradually while suppressing the background and noise in the shallower feature maps. Extensive experiments on three public datasets demonstrate that the proposed method achieves superior performance compared with the state-of-the-art detectors, especially on small object detection dataset. Specifically, our algorithm reaches 94.57% mAP on NWPU VHR-10 dataset, 95.93% mAP on RSOD dataset and 77.9% mAP on DIOR dataset, respectively.

The presence of shadows in high-resolution (HR) remote-sensing images reduces object detection accuracy. To address this problem, in this paper, we proposed a deep neural network algorithm for shadow detection by using the AISD and SSAD remote-sensing shadow image datasets. To improve the ability to extract spatial information from feature maps, we developed a cross-spatial attention module that focuses on semantic information in the horizontal and vertical directions at each position point on the remote-sensing image. This module overcomes the limitations of existing technologies in accurately judging small areas and suspected shadow areas and in missing or incorrectly detected shadow areas. In addition, to improve the ability to extract shadow features and the accuracy of shadow detection in remote-sensing images, we developed a channel attention module that assigns more attention to channels that conform to the shadow color characteristics. The network architecture comprises an encoder – decoder structure, with ResNeXt50 used as the backbone for the encoder and a multi resolution parallel fusion (MRPF) designed for the decoder; cross-spatial and channel attention were incorporated into the decoder unit. Experimental results demonstrated the superior performance of the proposed algorithm, with an F1 score of 92.6% for the shadow category on the test set, thus, outperforming other algorithms and making the proposed method an effective solution for shadow detection in HR remote-sensing images.

Withcontinuous advancements in remote sensing technology, object detection in remote sensing images has emerged as a critical research direction in maritime surveillance, port management, and national defense. Among these applications, ship detection is a key task. Due to the fact that ships in images typically exhibit arbitrary rotations, multi-scale distributions, and complex backgrounds, conventional detection methods based on horizontal or rotated bounding boxes often fail to adequately capture the fine-grained information of the targets, thereby compromising detection accuracy. This paper proposes the Dense Representative Points-Guided Rotated-Ship Detection (DenseRRSD) method. The proposed approach represents ship objects using dense representative points (RepPoints) to effectively capture local semantic information, thereby avoiding the background noise issues associated with traditional rectangular bounding box representations. To further enhance detection accuracy, an edge region sampling strategy is devised to uniformly sample RepPoints from critical ship parts, and a Weighted Residual Feature Pyramid Network (WRFPN) is introduced to efficiently fuse the multi-scale features through residual connections and learnable weights. In addition, a Weighted Chamfer Loss (WCL) combined with a staged localization loss strategy is employed to progressively refine localization from coarse to fine stages. Experimental results on both the HRSC2016 dataset and the newly constructed DOTA-SHIP dataset demonstrate that DenseRRSD achieves state-of-the-art detection accuracy, with mean Average Precision (mAP) scores of 91.2% and 83.2%, respectively, significantly outperforming existing methods. These results verify the effectiveness and robustness of the proposed approach in rotated-ship detection under diverse conditions.

With the wide applications of remote sensing technology in engineering, the demands of efficient object detection algorithms for remote sensing images have also been significantly increased in recent years. Traditional detection methods have the shortcomings of low accuracy and poor robustness, which is difficult to be applied to remote sensing images with complex background and varieties of objects. Recently, the deep convolutional neural networks have already shown great advances in object detection and outperformed many traditional methods. In this work, we study the performance of a region proposal-based method, Mask R-CNN, for detecting airplane and ship in remote sensing images. Specifically, we add an FPN module to improve the accuracy of small objects detection, and a mask branch is used to describe the shape of objects. In addition, we used a series of data augmentation strategies during training for meeting the CNN’s requirement of training samples diversity. The experimental results show that our model has superior performance in object detection of remote sensing images.

Oriented object detection in remote sensing images remains a challenging task due to arbitrary target rotations, extreme scale variations, and complex backgrounds. However, current rotated detectors still face several limitations: insufficient orientation-sensitive feature representation, feature misalignment for rotated proposals, and unstable optimization of rotation parameters. To address these issues, this paper proposes an enhanced Rotation-Sensitive Feature Pyramid Network (RSFPN) framework. Building upon the effective Oriented R-CNN paradigm, we introduce three novel core components: (1) a Dynamic Adaptive Feature Pyramid Network (DAFPN) that enables bidirectional multi-scale feature fusion through semantic-guided upsampling and structure-enhanced downsampling paths; (2) an Angle-Aware Collaborative Attention (AACA) module that incorporates orientation priors to guide feature refinement; (3) a Geometrically Consistent Multi-Task Loss (GC-MTL) that unifies the regression of rotation parameters with periodic smoothing and adaptive weight mechanisms. Comprehensive experiments on the DOTA-v1.0 and HRSC2016 benchmarks show that our RSFPN achieves superior performance. It attains a state-of-the-art mAP of 77.42% on DOTA-v1.0 and 91.85% on HRSC2016, while maintaining efficient inference at 14.5 FPS, demonstrating a favorable accuracy-efficiency trade-off. Visual analysis confirms that our method produces concentrated, rotation-aware feature responses and effectively suppresses background interference. The proposed approach provides a robust solution for detecting multi-oriented objects in high-resolution remote sensing imagery, with significant practical value for urban planning, environmental monitoring, and security applications.

Object detection plays a crucial role in scene understanding and has extensive practical applications. In the field of remote sensing object detection, both detection accuracy and robustness are of significant concern. Existing methods heavily rely on sophisticated adversarial training strategies that tend to improve robustness at the expense of accuracy. However, detection robustness is not always indicative of improved accuracy. Therefore, in this paper, we research how to enhance robustness, while still preserving high accuracy, or even improve both simultaneously, with simple vanilla adversarial training or even in the absence thereof. In pursuit of a solution, we first conduct an exploratory investigation by shifting our attention from adversarial training, referred to as adversarial fine-tuning, to adversarial pretraining. Specifically, we propose a novel pretraining paradigm, namely structured adversarial self-supervised (SASS) pretraining, to strengthen both clean accuracy and adversarial robustness for object detection in remote sensing images. At a high level, SASS pretraining aims to unify adversarial learning and self-supervised learning into pretraining and encode structured knowledge into pretrained representations for powerful transferability to downstream detection. Moreover, to fully explore the inherent robustness of vision Transformers and facilitate their pretraining efficiency, by leveraging the recent masked image modeling (MIM) as the pretext task, we further instantiate SASS pretraining into a concise end-to-end framework, named structured adversarial MIM (SA-MIM). SA-MIM consists of two pivotal components, structured adversarial attack and structured MIM (S-MIM). The former establishes structured adversaries for the context of adversarial pretraining, while the latter introduces a structured local-sampling global-masking strategy to adapt to hierarchical encoder architectures. Comprehensive experiments on three different datasets have demonstrated the significant superiority of the proposed pretraining paradigm over previous counterparts for remote sensing object detection. More importantly, regardless of with or without adversarial fine-tuning, it enables simultaneous improvements on detection accuracy and robustness as expected, promisingly alleviating the dependence on complicated adversarial fine-tuning.

In general object detection, scale variation is always a big challenge. At present, feature pyramid networks are employed in numerous methods to alleviate the problems caused by large scale range of objects in object detection, which makes use of multi-level features extracted from the backbone for top-down upsampling and fusion to acquire a set of multi-scale depth image features. However, the feature pyramid network proposed by Ghiasi et al. adopts a simple fusion method, which fails to consider the fusion feature context, and therefore, it is difficult to acquire good features. In addition, the fusion of multi-scale features directly by traditional upsampling is prone to feature misalignment and loss of details. In this paper, an adaptive feature pyramid network is proposed based on the feature pyramid network to alleviate the foregoing potential problems, which includes two major designs, i.e., adaptive feature upsampling and adaptive feature fusion. The adaptive feature upsampling aims to predict a group of sampling points of each pixel through some models, and constitute feature representation of the pixel by feature combination of sampling points, while adaptive feature fusion is to construct pixel-level fusion weights between fusion features through attention mechanism. The experimental results verified the effectiveness of the method proposed in this paper. On the public object detection dataset MS-COCO test-dev, Faster R-CNN model achieved performance improvement of 1.2 AP by virtue of the adaptive feature pyramid network, and FCOS model could achieve performance improvement of 1.0 AP. What's more, the experiments also validated that the adaptive feature pyramid network proposed herein was more accurate for object localization.

Aiming at the problems of leakage, misdetection and low detection efficiency of transmission line foreign object detection in railroad environment, we propose an improved railroad transmission line foreign object detection algorithm MRS-YOLO based on YOLOv11n. Firstly, a multi-scale Adaptive Kernel Depth Feature Fusion (MAKDF) module is proposed and fused with the C3k2 module to form C3k2_MAKDF, which enhances the model's feature extraction capability for foreign objects of different sizes and shapes. Secondly, a novel Re-calibration Feature Fusion Pyramid Network(RCFPN) is designed as a Neck structure to improve the feature fusion capability of the model. Then, Spatial and Channel Reconstruction Detect Head (SC_Detect) based on spatial and channel preprocessing is designed to improve the overall detection accuracy of the model. Finally, the channel pruning technique is used to reduce the redundancy of the improved model, drastically reduce Parameters and GFLOPs, and improve the detection efficiency. The experimental results show that the mAP50 and mAP50:95 of the MRS-YOLO algorithm proposed in this paper are improved to 94.8% and 86.4%, respectively, which are 0.7 and 2.3 percentage points higher compared to the baseline, while Parameters and GFLOPs are reduced by 44.2% and 17.5%, respectively. It is demonstrated that the improved algorithm can be better applied to the task of foreign object detection in railroad transmission lines.

For remorte sensing image object detection tasks in the small object feature, extraction ability is insufficient and difficult to locate, and other problems. This paper proposes an improved algorithm for small object detection in remote sensing images based on a window self-attention mechanism. On the basis of You Only Look Once (YOLO)v5s, a shallow feature extraction layer with four times downsampling is added to the feature fusion pyramid and the window self-attention mechanism is added to the Path Aggregation Network. Experiments show that the improved model obtained the Mean Average Precision (mAP) of 78.3% and 91.8% on the DIOR and Remote Sensing Object Detection public data sets with frames per second of 65 and 51, respectively. Compared with the basal YOLOv5s network, the mAP has improved by 5.8% and 3.3%, respectively. Compared with other object detection methods, the detection accuracy and real-time performance have been improved.

Aircraft detection in remote sensing images is always the research hotspot but a challenging task for the variations of aircraft type, pose, size and complex background. The paper proposes a region-based convolutional neural network to detect aircrafts. To enhance the learning ability of the network, a multi-resolution aircraft remote sensing dataset is collected from Google Earth. Then, the detection model is trained end to end by fine-tuning on the obtained dataset and realizes automatic aircraft recognition and positioning. Experiments show that the proposed method outperforms state-of-the-art method on the same dataset and the requirement for real-time can be satisfied simultaneously.

In recent years, deep learning-based remote sensing object detection has achieved remarkable progress, yet the detection of tiny objects remains a significant challenge. Tiny objects in remote sensing images typically occupy only a few pixels, resulting in low contrast, poor resolution, and high sensitivity to localization errors. Their diverse scales and appearances, combined with complex backgrounds and severe class imbalance, further complicate the detection tasks. Conventional spatial feature extraction methods often struggle to capture the discriminative characteristics of tiny objects, especially in the presence of noise and occlusion. To address these challenges, we propose a frequency-aware attention-based tiny-object detection network with two plug-and-play modules that leverage frequency-domain information to enhance the targets. Specifically, we introduce a Multi-Scale Frequency Feature Enhancement Module (MSFFEM) to adaptively highlight the contour and texture details of tiny objects while suppressing background noise. Additionally, a Channel Attention-based RoI Enhancement Module (CAREM) is proposed to selectively emphasize high-frequency responses within RoI features, further improving object localization and classification. Furthermore, to mitigate sample imbalance, we employ multi-directional flip sample augmentation and redundancy filtering strategies, which significantly boost detection performance for few-shot categories. Extensive experiments on public object detection datasets, i.e., AI-TOD, VisDrone2019, and DOTA-v1.5, demonstrate that the proposed FANet consistently improves detection performance for tiny objects, outperforming existing methods and providing new insights into the integration of frequency-domain analysis and attention mechanisms for robust tiny-object detection in remote sensing applications.

ABSTRACTIn this article, we propose a novel difference image (DI) creation method for unsupervised change detection in multi-temporal multi-spectral remote-sensing images based on deep learning theory. First, we apply deep belief network to learn local and high-level features from the local neighbour of a given pixel in an unsupervised manner. Second, a back propagation algorithm is improved to build a DI based on selected training samples, which can highlight the difference on changed regions and suppress the false changes on unchanged regions. Finally, we get the change trajectory map using simple clustering analysis. The proposed scheme is tested on three remote-sensing data sets. Qualitative and quantitative evaluations show its superior performance compared to the traditional pixel-level and texture-level-based approaches.

The study of remote sensing image object detection has excellent research value in environmental protection and public safety. However, the performance of the detectors is unsatisfactory due to the large variability of object size and complex background noise in remote sensing images. Therefore, it is essential to improve the detection performance of the detectors. Inspired by the idea of "divide and conquer", we proposed a Multiple Receptive Field Attention(MRFA) to solve these problems and which is a plug-and-play attention method. First, we use the method of multiple receptive field feature map generation to convert the input feature map into four feature maps with different receptive fields. In this way, the small, medium, large, and immense objects in the input feature maps are "seen" in these feature maps, respectively. Then, we used the multiple attention map fusion method to focus objects of different sizes separately, which can effectively suppress noise in the background of remote sensing images. Experiments on remote sensing object detection datasets DIOR and HRRSD demonstrate that the performance of our method is better than other state-of-the-art attention modules. At the same time, the experiments on remote sensing image semantic segmentation dataset WHDLD and classification dataset AID prove the generalization and superiority of our method.

Illegal waste dumping not only encroaches on land resources but also threatens the health of the surrounding residents. The traditional artificial waste monitoring solution requires professional workers to conduct field investigations. This solution not only requires high labor resources and economic costs but also demands a prolonged cycle for updating the monitoring status. Therefore, some scholars use deep learning to achieve automatic waste detection from satellite imagery. However, relevant models cannot effectively capture multi-scale features and enhance key information. To further bolster the monitoring efficiency of urban solid waste, we propose a novel multi-scale context fusion network for solid waste detection in remote sensing images, which can quickly collect waste distribution information in a large-scale range. Specifically, it introduces a new guidance fusion module that leverages spatial attention mechanisms alongside the use of large kernel convolutions. This module helps guide shallow features to retain useful details and adaptively adjust multi-scale spatial receptive fields. Meanwhile, it proposes a novel context awareness module based on heterogeneous convolutions and gating mechanisms. This module can effectively capture richer context information and provide anisotropic features for waste localization. In addition, it also designs an effective multi-scale interaction module based on cross-guidance and coordinate perception. This module not only enhances critical information but also fuses multi-scale semantic features. To substantiate the effectiveness of our approach, we conducted a series of comprehensive experiments on two representative urban waste detection datasets. The outcomes of relevant experiments indicate that our methodology surpasses other deep learning models. As plug-and-play components, these modules can be flexibly integrated into existing object detection frameworks, thereby delivering consistent enhancements in performance. Overall, we provide an efficient solution for monitoring illegal waste dumping, which contributes to promoting eco-friendly development.