FFO-DETR: Using Feature Filter to Improve the Performance of Oriented Detection Transformer

The development of technological devices, such as satellites and drones, has made it easier to collect images and videos from above. With rich and diverse data sources, the problem of detecting objects in aerial images is formed to serve real-life situations: rescue missions, social scenarios, security needs, and more. This issue attracts a lot of attention from researchers in computer vision because of its high applicability in real life. However, object detection in images collected from drones faces many significant challenges due to the complex factors of real-life environments. To overcome these difficulties, many studies focus on developing effective models for detecting and tracking objects from UAV systems. In this paper, our primary emphasis is comprehensive survey of methodologies pertaining to horizontal and oriented object detection. We analyzed and compared existing approaches such as CNN-based vs. Transformer-based, Anchor-based vs. Anchor-free. Moreover, we also learned about ways to represent objects using bounding boxes in oriented object detection. After researching many practical and innovative techniques in different approaches, we have carefully selected 15 methods for horizontal object detection and 18 for oriented object detection that were recently researched and developed to conduct classifications, surveys and assessments. We also provide a taxonomy mind map to assist readers in gaining an overview of the methods we surveyed. Additional, we examine 17 aerial image datasets collected from platforms and in various locations worldwide. These are typical datasets used to serve two primary tasks: Horizontal Object Detection and Oriented Object Detection. Then, we compared the experimental resultsof the previously introduced methods on four typical datasets: VisDrone, UAVDT, DOTAv1.0, and HRSC2016. The above datasets present diversity in the number of classes and the complexity of real-life conditions. Furthermore, they have been tested with various methods and shown remarkable results. For the reasons mentioned, we have enough basis to confirm that the comparison results are objective so that other research groups can make accurate comments and assessments on experimental methods on each dataset. We believe that updating and synthesizing the latest research will provide complete and comprehensive information, helping researchers to have more survey material for the development of the field of object detection in the future.

With the continuous progress of remote sensing image object detection tasks in recent years, researchers in this field have gradually shifted the focus of their research from horizontal object detection to the study of object detection in arbitrary directions. It is worth noting that some properties are different from horizontal object detection during oriented object detection that researchers have yet to notice much. This article presents the design of a straightforward and efficient arbitrary-oriented detection system, leveraging the inherent properties of the orientation task, including the rotation angle and box aspect ratio. In the detection of low aspect ratio objects, the angle is of little importance to the orientation bounding box, and it is even difficult to define the angle information in extreme categories. Conversely, in the detection of objects with high aspect ratios, the angle information plays a crucial role and can have a decisive impact on the quality of the detection results. By exploiting the aspect ratio of different targets, this letter proposes a ratio-balanced angle loss that allows the model to make a better trade-off between low-aspect ratio objects and high-aspect ratio objects. The rotation angle of each oriented object, which we naturally embed into a two-dimensional Euclidean space for regression, thus avoids an overly redundant design and preserving the topological properties of the circular space. The performance of the UCAS-AOD, HRSC2016, and DLR-3K datasets show that the proposed model in this paper achieves a leading level in terms of both accuracy and speed.

In aerial scenes, oriented object detection is sensitive to the orientation of objects, which makes the formulation of orientation-aware object representation become a critical problem. Existing methods mostly adopt rectangle anchor or discrete points as object representation, which may lead to the feature aliasing between overlapping objects and ignore the orientation information of objects. To solve these issues, we propose a novel anchor-free oriented object detection network named DRDet, which adopts Dual-angle Rotated Lines (DRL) as object representation. Different from other object representations, DRL can adaptively rotate and extend to the boundary of the object according to its orientation and shape, which explicitly introduces the orientation information into the formulation of object representation. And it can adaptively cope with the geometric deformation of objects. Based on the dual-angle rotated lines, we design an Orientation-guided Feature Encoder (OFE) to encode discriminant object feature along each rotated line, respectively. Instead of encoding rectangle feature, the OFE module adopts line features for orientation-guided feature encoding, which can alleviate the feature aliasing between neighboring objects or background. To further enhance the flexibility of dual-angle rotated lines, we design a Dual-angle Decoder (DD) that predicts two angle offsets according to the orientation-guided feature and converts the angle offsets and regression offsets into dual-angle rotated line representation, which can help to guide the adaptive rotation of each rotated line, respectively. Our proposed method achieves consistent improvement on both DOTA and HRSC2016 datasets. Extensive experimental results verify the effectiveness of our method in oriented object detection.

Object detection in aerial images having many practical applications in real life is becoming popular along with the surging development of deep learning and UAVs (Unmanned Aerial Vehicles). However, adverse weather conditions such as rain, night, and fog might reduce the quality of input images and significantly affect the performance of many perfectly trained detectors (detectors are trained in clear weather conditions). Moreover, object detection in aerial images often seeks high accuracy, while well-known object detection methods use horizontal bounding boxes to represent the object's location. These issues raise the inconsistency between classification and bounding box regression. Understanding the need for practical solutions, we create two experimental Fog datasets based on the original DOTA dataset to provide the in-depth analysis of fog density on multiple well-established oriented object detectors, namely Gliding Vertex, R3Det and ReDet. Furthermore, our training resources achieve promising results, flexibly ensemble to other methods to enhance models' performance and adapt to many adverse weather conditions.

With the advancement of object detection, oriented object detection is gradually replacing horizontal object detection as the preferred solution in remote sensing images. However, most existing datasets are annotated with horizontal bounding boxes, while re-annotating these datasets is both costly and inefficient. Recently, horizontal box supervised oriented object detection methods using merely horizontal box annotations have emerged as a solution. Nonetheless, these methods are relatively poor in predicting angles because of lacking angle supervision. Additionally, these methods overlook edge features, which can help capture and represent the oriented boundaries between objects and backgrounds, especially for objects with distinct edges or in blank backgrounds. To address these issues, we propose an edge feature self-attention and multi-task synthetic pattern enhanced horizontal box supervised oriented object detector named EAMSDet, aiming to achieve high precision based on horizontal box annotations. Specifically, we use horizontal and vertical Sobel operators to extract edge information and a self-attention module to fuse the edge information with the feature maps. Furthermore, to enrich the supervision of aspect ratio and angles, we add a multi-task synthetic branch and an extra regression loss to make full use of our introduced synthetic visual patterns. Based on H2RBox-v2, we conducted extensive experiments on remote sensing datasets such as DOTA-v1.0, DIOR, and HRSC2016, demonstrating the effectiveness and generalizability of our EAMSDet.

How to represent the object more appropriately in oriented object detection is an essential problem to be solved, there are many solutions for the object represented. It is a relatively novel approach to represent objects as a number of sample points useful for both localization and recognition. However, the current point-set-based representation methods do not effectively supervise all points for learning, and the internal information of the convex hull in the point set cannot be effectively learned. Therefore, this letter proposes point set distance (PSD) loss, which learns set-to-set supervision of objects to effectively represent objects. Besides, most of the current sample selection strategies are based on the Intersection over Union (IoU), but these methods cannot comprehensively measure candidate samples quality. To select high-quality point sets, we propose to use the probability distribution of point sets to select the positive samples. Our probabilistic point set sample selection (PPSS) scheme effectively exploits the classification information, regression information, and the distribution characteristics of the point set. Experimental results on remote sensing image datasets including DOTA, DIOR-R, and HRSC2016, demonstrate the proposed method for arbitrary-oriented object detection achieves consistent and substantial improvements.

Object detection for aerial images has achieved remarkable progress in recent years. Nevertheless, most exiting studies do not differentiate oriented object detection from horizontal detection. Certain schemes ignore the ambiguity of oriented object representation and leverage label assignment designed for horizontal object detection directly. Consequently, it leads to unstable training and causes performance degradation, because high-quality samples surrounding the oriented bounding boxes can not be leveraged effectively. To address this problem, we propose a gliding Free, orientation Free, and anchor Free Network (Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net) with high-efficiency for oriented object detection. Specifically, we propose an unambiguous oriented object representation scheme, named FreeGliding, by gliding the projection points of samples on each edge of horizontal bounding boxes. It makes the detection largely free from representation ambiguity and multi-task dependency. To overcome the restrictions of label assignment, we put forward a novel Loss-aware Outer Sample Selection (LOSS) scheme, which takes into consideration spatial information and localization capability to retain high-quality samples surrounding the objects. Moreover, we introduce an Oriented Feature Fusion (OFF) scheme to tackle feature alignment by adjusting the receptive field and fusing oriented features dynamically. Experimental results on two large-scale remote sensing datasets HRSC2016 and DOTA demonstrate that Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net outperforms the state-of-the-art schemes with a large margin. We hope our work can inspire rethinking the design of anchor-free detectors, and serve as a strong baseline for oriented object detection.

SAA-O2DINO: Oriented object detection transformer with improved denoising anchor boxes and shape-adaptive assigner

Oriented object detection is a challenging task in aerial images since the objects in aerial images are displayed in arbitrary directions and are frequently densely packed. The mainstream detectors describe rotating objects using a five-parameter or eight-parameter representations, which have representation ambiguity for orientated object definition and border loss discontinuity in the regression process. We proposed an innovative representation method based on area ratio of parallelogram, called area ratio of parallelogram (ARP). Specifically, ARP regresses the minimum bounding rectangle of the oriented object and three area ratios. Three area ratios include the area ratio of a directed object to the smallest circumscribed rectangle and two parallelograms to the minimum circumscribed rectangle. It simplifies offset learning and eliminates the issue of angular periodicity or label point sequences for oriented objects. To further remedy the confusion issue of nearly horizontal objects, the area ratio between the object and its minimal circumscribed rectangle is employed to guide the selection of horizontal or oriented detection for each object. Moreover, the rotated efficient intersection over union loss with horizontal bounding box and three area ratios are designed to optimize the bounding box regression for rotating objects. Experimental results on remote sensing datasets, including HRSC2016, DOTA, and UCAS-AOD, show that the proposed method achieves superior detection performance than many state-of-the-art approaches.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Light-weight neural networks for remote sensing (RS) visual analysis must overcome two inherent redundancies: spatial redundancy from vast, homogeneous backgrounds, and channel redundancy, where extreme scale variations render a single feature space inefficient. Existing models, often designed for natural images, fail to address this dual challenge in RS scenarios. To bridge this gap, we propose LWGANet, a light-weight backbone engineered for RS-specific properties. LWGANet introduces two core innovations: a Top-K Global Feature Interaction (TGFI) module that mitigates spatial redundancy by focusing computation on salient regions, and a Light-Weight Grouped Attention (LWGA) module that resolves channel redundancy by partitioning channels into specialized, scale-specific pathways. By synergistically resolving these core inefficiencies, LWGANet achieves a superior trade-off between feature representation quality and computational cost. Extensive experiments on twelve diverse datasets across four major RS tasks—scene classification, oriented object detection, semantic segmentation, and change detection—demonstrate that LWGANet consistently outperforms state-of-the-art light-weight backbones in both accuracy and efficiency. Our work establishes a new, robust baseline for efficient visual analysis in RS images.

Characterized by complicated backgrounds, various types, large size variations, and arbitrary orientations, the detection and recognition of arbitrary-oriented objects in remote sensing images are challenging. To address the aforementioned problem, an anchor-free arbitrary-oriented object detector using box boundary-aware vectors is proposed. With the idea of CenterNet to detect objects as points, oriented object detection is achieved by predicting the center, the box boundary-aware vectors, the size, and the type of the bounding box. In the feature extraction stage of the designed architecture, Res2Net, a multiscale convolutional neural network, is used to extract feature maps of different scales and adaptively spatial feature fusion is adopted to improve the detector&#x0027;s adaptability to objects of different sizes. In the detector, a context enhancement module with a multibranch network is designed to enhance the contextual information of the objects and improve the detector&#x0027;s robustness to the complicated backgrounds. Experiments are carried on three challenging benchmarks (i.e., HRSC2016, UCAS-AOD, and DOTA) and our method achieves state-of-the-art performance with 90.30&#x0025;, 89.70&#x0025;, and 77.18&#x0025; mAP, respectively.

Oriented object detection in remote sensing images remains a challenging task due to arbitrary target rotations, extreme scale variations, and complex backgrounds. However, current rotated detectors still face several limitations: insufficient orientation-sensitive feature representation, feature misalignment for rotated proposals, and unstable optimization of rotation parameters. To address these issues, this paper proposes an enhanced Rotation-Sensitive Feature Pyramid Network (RSFPN) framework. Building upon the effective Oriented R-CNN paradigm, we introduce three novel core components: (1) a Dynamic Adaptive Feature Pyramid Network (DAFPN) that enables bidirectional multi-scale feature fusion through semantic-guided upsampling and structure-enhanced downsampling paths; (2) an Angle-Aware Collaborative Attention (AACA) module that incorporates orientation priors to guide feature refinement; (3) a Geometrically Consistent Multi-Task Loss (GC-MTL) that unifies the regression of rotation parameters with periodic smoothing and adaptive weight mechanisms. Comprehensive experiments on the DOTA-v1.0 and HRSC2016 benchmarks show that our RSFPN achieves superior performance. It attains a state-of-the-art mAP of 77.42% on DOTA-v1.0 and 91.85% on HRSC2016, while maintaining efficient inference at 14.5 FPS, demonstrating a favorable accuracy-efficiency trade-off. Visual analysis confirms that our method produces concentrated, rotation-aware feature responses and effectively suppresses background interference. The proposed approach provides a robust solution for detecting multi-oriented objects in high-resolution remote sensing imagery, with significant practical value for urban planning, environmental monitoring, and security applications.

Multispectral oriented object detection faces challenges due to both inter-modal and intra-modal discrepancies. Recent studies often rely on transformer-based models to address these issues and achieve cross-modal fusion detection. However, the quadratic computational complexity of transformers limits their performance in remote sensing imagery. Inspired by the efficiency and lower complexity of Mamba in long sequence tasks, we propose Disparity-guided Multispectral Mamba (DMM), a multispectral oriented object detection framework comprised of a Disparity-guided Cross-modal Fusion Mamba (DCFM) module, a Multi-scale Target-aware Attention (MTA) module, and a Target-Prior Aware (TPA) auxiliary task. The DCFM module leverages disparity information between modalities to adaptively merge features from RGB and IR images, mitigating inter-modal conflicts. The MTA module aims to enhance feature representation by focusing on relevant target regions within the RGB modality, addressing intra-modal variations. The TPA auxiliary task utilizes single-modal labels to guide the optimization of the MTA module, ensuring it focuses on targets and their local context. Extensive experiments on the DroneVehicle and VEDAI datasets demonstrate the effectiveness of our method, which outperforms state-of-the-art methods while maintaining computational efficiency. Code will be available at https://github.com/Another-0/DMM.