Toward Oriented Fisheye Object Detection: Dataset and Baseline

In recent years, data-driven deep learning has yielded fruitful results in synthetic aperture radar (SAR) ship detection; weakly supervised learning methods based on horizontal bounding boxes (HBBs) train oriented bounding box (OBB) detectors using HBB labels, effectively addressing scarce OBB annotation data and advancing SAR ship OBB detection. However, current methods for oriented SAR ship detection still suffer from issues such as insufficient quantity and quality of pseudo-labels, low inference efficiency, large model parameters, and limited global information capture, making it difficult to balance detection performance and efficiency. To tackle these, we propose the weakly supervised oriented SAR ship detection algorithm based on optimized pseudo-label generation and guidance. The method introduces pseudo-labels into a single-stage detector via a two-stage training process: the first stage coarsely learns target angles and scales using horizontal bounding box weak supervision and angle self-supervision, while the second stage refines angle and scale learning guided by pseudo-labels, improving performance and reducing missed detections. To generate high-quality pseudo-labels in large quantities, we propose three optimization strategies: Adaptive Kernel Growth Pseudo-Label Generation Strategy (AKG-PLGS), Pseudo-Label Selection Strategy based on PCA angle estimation and horizontal bounding box constraints (PCA-HBB-PLSS), and Long-Edge Scanning Refinement Strategy (LES-RS). Additionally, we designed a backbone and neck network incorporating window attention and adaptive feature fusion, effectively enhancing global information capture and multiscale feature integration while reducing model parameters. Experiments on SSDD and HRSID show that our algorithm achieves an mAP50 of 85.389% and 82.508%, respectively, with significantly reduced model parameters and computational consumption.

Deep learning have been playing an increasing important role in ship detection in synthetic aperture radar (SAR) image because of its high accuracy, fast speed and no need of manual design features. It is difficult to set appropriate anchors when using anchor-based method because of multi-resolution SAR images and multi-scale ships, thus resulting in degradation of detection performance in operational applications. Recently, a lot of anchor-free architecture have been put forward, among which most of them apply horizontal bounding boxes (HBBs) to encode ship location, leading to inaccurate location description and decreased detection performance especially for arbitrary-oriented ships in complex scenes. Therefore, an anchor-free detector for arbitrary-oriented ship detection in SAR images, called R-FCOS is proposed in this paper. Specifically, based on the FCOS baseline method, ship aspect angle and rotate regression branch are introduced to regress oriented bounding boxes (OBBs) for arbitrary-oriented ships detection. Besides, to alleviate the effects of feature misalignment problem, ratio-ness branch is proposed to predict the aspect ratio of ship, which is used to filter and refine the OBBs in the post-processing step to improve detection accuracy. Finally, we carry out experiments on the SSDD+ data set to verify the effectiveness of our method. It turns out that our method obtains mAP 90.65% under an IoU threshold of 0.5.

Airport runways, as the core part of airports, belong to vital national infrastructure, and the target detection and segmentation of airport runways in remote sensing images using deep learning methods have significant research value. Most of the existing airport target detection methods based on deep learning rely on horizontal bounding boxes for localization, which often contain irrelevant background information. Moreover, when detecting multiple intersecting airport runways in a single remote sensing image, issues such as false positives and false negatives are apt to occur. To address these challenges, this study proposes an end-to-end remote sensing image airport runway detection and segmentation method based on an improved Mask RCNN (CR-Mask RCNN). The proposed method uses a rotated region generation network instead of a non-rotated region generation network, allowing it to generate rotated bounding boxes that fit the shape of the airport runway more closely, thus avoiding the interference of a large amount of invalid background information brought about by horizontal bounding boxes. Furthermore, the method incorporates an attention mechanism into the backbone feature extraction network to allocate attention to different airport runway feature map scales, which enhances the extraction of local feature information, captures detailed information more effectively, and reduces issues of false positives and false negatives when detecting airport runway targets. The results indicate that, when comparing horizontal bounding boxes with rotated bounding boxes for detecting and segmenting airport runways, the latter are more precise for complex backgrounds. Furthermore, incorporating an attention mechanism enhances the accuracy of airport runway recognition, making it highly effective and practical.

It is challenging to accurately detect flexible objects with arbitrary orientation from monitoring images in power grid maintenance and inspection sites. This is because these images exhibit a significant imbalance between the foreground and background, which can lead to low detection accuracy when using a horizontal bounding box (HBB) as the detector in general object detection algorithms. Existing multi-oriented detection algorithms that use irregular polygons as the detector can improve accuracy to some extent, but their accuracy is limited due to boundary problems during the training process. This paper proposes a rotation-adaptive YOLOv5 (R_YOLOv5) with a rotated bounding box (RBB) to detect flexible objects with arbitrary orientation, effectively addressing the above issues and achieving high accuracy. Firstly, a long-side representation method is used to add the degree of freedom (DOF) for bounding boxes, enabling accurate detection of flexible objects with large spans, deformable shapes, and small foreground-to-background ratios. Furthermore, the further boundary problem induced by the proposed bounding box strategy is overcome by using classification discretization and symmetric function mapping methods. Finally, the loss function is optimized to ensure training convergence for the new bounding box. To meet various practical requirements, we propose four models with different scales based on YOLOv5, namely R_YOLOv5s, R_YOLOv5m, R_YOLOv5l, and R_YOLOv5x. Experimental results demonstrate that these four models achieve mean average precision (mAP) values of 0.712, 0.731, 0.736, and 0.745 on the DOTA-v1.5 dataset and 0.579, 0.629, 0.689, and 0.713 on our self-built FO dataset, exhibiting higher recognition accuracy and a stronger generalization ability. Among them, R_YOLOv5x achieves a mAP that is about 6.84% higher than ReDet on the DOTAv-1.5 dataset and at least 2% higher than the original YOLOv5 model on the FO dataset.

Automatic detection of weapons is significant for improving security and well being of individuals, nonetheless, it is a difficult task due to large variety of size, shape and appearance of weapons. View point variations and occlusion also are reasons which makes this task more difficult. Further, the current object detection algorithms process rectangular areas, however a slender and long rifle may really cover just a little portion of area and the rest may contain unessential details. To overcome these problem, we propose a CNN architecture for Orientation Aware Weapons Detection, which provides oriented bounding box with improved weapons detection performance. The proposed model provides orientation not only using angle as classification problem by dividing angle into eight classes but also angle as regression problem. For training our model for weapon detection a new dataset comprising of total 7801 weapons images is gathered from the web and then manually annotated with position oriented bounding boxes. Our dataset provides not only oriented bounding box as ground truth but also horizontal bounding box. We also provide our dataset in multiple formats of modern object detectors for further research in this area. The proposed model is evaluated on this dataset, and the comparative analysis with off-the shelf object detectors yields superior performance of proposed model, measured with standard evaluation strategies. The dataset and the model implementation are made publicly available at this link: https://bit.ly/2TyZICF .

In recent years, in the field of remote sensing image target detection, oriented target detection has received extensive attention. Traditional detectors use horizontal bounding box (HBB) to locate targets, which may cause inaccurate detection when detecting targets with arbitrary orientation angles, dense distribution and large aspect ratios. The Oriented Bounding Box (OBB) adds different rotation angles to the horizontal bounding box, which can better deal with the above problems. However, with the introduction of oriented bounding boxes in the detector, new problems have emerged, such as the increase in the number of anchor points with different scales, angles, and aspect ratios, and the imbalance of IoU matching of positive and negative anchor boxes, which leads to classification confidence and Inconsistency between positioning accuracy. In order to overcome these shortcomings while using the oriented bounding box, this paper proposes a novel oriented target detector and redesigns a matching strategy for evaluating the position of the anchor frame. By selecting a higher quality anchor frame to achieve more accurate target detection, thereby reducing the instability of the angle in the matching process. Experiments conducted on public remote sensing data sets such as HRSC2016 and UCAS-AOD show that the proposed method achieves advanced detection accuracy.

Weakly supervised instance segmentation using multi-prior fusion

In the field of synthetic aperture radar (SAR) vehicle detection, detectors based on deep learning have gained extensive application in recent years. However, most methods can only generate horizontal bounding boxes (HBBs). Due to the lack of angle information, the presence of excessive background noise in HBB leads to an inaccurate description of vehicles. Although research has been conducted on oriented SAR target detection, several issues persist, including the inadequate utilization of vehicle characteristics, inaccurate feature extraction for vehicles exhibiting size and rotation variations, and imprecise positioning of oriented bounding boxes (OBBs). To address these issues, we propose a novel-oriented SAR vehicle detection network. First, leveraging the shape property of vehicles, we devise a constrained rectangular-invariant rotatable convolution (Cons-RIRConv) for the backbone. Under the constraints of a carefully designed size- and rotation-invariant regularization term, Cons-RIRConv determines convolution sampling positions that are adaptive to the size and rotation variations of vehicles. This approach is conducive to Cons-RIRConv in extracting size- and rotation-invariant features of vehicles. Furthermore, as the sampling positions of Cons-RIRConv align closely with vehicles, we integrate them into the anchor generation process within the detection head, thereby developing an adaptive anchor generation mechanism (AAGM). Guided by the sampling positions of Cons-RIRConv, the anchor boxes generated by AAGM exhibit a high degree of conformity with the vehicles and low redundancy, ultimately improving the localization accuracy of OBB and reducing computation costs. Experiments on three authoritative measured SAR vehicle detection datasets show the effectiveness of our method.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

Ship detection needs to identify ship locations from remote sensing scenes. Due to different imaging payloads, various appearances of ships, and complicated background interference from the bird's eye view, it is difficult to setup a unified paradigm for achieving multisource ship detection. To address this challenge, in this article, leveraging the large language models powerful generalization ability, a unified visual-language model called Popeye is proposed for multisource ship detection from RS imagery. Specifically, to bridge the interpretation gap across the multisource images for ship detection, a novel unified labeling paradigm is designed to integrate different visual modalities and the various ship detection ways, i.e., horizontal bounding box and oriented bounding box. Subsequently, the hybrid experts encoder is designed to refine multiscale visual features, thereby enhancing visual perception. Then, a visual-language alignment method is developed for Popeye to enhance interactive comprehension ability between visual and language content. Furthermore, an instruction adaption mechanism is proposed for transferring the pretrained visual-language knowledge from the nature scene into the RS domain for multisource ship detection. In addition, the segment anything model is also seamlessly integrated into the proposed Popeye to achieve pixel-level ship segmentation without additional training costs. Finally, extensive experiments are conducted on the newly constructed ship instruction dataset named MMShip, and the results indicate that the proposed Popeye outperforms current specialist, open-vocabulary, and other visual-language models in zero-shot multisource various ship detection tasks.

Vehicle detection in aerial images is an important and challenging task in the field of remote sensing. Recently, deep learning technologies have yielded superior performance for object detection in remote sensing images. However, the detection results of the existing methods are horizontal bounding boxes that ignore vehicle orientations, thereby having limited applicability in scenes with dense vehicles or clutter backgrounds. In this article, we propose a one-stage, anchor-free detection approach to detect arbitrarily oriented vehicles in high-resolution aerial images. The vehicle detection task is transformed into a multitask learning problem by directly predicting high-level vehicle features via a fully convolutional network. That is, a classification subtask is created to look for vehicle central points and three regression subtasks are created to predict vehicle orientations, scales, and offsets of vehicle central points. First, coarse and fine feature maps outputted from different stages of a residual network are concatenated together by a feature pyramid fusion strategy. Upon the concatenated features, four convolutional layers are attached in parallel to predict high-level vehicle features. During training, task uncertainty learned from the training data is used to weight loss function in the multitask learning setting. For inferencing, oriented bounding boxes are generated using the predicted vehicle features, and oriented nonmaximum suppression (NMS) postprocessing is used to reduce redundant results. Experiments on two public aerial image data sets have shown the effectiveness of the proposed approach.

In recent years, synthetic aperture radar (SAR) vehicle detection has become a research hotspot. However, algorithms using horizontal bounding boxes can lead to redundant detection areas due to the varying aspect ratio and arbitrary orientation of vehicle targets. This paper proposes a morphology-based feature extraction network (MFE-Net), which fully uses the prior shape knowledge of the vehicle targets. Specifically, we adopt rotatable bounding boxes to predict the targets, and a novel rectangular rotation-invariant coordinate convolution (RRICC) is proposed to extract the feature, which can determine more accurately the convolutional sampling location of the vehicles. The adaptive thresholding denoising module (ATDM) is designed to suppress background clutter. Furthermore, inspired by the convolutional neural networks (CNNs) and self-attention, we propose the hybrid representation enhancement module (HREM) to highlight the vehicle target features. The experiment results show that the proposed model obtains an average precision (AP) of 93.1% on the SAR vehicle detection data set (SVDD).

Ship detection is one of the main problems of satellite image analysis. Since ships are scattered on the sea and major ports, large-area remote-sensing images need to be processed in order to realize the detection of ships. In addition, since the satellite is a top-down view, the ship with aspect ratios cannot be covered in complex backgrounds by a horizontal bounding box very well and need a rotating bounding box to achieve this task. Although considerable progress has been made in object detection techniques, there are still challenges for fast detection of ships in large-area remote-sensing images. In this letter, an arbitrary-oriented detector for large-area remote-sensing images is proposed to quickly locate ship positions. A new feature extraction network DCNDarknet25 based on you only look once (YOLO) is designed by reducing paraments and adding deformable convolution (DCN) to improve the speed and accuracy. And the rotation detection capability without angle regression is added to the YOLO detection algorithm for the first time. Finally, thanks to the advantages of our fully convolutional lightweight network, a method for detecting large-area remote-sensing images at once is proposed. In the public dataset HRSC2016 and our own large-area remote-sensing (LARS) image dataset, it has achieved very good accuracy and several times the speed of other algorithms.

Object detection has a great significance to remote sensing image recognition. Conventional object detection methods by using horizontal bounding box have shown good performance in general images. However, because of looking down perspective of remote sensing images, rotation bounding box is more suitable and precise for object detection. Although rotation bounding box has been researched by some works for remote sensing images, most of them are still two stages. In this paper, by improvement of YOLO architecture, we propose an one-stage object detection method to predict rotation bounding box for remote sensing images. Firstly, in the data preprocessing stage, an image cropping operation is utilized instead of zooming operation to generate suitable input data for network. Meanwhile, in order to avoid an object from being divided, an overlapping band is set during cropping. Secondly, a novel rotation bounding box representation method is introduced, and a corresponding loss function is designed in training process. Experiments on the Dota dataset demonstrate that our method outperforms state-of-the-art rotation objects detection methods, in terms of mAP, our method achieves 73.50%mAP, which is higher than methods based on rotation bounding box by 2.0%.

Oriented object detection is a challenging task in aerial images since the objects in aerial images are displayed in arbitrary directions and are frequently densely packed. The mainstream detectors describe rotating objects using a five-parameter or eight-parameter representations, which have representation ambiguity for orientated object definition and border loss discontinuity in the regression process. We proposed an innovative representation method based on area ratio of parallelogram, called area ratio of parallelogram (ARP). Specifically, ARP regresses the minimum bounding rectangle of the oriented object and three area ratios. Three area ratios include the area ratio of a directed object to the smallest circumscribed rectangle and two parallelograms to the minimum circumscribed rectangle. It simplifies offset learning and eliminates the issue of angular periodicity or label point sequences for oriented objects. To further remedy the confusion issue of nearly horizontal objects, the area ratio between the object and its minimal circumscribed rectangle is employed to guide the selection of horizontal or oriented detection for each object. Moreover, the rotated efficient intersection over union loss with horizontal bounding box and three area ratios are designed to optimize the bounding box regression for rotating objects. Experimental results on remote sensing datasets, including HRSC2016, DOTA, and UCAS-AOD, show that the proposed method achieves superior detection performance than many state-of-the-art approaches.