DRDet: Dual-Angle Rotated Line Representation for Oriented Object Detection

Detecting oriented objects in optical remote sensing images has been consistently challenging due to difficulties in bounding boxes localization. The cascaded regression framework, widely employed for high-quality bounding box refinement, has demonstrated effectiveness in this domain. However, our experiments reveal a discontinuity issue in bounding box optimization in cascaded regression framework. As a result, performance gain is not guaranteed across all stages in this framework. In this paper, we propose a Distribution Discriminative Detector(DDDet) to address the above issues and enhance the optimization of bounding boxes in oriented object detection. Specifically, a novel Conditional Anchor Refinement Framework(CARF) is designed to improve cascaded regression structure. CARF distinguishes bounding boxes with different distributions, adaptively optimizing them within the well-assigned regressors. Subsequently, the Aligned Convolution Module(ACM) is integrated into each regressor, facilitating the continuous alignment between features and refined anchors. Furthermore, the Geometry-guided Training Sample Selection(GTSS) method is incorporated into CARF to assign labels based on object shape priors. Experimental results show that DDDet obtains state-of-the-art performance on mainstream datasets for oriented object detection in remote sensing image, which demonstrates the effectiveness of the proposed method. Our method surpasses many current single-stage detectors, two-stage detectors, and refine-stage detectors, achieving the mAP of 79.41% on DOTA dataset, and 44.15% on FAIR1M dataset.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

Object detection for aerial images has achieved remarkable progress in recent years. Nevertheless, most exiting studies do not differentiate oriented object detection from horizontal detection. Certain schemes ignore the ambiguity of oriented object representation and leverage label assignment designed for horizontal object detection directly. Consequently, it leads to unstable training and causes performance degradation, because high-quality samples surrounding the oriented bounding boxes can not be leveraged effectively. To address this problem, we propose a gliding Free, orientation Free, and anchor Free Network (Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net) with high-efficiency for oriented object detection. Specifically, we propose an unambiguous oriented object representation scheme, named FreeGliding, by gliding the projection points of samples on each edge of horizontal bounding boxes. It makes the detection largely free from representation ambiguity and multi-task dependency. To overcome the restrictions of label assignment, we put forward a novel Loss-aware Outer Sample Selection (LOSS) scheme, which takes into consideration spatial information and localization capability to retain high-quality samples surrounding the objects. Moreover, we introduce an Oriented Feature Fusion (OFF) scheme to tackle feature alignment by adjusting the receptive field and fusing oriented features dynamically. Experimental results on two large-scale remote sensing datasets HRSC2016 and DOTA demonstrate that Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net outperforms the state-of-the-art schemes with a large margin. We hope our work can inspire rethinking the design of anchor-free detectors, and serve as a strong baseline for oriented object detection.

Object detection in aerial images having many practical applications in real life is becoming popular along with the surging development of deep learning and UAVs (Unmanned Aerial Vehicles). However, adverse weather conditions such as rain, night, and fog might reduce the quality of input images and significantly affect the performance of many perfectly trained detectors (detectors are trained in clear weather conditions). Moreover, object detection in aerial images often seeks high accuracy, while well-known object detection methods use horizontal bounding boxes to represent the object's location. These issues raise the inconsistency between classification and bounding box regression. Understanding the need for practical solutions, we create two experimental Fog datasets based on the original DOTA dataset to provide the in-depth analysis of fog density on multiple well-established oriented object detectors, namely Gliding Vertex, R3Det and ReDet. Furthermore, our training resources achieve promising results, flexibly ensemble to other methods to enhance models' performance and adapt to many adverse weather conditions.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Detecting small objects is notoriously challenging due to their low resolution and noisy representation. Existing object detection pipelines usually detect small objects through learning representations of all the objects at multiple scales. However, the performance gain of such ad hoc architectures is usually limited to pay off the computational cost. In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to "super-resolved" ones, achieving similar characteristics as large objects and thus more discriminative for detection. For this purpose, we propose a new Perceptual Generative Adversarial Network (Perceptual GAN) model that improves small object detection through narrowing representation difference of small objects from the large ones. Specifically, its generator learns to transfer perceived poor representations of the small objects to super-resolved ones that are similar enough to real large objects to fool a competing discriminator. Meanwhile its discriminator competes with the generator to identify the generated representation and imposes an additional perceptual requirement - generated representations of small objects must be beneficial for detection purpose - on the generator. Extensive evaluations on the challenging Tsinghua-Tencent 100K and the Caltech benchmark well demonstrate the superiority of Perceptual GAN in detecting small objects, including traffic signs and pedestrians, over well-established state-of-the-arts.

Recently, the compressive tracking (CT) method has attracted much attention due to its high efficiency, but it cannot well deal with the large scale target appearance variations due to its data-independent random projection matrix that results in less discriminative features. To address this issue, in this paper, we propose an adaptive CT approach, which selects the most discriminative features to design an effective appearance model. Our method significantly improves CT in three aspects. First, the most discriminative features are selected via an online vector boosting method. Second, the object representation is updated in an effective online manner, which preserves the stable features while filtering out the noisy ones. Furthermore, a simple and effective trajectory rectification approach is adopted that can make the estimated location more accurate. Finally, a multiple scale adaptation mechanism is explored to estimate object size, which helps to relieve interference from background information. Extensive experiments on the CVPR2013 tracking benchmark and the VOT2014 challenges demonstrate the superior performance of our method.

This paper proposes a new object representation using a panoramic image and addresses a SIFT-based matching approach for counting the number of this object in an image. The proposed object panoramic image records the texture information of the object from all (360- degree) directions. Using it as a reference for finding the same object in another image involves the task of feature matching between a cylindrical panoramic and a planar view of the object. The main advantage of using this object representation is that it is possible to derive the orientation information of the detected object in the image. Preliminary experiments were conducted to verify the proposed algorithm.

A feature representation approach is proposed for discriminative features extraction and this object representation tend to handle the large amount of local features in feature correspondence. Object representation with shape and color feature tends to certify the strength of proposed feature extraction method. In the proposed method, HOG are extracted on 300 corner points which are the strongest points on detected corners and these points are supposed as in one block to get the HOG vector. As a second portion of feature extraction, the moments on HSI are extracted on each separated channel. The proposed feature extraction method is tested intensively on the different sequences of the Online Benchmark Tracking dataset, CAVIAR Test Case Scenarios and Change Detection dataset (CDnet 2014) with the comparison of other related feature extraction methods. Classification of proposed approach receives 98.1%, 93.8%, 96.8%, 97.7% and 90.5% for walking, crossing, walk1, pedestrians and twopositionPTZCam respectively.

A survey on object deformation and decomposition in computer graphics

Chaotic target representation for robust object tracking

A framework is presented which uses a polar representation of a segmented object for shape classification. This method produces a position, rotation and scale invariant representation of the shape. An efficient method for extracting multiple contours from the polar representation is used to handle the problem of many-to-one mappings in the radial and angular parameters. The contours are used to find interesting vertices of the shape. The shape information is mapped to spatial regions on a polar grid and fed into a multi-layer perceptron for classification. The framework is tested on manually segmented images of people's hands and on side views of automobiles. The results show that the network can achieve approximately 100% generalization on test data even though the network is under trained.

Source-free object detection (SFOD) aims to adapt pre-trained source models to the target domain without accessing the source domain. In this paper, we focus on two challenges that are unexplored in existing SFOD methods but prevalent in aerial scenes: Ⅰ) class imbalance, and Ⅱ) scale representation imbalance, i.e., the feature representations of objects at different scales vary significantly, even within classes. To address these problems, we propose a novel SFOD method in aerial images known as dual imbalance suppression. Specifically, for challenge Ⅰ, we introduce a class-aware weighting strategy, which adjusts the weights of each class according to the model learning state to realize dynamic balance among classes. For challenge Ⅱ, we design a scale-aware contrastive learning (SACL), which combines fine constraints on intra-scale features and coarse constraints on inter-scale features to improve the compactness of intra-class features, while eliminating the negative impact of forcibly aligning different scale features with significant representation differences. In addition, we propose a multi-scale interaction transformer module that not only enhances feature representation but also constructs high-quality positive/negative sample pairs for SACL. Extensive experiments on 5 aerial scenes demonstrate that our method outperforms existing state-of-the-art SFOD methods.

This paper presents a unified haptic modeling method for physical systems irrespective of their types. Previously, we have proposed an energy-based approach which haptically renders rigid and deformable objects without reference to their material. However, it is not easy to represent the haptic behavior of a fluid or fluid-deformable object with the previously proposed method because there is no method for handling fluid objects and no additional interface between a fluid object and the other objects. Therefore, this paper proposes a modeling framework in which rigid, deformable, and fluid objects are represented by one modeling method.

In this letter, we explore generalizable, perception-to-action robotic manipulation for precise, contact-rich tasks. In particular, we contribute a framework for closed-loop robotic manipulation that automatically handles a category of objects, despite potentially unseen object instances and significant intra-category variations in shape, size and appearance. Previous approaches typically build a feedback loop on top of a realtime 6-DOF pose estimator. However, representing an object with a parameterized transformation from a fixed geometric template does not capture large intra-category shape variation. Hence we adopt the keypoint-based object representation proposed in [13] for category-level pick-and-place, and extend it to closed-loop manipulation policies with contact-rich tasks. We first augment keypoints with local orientation information. Using the oriented keypoints, we propose a novel object-centric action representation in terms of regulating the linear/angular velocity or force/torque of these oriented keypoints. This formulation is surprisingly versatile – we demonstrate that it can accomplish contact-rich manipulation tasks that require precision and dexterity for a category of objects with different shapes, sizes and appearances, such as peg-hole insertion for pegs and holes with significant shape variation and tight clearance. With the proposed object and action representation, our framework is also agnostic to the robot grasp pose and initial object configuration, making it flexible for integration and deployment. Video demonstration, source code and supplemental materials are available on https://sites.google.com/view/kpam2/home .