Single Shot Anchor Refinement Network for Oriented Object Detection in Optical Remote Sensing Imagery

Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$ by introducing a generalized version as both a new loss and a new metric. By incorporating this generalized $IoU$ ($GIoU$) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, $IoU$ based, and new, $GIoU$ based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.

Among the artificial intelligence based studies conducted in the field of agriculture, disease recognition methods founded on deep learning are observed to become widespread. Due to the diversity and regional specificity of many plant species, studies performed in this field are not at the desired level. Olive peacock spot disease of the olive plant which grows only in certain regions in the world is a widely encountered disease particularly in Turkey. The aim of this research is to develop an olive peacock spot disease detection system using a Single Shot Detector (SSD) which is one the popular deep learning architectures to support olive farmers. This study presents a data set consisting of 1460 olive leaves samples for the detection of olive peacock spot disease. All of the images of the olive leaves which produced under controlled conditions were collected from Aegean region of Turkey during spring and summer. The data set was trained with different intersection over union (IoU) threshold values using SSD architecture. A 96 % average precision (AP) value was obtained with IoU=0.5. As IOU value goes up from 0.5, erroneously classified olive peacock spot disease symptoms growed larger as well. The AP curve becomes flat when between 0.1 and 0.5, and it decreases when greater than 0.5. This analysis showed that the IoU significantly influenced the performance of SSD based model in detection of olive peacock spot disease. In addition to, trainings were performed by employing Pytorch library and a GUI was developed for the SSD based application using PyQt5 which is one of Pyhton's libraries. Results showed that the SSD was a robust tool for recognizing the olive peacock spot disease.

Oriented object detection is a challenging task in aerial images since the objects in aerial images are displayed in arbitrary directions and are frequently densely packed. The mainstream detectors describe rotating objects using a five-parameter or eight-parameter representations, which have representation ambiguity for orientated object definition and border loss discontinuity in the regression process. We proposed an innovative representation method based on area ratio of parallelogram, called area ratio of parallelogram (ARP). Specifically, ARP regresses the minimum bounding rectangle of the oriented object and three area ratios. Three area ratios include the area ratio of a directed object to the smallest circumscribed rectangle and two parallelograms to the minimum circumscribed rectangle. It simplifies offset learning and eliminates the issue of angular periodicity or label point sequences for oriented objects. To further remedy the confusion issue of nearly horizontal objects, the area ratio between the object and its minimal circumscribed rectangle is employed to guide the selection of horizontal or oriented detection for each object. Moreover, the rotated efficient intersection over union loss with horizontal bounding box and three area ratios are designed to optimize the bounding box regression for rotating objects. Experimental results on remote sensing datasets, including HRSC2016, DOTA, and UCAS-AOD, show that the proposed method achieves superior detection performance than many state-of-the-art approaches.

It is challenging to accurately detect flexible objects with arbitrary orientation from monitoring images in power grid maintenance and inspection sites. This is because these images exhibit a significant imbalance between the foreground and background, which can lead to low detection accuracy when using a horizontal bounding box (HBB) as the detector in general object detection algorithms. Existing multi-oriented detection algorithms that use irregular polygons as the detector can improve accuracy to some extent, but their accuracy is limited due to boundary problems during the training process. This paper proposes a rotation-adaptive YOLOv5 (R_YOLOv5) with a rotated bounding box (RBB) to detect flexible objects with arbitrary orientation, effectively addressing the above issues and achieving high accuracy. Firstly, a long-side representation method is used to add the degree of freedom (DOF) for bounding boxes, enabling accurate detection of flexible objects with large spans, deformable shapes, and small foreground-to-background ratios. Furthermore, the further boundary problem induced by the proposed bounding box strategy is overcome by using classification discretization and symmetric function mapping methods. Finally, the loss function is optimized to ensure training convergence for the new bounding box. To meet various practical requirements, we propose four models with different scales based on YOLOv5, namely R_YOLOv5s, R_YOLOv5m, R_YOLOv5l, and R_YOLOv5x. Experimental results demonstrate that these four models achieve mean average precision (mAP) values of 0.712, 0.731, 0.736, and 0.745 on the DOTA-v1.5 dataset and 0.579, 0.629, 0.689, and 0.713 on our self-built FO dataset, exhibiting higher recognition accuracy and a stronger generalization ability. Among them, R_YOLOv5x achieves a mAP that is about 6.84% higher than ReDet on the DOTAv-1.5 dataset and at least 2% higher than the original YOLOv5 model on the FO dataset.

The development of technological devices, such as satellites and drones, has made it easier to collect images and videos from above. With rich and diverse data sources, the problem of detecting objects in aerial images is formed to serve real-life situations: rescue missions, social scenarios, security needs, and more. This issue attracts a lot of attention from researchers in computer vision because of its high applicability in real life. However, object detection in images collected from drones faces many significant challenges due to the complex factors of real-life environments. To overcome these difficulties, many studies focus on developing effective models for detecting and tracking objects from UAV systems. In this paper, our primary emphasis is comprehensive survey of methodologies pertaining to horizontal and oriented object detection. We analyzed and compared existing approaches such as CNN-based vs. Transformer-based, Anchor-based vs. Anchor-free. Moreover, we also learned about ways to represent objects using bounding boxes in oriented object detection. After researching many practical and innovative techniques in different approaches, we have carefully selected 15 methods for horizontal object detection and 18 for oriented object detection that were recently researched and developed to conduct classifications, surveys and assessments. We also provide a taxonomy mind map to assist readers in gaining an overview of the methods we surveyed. Additional, we examine 17 aerial image datasets collected from platforms and in various locations worldwide. These are typical datasets used to serve two primary tasks: Horizontal Object Detection and Oriented Object Detection. Then, we compared the experimental resultsof the previously introduced methods on four typical datasets: VisDrone, UAVDT, DOTAv1.0, and HRSC2016. The above datasets present diversity in the number of classes and the complexity of real-life conditions. Furthermore, they have been tested with various methods and shown remarkable results. For the reasons mentioned, we have enough basis to confirm that the comparison results are objective so that other research groups can make accurate comments and assessments on experimental methods on each dataset. We believe that updating and synthesizing the latest research will provide complete and comprehensive information, helping researchers to have more survey material for the development of the field of object detection in the future.

Accurate object detection requires correct classification and high-quality localization. Currently, most of the single shot detectors (SSDs) conduct simultaneous classification and regression using a fully convolutional network. Despite high efficiency, this structure has some inappropriate designs for accurate object detection. The first one is the mismatch of bounding box classification, where the classification results of the default bounding boxes are improperly treated as the results of the regressed bounding boxes during the inference. The second one is that only one-time regression is not good enough for high-quality object localization. To solve the problem of classification mismatch, we propose a novel reg-offset-cls (ROC) module including three hierarchical steps: the regression of the default bounding box, the prediction of new feature sampling locations, and the classification of the regressed bounding box with more accurate features. For high-quality localization, we stack two ROC modules together. The input of the second ROC module is the output of the first ROC module. In addition, we inject a feature enhanced (FE) module between two stacked ROC modules to extract more contextual information. The experiments on three different datasets (i.e., MS COCO, PASCAL VOC, and UAVDT) are performed to demonstrate the effectiveness and superiority of our method. Without any bells or whistles, our proposed method outperforms state-of-the-art one-stage methods at a real-time speed. The source code is available at https://github.com/JialeCao001/HSD.

With the advancement of object detection, oriented object detection is gradually replacing horizontal object detection as the preferred solution in remote sensing images. However, most existing datasets are annotated with horizontal bounding boxes, while re-annotating these datasets is both costly and inefficient. Recently, horizontal box supervised oriented object detection methods using merely horizontal box annotations have emerged as a solution. Nonetheless, these methods are relatively poor in predicting angles because of lacking angle supervision. Additionally, these methods overlook edge features, which can help capture and represent the oriented boundaries between objects and backgrounds, especially for objects with distinct edges or in blank backgrounds. To address these issues, we propose an edge feature self-attention and multi-task synthetic pattern enhanced horizontal box supervised oriented object detector named EAMSDet, aiming to achieve high precision based on horizontal box annotations. Specifically, we use horizontal and vertical Sobel operators to extract edge information and a self-attention module to fuse the edge information with the feature maps. Furthermore, to enrich the supervision of aspect ratio and angles, we add a multi-task synthetic branch and an extra regression loss to make full use of our introduced synthetic visual patterns. Based on H2RBox-v2, we conducted extensive experiments on remote sensing datasets such as DOTA-v1.0, DIOR, and HRSC2016, demonstrating the effectiveness and generalizability of our EAMSDet.

Software was developed in Python for the digital processing of object images before their detection using the You Only Look Once (YOLO) artificial neural network. Detection was carried out on object images, specifically vehicles and other road users. The preprocessing of images was performed through histogram equalization and local contrast enhancement. The original images were read from video cameras or graphic files. Image detection was performed using the medium-sized YOLO convolutional neural network model, YOLOv8m. As a result of detection, the studied objects, particularly cars, were highlighted with rectangular bounding boxes. To evaluate the accuracy of object detection, several metrics (parameters) were used: Precision, Recall, Intersection over Union (IoU), F1, and Average Precision (AP). The YOLO network was pre-trained on the Common Objects in Context dataset. The YOLO neural network returns the object recognition confidence, the coordinates of its center (x, y) on the image, and its width and height (w, h). The detected objects are marked with rectangular bounding boxes. The positioning of objects in the images was manually performed using the Roboflow tool, which allows for accurate spatial positioning of each object. The study of the impact of image preprocessing on the accuracy of object detection by the YOLO network was conducted using a typical traffic scene image. After histogram equalization, the histogram becomes more symmetrical and uniform, improving the visual quality of both dark and bright areas. This led to improved object detection accuracy across almost all metrics. The increase in detection accuracy by the IoU parameter is 0.169. The summary improvement in detection accuracy by the IoU, F1, AP parameters is 0.502. It was shown that after image preprocessing, detection accuracy increased, and all vehicles were correctly detected. The developed software can be used in various computer vision systems as well as in Internet of Things systems for remote monitoring and control.

How to represent the object more appropriately in oriented object detection is an essential problem to be solved, there are many solutions for the object represented. It is a relatively novel approach to represent objects as a number of sample points useful for both localization and recognition. However, the current point-set-based representation methods do not effectively supervise all points for learning, and the internal information of the convex hull in the point set cannot be effectively learned. Therefore, this letter proposes point set distance (PSD) loss, which learns set-to-set supervision of objects to effectively represent objects. Besides, most of the current sample selection strategies are based on the Intersection over Union (IoU), but these methods cannot comprehensively measure candidate samples quality. To select high-quality point sets, we propose to use the probability distribution of point sets to select the positive samples. Our probabilistic point set sample selection (PPSS) scheme effectively exploits the classification information, regression information, and the distribution characteristics of the point set. Experimental results on remote sensing image datasets including DOTA, DIOR-R, and HRSC2016, demonstrate the proposed method for arbitrary-oriented object detection achieves consistent and substantial improvements.

This research focuses on assessing and developing a real-time detection system using the YOLOv8 algorithm. Accurate and fast vehicle detection is a big challenge in modern traffic management, especially in various environmental conditions such as bad weather, low lighting, and high traffic density. The aim of this study was to evaluate the performance of YOLOv8 under these conditions and identify potential improvements. The dataset used consists of 16,990 vehicle images with various variations and environmental conditions. After being trained, the model is evaluated using metrics such as precision, recall, and F1-score, as well as Intersection over Union (IoU) with a threshold of 0.8 on IoU. The results show that YOLOv8 is superior with a fairly high detection accuracy of 78%, with precision of 82% and recall above 90%, and is able to detect vehicles in real-time conditions. However, the challenge of detecting small objects or irregularly shaped vehicles such as tractors still needs to be optimized. This research also compared the performance of YOLOv8 with the SSD (Single Shot Detector) algorithm, where YOLOv8 was proven to be superior in terms of accuracy, precision, recall and F1-score. The research results obtained provide valuable insights for the development of traffic management systems based on deep learning technology. The main contribution of this research is to provide a more efficient and effective vehicle detection solution, which can be applied in modern traffic management systems. Thus, it is hoped that the results of this research can increase the efficiency of traffic management and have a positive impact on the development of intelligent transportation systems in the future.

Compared with general object detection with horizontal bounding boxes in natural images, oriented object detection in remote sensing images is an active and challenging research topic as objects are usually displayed in arbitrary orientations. To model the variant orientations of oriented objects, general CNN-based methods usually adopt more parameters or well-designed modules, which are often complex and inefficient. To address this issue, the detector requires two key components to deal with: (i) generating oriented proposals in a light-weight network to achieve effective representation of arbitrarily oriented objects; (ii) extracting the rotation-invariant feature map in both spatial and orientation dimensions. In this paper, we propose a novel, lightweight rotated region proposal network to produce arbitrary-oriented proposals by sliding two vertexes only on adjacent sides and adopt a simple yet effective representation to describe oriented objects. This may decrease the complexity of modeling orientation information. Meanwhile, we adopt the rotation-equivariant backbone to generate the feature map with explicit orientation channel information and utilize the spatial and orientation modules to obtain completely rotation-invariant features in both dimensions. Without tricks, extensive experiments performed on three challenging datasets DOTA-v1.0, DOTA-v1.5 and HRSC2016 demonstrate that our proposed method can reach state-of-the-art accuracy while reducing the model size by 40% in comparison with the previous best method.

Currently, reliable and accurate oriented detection in remote sensing images still needs to be improved. The wide variation of object shapes and orientations in the remote sensing images usually leads to two issues in two-stage oriented object detectors. One issue is how to generate high-quality rotation proposals. The other is the angular error sensitivity to the aspect ratios in the angle optimization process. In this paper, we propose a novel rotation proposal generation and optimization detector, which is based on high-quality rotation proposal generation and adaptive angle optimization to solve these two issues. The proposed method mainly establishes the geometric relationship guided region proposal networks (GRG-RPN) and the adaptive angle optimization head (AAO-Head) to achieve more accurate oriented object detection. The GRG-RPN only uses a simple network and a small number of horizontal anchors to predict high-quality rotation proposals. This approach was derived via the calculation based on the theoretical analysis of the geometric relationship between the oriented bounding boxes (OBBs) and their external horizontal bounding boxes (EHBBs). The AAO-Head solves the angular error sensitivity to the aspect ratios and achieves adaptive angle optimization using a new regression parameter, which is defined based on the theoretical analysis of the relationship between the Intersection over Union (IoU), the angular errors and the aspect ratios. Experiments show that our method can achieve 2.5% mAP improvement averagely versus the compared SOTA methods, and achieve 0.5% mAP improvement versus the next best method Oriented R-CNN with fewer regression parameters and a simpler regression approach.

Object detection for aerial images has achieved remarkable progress in recent years. Nevertheless, most exiting studies do not differentiate oriented object detection from horizontal detection. Certain schemes ignore the ambiguity of oriented object representation and leverage label assignment designed for horizontal object detection directly. Consequently, it leads to unstable training and causes performance degradation, because high-quality samples surrounding the oriented bounding boxes can not be leveraged effectively. To address this problem, we propose a gliding Free, orientation Free, and anchor Free Network (Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net) with high-efficiency for oriented object detection. Specifically, we propose an unambiguous oriented object representation scheme, named FreeGliding, by gliding the projection points of samples on each edge of horizontal bounding boxes. It makes the detection largely free from representation ambiguity and multi-task dependency. To overcome the restrictions of label assignment, we put forward a novel Loss-aware Outer Sample Selection (LOSS) scheme, which takes into consideration spatial information and localization capability to retain high-quality samples surrounding the objects. Moreover, we introduce an Oriented Feature Fusion (OFF) scheme to tackle feature alignment by adjusting the receptive field and fusing oriented features dynamically. Experimental results on two large-scale remote sensing datasets HRSC2016 and DOTA demonstrate that Free <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\rm ^{3}$</tex-math></inline-formula> Net outperforms the state-of-the-art schemes with a large margin. We hope our work can inspire rethinking the design of anchor-free detectors, and serve as a strong baseline for oriented object detection.

In recent years, in the field of remote sensing image target detection, oriented target detection has received extensive attention. Traditional detectors use horizontal bounding box (HBB) to locate targets, which may cause inaccurate detection when detecting targets with arbitrary orientation angles, dense distribution and large aspect ratios. The Oriented Bounding Box (OBB) adds different rotation angles to the horizontal bounding box, which can better deal with the above problems. However, with the introduction of oriented bounding boxes in the detector, new problems have emerged, such as the increase in the number of anchor points with different scales, angles, and aspect ratios, and the imbalance of IoU matching of positive and negative anchor boxes, which leads to classification confidence and Inconsistency between positioning accuracy. In order to overcome these shortcomings while using the oriented bounding box, this paper proposes a novel oriented target detector and redesigns a matching strategy for evaluating the position of the anchor frame. By selecting a higher quality anchor frame to achieve more accurate target detection, thereby reducing the instability of the angle in the matching process. Experiments conducted on public remote sensing data sets such as HRSC2016 and UCAS-AOD show that the proposed method achieves advanced detection accuracy.

Intersection over Union (IoU) is an important function in object detection based on deep learning. But, there is a gap between the used distance losses and this metric value of maximizing. This paper improves the IoU function based on IoU. Redesigned the neural network structure, and used the PASCAL VOC2012 dataset. The structure model of the neural network is compare with others in the accuracy of object detection. Experimental results show that our method can achieve good results in object detection. The approach provides a new idea for object detection.