A Novel Object Detector Based on High-Quality Rotation Proposal Generation and Adaptive Angle Optimization

Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$ by introducing a generalized version as both a new loss and a new metric. By incorporating this generalized $IoU$ ($GIoU$) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, $IoU$ based, and new, $GIoU$ based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.

Coffee sequential flowering events detection and flower density estimation is essential to predict the ripening time and yield of coffee. In this study, we detected coffee flowering events automatically based on estimated flower densities in high spatial time-series digital images, using a multi-scale region based flower segmentation method. The study area is a coffee plantation in Lujiangba in the Yunnan Province of China. There are 5 flowering events in the coffee flowering period. A digital camera obtained 24 RGB images at each shooting time of day (8:00, 9:00, 10:00, 11:00, 12:00, 13:00, 14:00, 15:00, 16:00, 17:00, 17:30) by automatic adjusting the sensor with 3 depression angles and 8 azimuth angles during the coffee flowering period from March 1st to May 31st. For segment the flowers in one image, multi-scale regions were primarily generated by equally-sized superpixel segmentation and subsequent superpixel merging process. Next, the feature vectors of each region were extracted by color moments (CM) operator and local binary patterns (LBP) operator. Afterwards, the support vector machine (SVM) classifier trained on these features was applied to recognize the flower regions. Thus, the percentage of flower pixels referred as flower proportion (FP), which can estimate the image-based flower density, was calculated in preparation for detecting flowering events of time-series images. In this stage, coefficients of Recall, Precision and intersection over union (IoU) were employed to evaluate the performance of segmentation methods on 14 test images at threes depression angles and then the best flower segmentation algorithm and the optimal angle can be determined. In the stage of flowering events detection, the FPs of different shooting time multitemporal images under the optimal depression angle were calculated and plotted. Then, a threshold of FP, K, was selected to determine whether an image is on a flowering day. To determine the best shooting time for flowering events detection, Recall, Precision and IoU were also employed to evaluate the performance of time-series images shot at 11 time of day for flowering day detection. The results show that the images shot at the depression angle of 77.5 degree is the optimal depression angle for flower segmentation and meanwhile our proposed method achieves the best performance with Recall, Precision and IoU of 84.89%, 74.83% and 65.46% respectively. In the test of flowering day detection of 11 shooting time multitemporal images, the time-series images shot at 13:00 is superior to the time-series images shot at other time of day, with Recall of 65.00%, Precision of 100% and IoU of 65.00% when the K set as 0.4%. Meanwhile, all flowering events can be detected except the fifth event which has few flowers and the FPs of time-series images can correctly indicate the flower densities of each events. In conclusion, our approach can estimate the image-based flower density and detect the coffee sequential flowering events in small fields, so the results can be used for coffee fruit maturity prediction and yield estimation.

An object detector based on convolutional neural network (CNN) has been widely used in the field of computer vision because of its simplicity and efficiency. The average accuracy of CNN model detection results in the object detector is greatly affected by the loss function. The precision of the localization algorithm in the loss function is the main factor affecting the result. Based on the complete intersection over union (CIoU) loss function, an improved penalty function is proposed to improve the localization accuracy. Specifically, the algorithm more comprehensively considers matching bounding boxes between prediction with ground truth, using the proportional relationship of the aspect ratio from both bounding boxes. Under the same aspect ratio of the two bounding boxes, the influence factors of the prediction box on localization accuracy were considered. In this way, the function of the penalty function is strengthened, and localization accuracy of the network model improved. This loss function is called Improved CIoU (ICIoU). Experiments on the Udacity, PASCAL VOC, and MS COCO datasets have demonstrated the effectiveness of ICIoU in improving localization accuracy of network models by using the one-stage object detector YOLOv4. Compared with CIoU, the proposed ICIoU improved average precision (AP) by 0.57% and AP75 by 0.12% on Udacity, AP by 0.26% and AP75 by 1.28% on PASCAL VOC, and AP by 0.06% and AP75 by 0.65% on MS COCO.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

Existing works in object detection usually adopt different Intersection over Union (IoU) metrics to define the similarity between predicted box and target box, which greatly boost the detection performance. For the task of aerial object detection, since the predicted and target boxes are with different sizes, aspect ratios and orientations, existing IoU metrics can not measure their similarity well. In this work, we propose a new and generic metric for robust aerial object detection, called EllipseIoU, which incorporates the ellipse distance between the predicted box and the target box. In particular, we find the inner ellipse from the box, and then calculate the ellipse distance between predicted and target boxes based on the distances of focal points. We prove that the proposed ellipse distance well measures the relations of sizes, aspect ratios and orientations between predicted and target boxes. Moreover, we also incorporate the relation of short axis length between two boxes to measure the relation of sizes. Based on the ellipse distance, we design the EllipseIoU metric by combining with the IoU metric and integrate it into detectors for performance boosting of aerial object detection. Experimental results on public datasets show that the detector with EllipseIoU significantly improve the baseline without impact on the efficiency and achieves state-of-the-art performance.KeywordsAerial object detectionIntersection over Union (IoU)

The you-only-look-once (YOLO) v2 object detector uses a single stage object detection network. YOLO v2 is faster than other two-stage deep learning object detectors, such as regions with convolutional neural networks (Faster R-CNNs).The YOLO v2 model runs a deep learning CNN on an input image to produce network predictions. The object detector decodes the predictions and generates bounding boxes YOLO v2 uses anchor boxes to detect classes of objects in an image. For more details, see Anchor Boxes for Object Detection. The YOLO v2 predicts these three attributes for each anchor box: Intersection over union (IoU) — Predicts the objectness score of each anchor box. Anchor box offsets — Refine the anchor box position. Class probability — Predicts the class label assigned to each anchor box. The figure shows predefined anchor boxes (the dotted lines) at each location in a feature map and the refined location after offsets are applied. Matched boxes with a class are in color. You can design a custom YOLO v2 model layer by layer. The model starts with a feature extractor network, which can be initialized from a pretrained CNN or trained from scratch. The detection subnetwork contains a series of Conv, Batch norm, and ReLu layers, followed by the transform and output layers, yolov2TransformLayer and yolov2OutputLayer objects, respectively.yolov2TransformLayertransforms the raw CNN output into a form required to produce object detections.yolov2OutputLayerdefines the anchor box parameters and implements the loss function used to train the detect.

Siamese network-based visual tracking algorithms have achieved excellent performance in recent years, but challenges such as fast target motion, shape and scale variations have made the tracking extremely difficult. The regression of anchor-free tracking has low computational complexity, strong real-time performance, and is suitable for visual tracking. Based on the anchor-free siamese tracking framework, this paper firstly introduces balance factors and modulation coefficients into the cross-entropy loss function to solve the classification inaccuracy caused by the imbalance between positive and negative samples as well as the imbalance between hard and easy samples during the training process, so that the model focuses more on the positive samples and the hard samples that make the major contribution to the training. Secondly, the intersection over union (IoU) loss function of the regression branch is improved, not only focusing on the IoU between the predicted box and the ground truth box, but also considering the aspect ratios of the two boxes and the minimum bounding box area that accommodate the two, which guides the generation of more accurate regression offsets. The overall loss of classification and regression is iteratively minimized and improves the accuracy and robustness of visual tracking. Experiments on four public datasets, OTB2015, VOT2016, UAV123 and GOT-10k, show that the proposed algorithm achieves the state-of-the-art performance.

YOLO-YCbCr based smoke segmentation method for fire detection was proposed in this study to enhance smoke segmentation performance with reduction in computational cost. The proposed method consists of the deep learning object detection model, You Only Look Once (YOLO), and the color space-based segmentation model, rule-based YCbCr. YOLO is used to detect smoke objects in the input fire images through bounding boxes. The images in the bounding boxes are characterized based on YCbCr color space and the proposed YCbCr rules derived by considering smoke image characteristics segment smoke region. In order to evaluate the smoke segmentation performance, six different fire incident video data were used. The mean intersection over union (IoU) value of the proposed method was improved by approximately 31.3% and 24.5% respectively when compared to the reference models: YOLO-RGB model and YOLO-CIELAB model. It was found that YOLO significantly reduced the erroneous smoke segmentation and YCbCr rules derived from the smoke color features were effective in the smoke segmentation.

Since Intersection-over-Union (IoU) based optimization maintains the consistency of the final IoU prediction metric and losses, it has been widely used in both regression and classification branches of single-stage 2D object detectors. Recently, several 3D object detection methods adopt IoU-based optimization and directly replace the 2D IoU with 3D IoU. However, such a direct computation in 3D is very costly due to the complex implementation and inefficient backward operations. Moreover, 3D IoU-based optimization is sub-optimal as it is sensitive to rotation and thus can cause training instability and detection performance deterioration. In this paper, we propose a novel Rotation-Decoupled IoU (RDIoU) method that can mitigate the rotation-sensitivity issue, and produce more efficient optimization objectives compared with 3D IoU during the training stage. Specifically, our RDIoU simplifies the complex interactions of regression parameters by decoupling the rotation variable as an independent term, yet preserving the geometry of 3D IoU. By incorporating RDIoU into both the regression and classification branches, the network is encouraged to learn more precise bounding boxes and concurrently overcome the misalignment issue between classification and regression. Extensive experiments on the benchmark KITTI and Waymo Open Dataset validate that our RDIoU method can bring substantial improvement for the single-stage 3D object detection. The code is available at https://github.com/hlsheng1/RDIoU .

Accurate bounding-box regression with distance-IoU loss for visual tracking

Detecting oriented small objects is a critical task in remote sensing, but the development of high-performance deep learning-based detectors is hindered by the need for large-scale and well-annotated datasets. The high cost of creating these datasets, due to the dense and numerous distribution of small objects, significantly limits the application and development of such detectors. To address this problem, we propose a single-point-based annotation approach (SPA) based on the graph cut method. In this framework, user annotations act as the origin of positive sample points, and a similarity matrix, computed from feature maps extracted by deep learning networks, facilitates an intuitive and efficient annotation process for building graph elements. Utilizing the Maximum Flow algorithm, SPA derives positive sample regions from these points and generates oriented bounding boxes (OBBOXs). Experimental results demonstrate the effectiveness of SPA, with at least a 50% improvement in annotation efficiency. Furthermore, the intersection-over-union (IoU) metric of our OBBOX is 3.6% higher than existing methods such as the “Segment Anything Model”. When applied in training, the model annotated with SPA shows a 4.7% higher mean average precision (mAP) compared to models using traditional annotation methods. These results confirm the technical advantages and practical impact of SPA in advancing small object detection in remote sensing.

There is a growing demand for surveillance systems that can detect fall-down events because of the increased number of surveillance cameras being installed in many public indoor and outdoor locations. Fall-down event detection has been vigorously and extensively researched for safety purposes, particularly to monitor elderly peoples, patients, and toddlers. This computer vision detector has become more affordable with the development of high-speed computer networks and low-cost video cameras. This paper proposes moving object detection method based on human motion analysis for human fall-down events. The method comprises of three parts, which are preprocessing part to reduce image noises, motion detection part by using TV-L1 optical flow algorithm, and performance measure part. The last part will analyze the results of the object detection part in term of the bounding boxes, which are compared with the given ground truth. The proposed method is tested on Fall Down Detection (FDD) dataset and compared with Gunnar-Farneback optical flow by measuring intersection over union (IoU) of the output with respect to the ground truth bounding box. The experimental results show that the proposed method achieves an average IoU of 0.92524.

Object detection is a fundamental task in computer vision. Bounding box regression (BBR) losses are critical to detector performance. However, evaluation measures that rely on the Intersection over Union (IoU) between the predicted and ground truth boxes are highly sensitive to positional deviations, which can hinder optimization. To alleviate this issue, we propose ATDIoU, a novel arctangent-differential loss for bounding-box regression. ATDIoU computes distance similarity between a predicted and a ground truth box by modeling the distances between their corresponding vertices as a two-dimensional arctangent differential distribution (ATD). This arctangent differential-based design mitigates bounding box drift and reduces sensitivity to localization errors. As a result, it guides the model to learn target positions more effectively. We evaluate ATDIoU by integrating it into YOLOv6 and conducting experiments on PASCAL VOC and VisDrone2019. The results demonstrate that ATDIoU yields improvements of 1.4% and 0.7% in mean average precision (mAP) relative to MPDIoU.

In object detection algorithms, the bounding box regression (BBR) loss directly influences the model's accuracy in predicting object positions. Initially, we conducted simulation experiments on the proposed intersection over union-based loss. Through in-depth analysis, we discovered that the width and height components of the distance term could hinder BBR. To address this, we proposed the AIoU-v1 loss, which decouples the width and height components of the distance term, thereby preventing the suppression of BBR loss by the distance term. Additionally, issues such as the imbalance between sample quantity and sample quality in the dataset, as well as labeling errors, can adversely affect BBR. To tackle these dataset problems, we designed an adaptive dynamic non-monotonic focusing mechanism with strong robustness and wide applicability. Finally, we proposed a post-processing algorithm that combines fusion and non-maximum suppression, resulting in more accurate bounding boxes during the post-processing stage. Our source code and data are available at https://github.com/Wenhua-Zheng/AIOU.

<title>Abstract</title> <bold>Objectives</bold> Tooth numbering plays a significant role in dental care. The aim of this study was to evaluate the Mask Region-based Convolutional Neural Network (Mask R-CNN) model for tooth number detection, classification, and segmentation from smartphone photographs. <bold>Materials and Methods</bold> This retrospective study utilized the data collected in Finland during 2022. A total of 448 individuals, aged 13–78 years, participated in this study. Participants uploaded dental photographs taken by smartphone. The dataset consisted of upper and lower occlusal, right and left lateral and frontal views. Teeth were labelled using polygon annotation for the FDI World Dental Federation (FDI) notation. The final number of images and tooth polygons were 1,272 and 14,736 respectively. The Mask R-CNN model was used for tooth number recognition. Performance of trained models was measured based on the bounding box and class prediction for 28 tooth numbers, using PASCAL performance metric’s open toolset. The mean average precision (mAP) was calculated for each class with an intersection over union (IoU) threshold of 0.5. External validation was conducted using open Mendeley dataset from India, consisting of 40 preliminary selected images. <bold>Results</bold> The tooth number recognition model demonstrated an overall mAP of 88.2%, with a standard deviation of 0.0075 on 10-fold cross validation, which indicates stable performance of the model. The external validation showed excellent performance of the model, with a mAP of 84.1%. <bold>Conclusions</bold> The Mask R-CNN model developed in this study demonstrated excellent performance and good generalizability. Clinical Relevance: The developed mHealth system facilitates remote dental screening in real-world community settings.