Accurate bounding-box regression with distance-IoU loss for visual tracking

Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$ by introducing a generalized version as both a new loss and a new metric. By incorporating this generalized $IoU$ ($GIoU$) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, $IoU$ based, and new, $GIoU$ based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.

Object detection is a challenging task in the field of remote sensing applications due to the complex backgrounds and uncertain orientation of targets. Compared with the horizontal bounding box, the oriented bounding box can provide orientation information while retaining the true size. Most existing oriented object detection methods are based on Faster-RCNN and the other one-stage methods that can achieve real-time speed but have shortcomings in localization and detection accuracy. To further enhance the performance of one-stage methods, we propose an oriented object detection framework that is based on the single shot detector, namely, single shot anchor refinement network (S²ARN). The S²ARN obtains the accurate detection results by performing two consecutive regressions. More precisely, the multilevel features of the backbone are used to regress the coordinate offsets between the predefined rotated anchors and the ground-truth boxes to generate the refined anchors. The classification and regression subnetworks assigned to the output features are used to perform the second regression to determine the class labels and further adjust the location of the refined anchors. In addition, receptive field amplification modules (RFAMs) are inserted to enlarge the receptive field and extract more discriminative features. Furthermore, in the anchor matching step, angle-related Intersection over Union (ArIoU) is used to calculate the Intersection over Union (IoU) score instead of the traditional method. Benefiting from the multiple regressions and the insensitivity of the ArIoU score to the angle deviation, the angle sampling interval of the rotated anchor can be reduced. The experimental results for the two public datasets, HRSC2016 and UCAS-AOD, demonstrate the effectiveness of the proposed network.

The performance of visual object tracking depends largely on the target appearance model. Benefited from the success of CNN in feature extraction, recent studies have paid much attention to CNN representation learning and feature fusion model. However, the existing feature fusion models ignore the relation between the features of different layers. In this paper, we propose a deep feature fusion model based on the siamese network by considering the connection between feature maps of CNN. To tackle the limitation of different feature map sizes in CNN, we propose to fuse different resolution feature maps by introducing de-convolutional layers in the offline training stage. Specifically, a top-down modulation is adopted for feature fusion. In the tracking stage, a simple matching operation between the fused feature of the examplar and search region is conducted with the learned model, which can maintain the real-time tracking speed. Experimental results show that, the proposed method obtains favorable tracking accuracy against the state-of-the-art trackers with a real-time tracking speed.

In object detection algorithms, the bounding box regression (BBR) loss directly influences the model's accuracy in predicting object positions. Initially, we conducted simulation experiments on the proposed intersection over union-based loss. Through in-depth analysis, we discovered that the width and height components of the distance term could hinder BBR. To address this, we proposed the AIoU-v1 loss, which decouples the width and height components of the distance term, thereby preventing the suppression of BBR loss by the distance term. Additionally, issues such as the imbalance between sample quantity and sample quality in the dataset, as well as labeling errors, can adversely affect BBR. To tackle these dataset problems, we designed an adaptive dynamic non-monotonic focusing mechanism with strong robustness and wide applicability. Finally, we proposed a post-processing algorithm that combines fusion and non-maximum suppression, resulting in more accurate bounding boxes during the post-processing stage. Our source code and data are available at https://github.com/Wenhua-Zheng/AIOU.

YOLO-YCbCr based smoke segmentation method for fire detection was proposed in this study to enhance smoke segmentation performance with reduction in computational cost. The proposed method consists of the deep learning object detection model, You Only Look Once (YOLO), and the color space-based segmentation model, rule-based YCbCr. YOLO is used to detect smoke objects in the input fire images through bounding boxes. The images in the bounding boxes are characterized based on YCbCr color space and the proposed YCbCr rules derived by considering smoke image characteristics segment smoke region. In order to evaluate the smoke segmentation performance, six different fire incident video data were used. The mean intersection over union (IoU) value of the proposed method was improved by approximately 31.3% and 24.5% respectively when compared to the reference models: YOLO-RGB model and YOLO-CIELAB model. It was found that YOLO significantly reduced the erroneous smoke segmentation and YCbCr rules derived from the smoke color features were effective in the smoke segmentation.

Accurate object detection requires correct classification and high-quality localization. Currently, most of the single shot detectors (SSDs) conduct simultaneous classification and regression using a fully convolutional network. Despite high efficiency, this structure has some inappropriate designs for accurate object detection. The first one is the mismatch of bounding box classification, where the classification results of the default bounding boxes are improperly treated as the results of the regressed bounding boxes during the inference. The second one is that only one-time regression is not good enough for high-quality object localization. To solve the problem of classification mismatch, we propose a novel reg-offset-cls (ROC) module including three hierarchical steps: the regression of the default bounding box, the prediction of new feature sampling locations, and the classification of the regressed bounding box with more accurate features. For high-quality localization, we stack two ROC modules together. The input of the second ROC module is the output of the first ROC module. In addition, we inject a feature enhanced (FE) module between two stacked ROC modules to extract more contextual information. The experiments on three different datasets (i.e., MS COCO, PASCAL VOC, and UAVDT) are performed to demonstrate the effectiveness and superiority of our method. Without any bells or whistles, our proposed method outperforms state-of-the-art one-stage methods at a real-time speed. The source code is available at https://github.com/JialeCao001/HSD.

Object detection is a fundamental task in computer vision. Bounding box regression (BBR) losses are critical to detector performance. However, evaluation measures that rely on the Intersection over Union (IoU) between the predicted and ground truth boxes are highly sensitive to positional deviations, which can hinder optimization. To alleviate this issue, we propose ATDIoU, a novel arctangent-differential loss for bounding-box regression. ATDIoU computes distance similarity between a predicted and a ground truth box by modeling the distances between their corresponding vertices as a two-dimensional arctangent differential distribution (ATD). This arctangent differential-based design mitigates bounding box drift and reduces sensitivity to localization errors. As a result, it guides the model to learn target positions more effectively. We evaluate ATDIoU by integrating it into YOLOv6 and conducting experiments on PASCAL VOC and VisDrone2019. The results demonstrate that ATDIoU yields improvements of 1.4% and 0.7% in mean average precision (mAP) relative to MPDIoU.

Siamese network-based visual tracking algorithms have achieved excellent performance in recent years, but challenges such as fast target motion, shape and scale variations have made the tracking extremely difficult. The regression of anchor-free tracking has low computational complexity, strong real-time performance, and is suitable for visual tracking. Based on the anchor-free siamese tracking framework, this paper firstly introduces balance factors and modulation coefficients into the cross-entropy loss function to solve the classification inaccuracy caused by the imbalance between positive and negative samples as well as the imbalance between hard and easy samples during the training process, so that the model focuses more on the positive samples and the hard samples that make the major contribution to the training. Secondly, the intersection over union (IoU) loss function of the regression branch is improved, not only focusing on the IoU between the predicted box and the ground truth box, but also considering the aspect ratios of the two boxes and the minimum bounding box area that accommodate the two, which guides the generation of more accurate regression offsets. The overall loss of classification and regression is iteratively minimized and improves the accuracy and robustness of visual tracking. Experiments on four public datasets, OTB2015, VOT2016, UAV123 and GOT-10k, show that the proposed algorithm achieves the state-of-the-art performance.

Detecting oriented small objects is a critical task in remote sensing, but the development of high-performance deep learning-based detectors is hindered by the need for large-scale and well-annotated datasets. The high cost of creating these datasets, due to the dense and numerous distribution of small objects, significantly limits the application and development of such detectors. To address this problem, we propose a single-point-based annotation approach (SPA) based on the graph cut method. In this framework, user annotations act as the origin of positive sample points, and a similarity matrix, computed from feature maps extracted by deep learning networks, facilitates an intuitive and efficient annotation process for building graph elements. Utilizing the Maximum Flow algorithm, SPA derives positive sample regions from these points and generates oriented bounding boxes (OBBOXs). Experimental results demonstrate the effectiveness of SPA, with at least a 50% improvement in annotation efficiency. Furthermore, the intersection-over-union (IoU) metric of our OBBOX is 3.6% higher than existing methods such as the “Segment Anything Model”. When applied in training, the model annotated with SPA shows a 4.7% higher mean average precision (mAP) compared to models using traditional annotation methods. These results confirm the technical advantages and practical impact of SPA in advancing small object detection in remote sensing.

Currently, reliable and accurate oriented detection in remote sensing images still needs to be improved. The wide variation of object shapes and orientations in the remote sensing images usually leads to two issues in two-stage oriented object detectors. One issue is how to generate high-quality rotation proposals. The other is the angular error sensitivity to the aspect ratios in the angle optimization process. In this paper, we propose a novel rotation proposal generation and optimization detector, which is based on high-quality rotation proposal generation and adaptive angle optimization to solve these two issues. The proposed method mainly establishes the geometric relationship guided region proposal networks (GRG-RPN) and the adaptive angle optimization head (AAO-Head) to achieve more accurate oriented object detection. The GRG-RPN only uses a simple network and a small number of horizontal anchors to predict high-quality rotation proposals. This approach was derived via the calculation based on the theoretical analysis of the geometric relationship between the oriented bounding boxes (OBBs) and their external horizontal bounding boxes (EHBBs). The AAO-Head solves the angular error sensitivity to the aspect ratios and achieves adaptive angle optimization using a new regression parameter, which is defined based on the theoretical analysis of the relationship between the Intersection over Union (IoU), the angular errors and the aspect ratios. Experiments show that our method can achieve 2.5% mAP improvement averagely versus the compared SOTA methods, and achieve 0.5% mAP improvement versus the next best method Oriented R-CNN with fewer regression parameters and a simpler regression approach.

Automatic spondylolisthesis grading from MRIs across modalities using faster adversarial recognition network.

Visual tracking of generic objects is one of the fundamental but challenging problems in computer vision. Here, we propose a novel fully convolutional Siamese network to solve visual tracking by directly predicting the target bounding box in an end-to-end manner. We first reformulate the visual tracking task as two subproblems: a classification problem for pixel category prediction and a regression task for object status estimation at this pixel. With this decomposition, we design a simple yet effective Siamese architecture based classification and regression framework, termed SiamCAR, which consists of two subnetworks: a Siamese subnetwork for feature extraction and a classification-regression subnetwork for direct bounding box prediction. Since the proposed framework is both proposal- and anchor-free, SiamCAR can avoid the tedious hyper-parameter tuning of anchors, considerably simplifying the training. To demonstrate that a much simpler tracking framework can achieve superior tracking results, we conduct extensive experiments and comparisons with state-of-the-art trackers on a few challenging benchmarks. Without bells and whistles, SiamCAR achieves leading performance with a real-time speed. Furthermore, the ablation study validates that the proposed framework is effective with various backbone networks, and can benefit from deeper networks. Code is available at https://github.com/ohhhyeahhh/SiamCAR.

There is a growing demand for surveillance systems that can detect fall-down events because of the increased number of surveillance cameras being installed in many public indoor and outdoor locations. Fall-down event detection has been vigorously and extensively researched for safety purposes, particularly to monitor elderly peoples, patients, and toddlers. This computer vision detector has become more affordable with the development of high-speed computer networks and low-cost video cameras. This paper proposes moving object detection method based on human motion analysis for human fall-down events. The method comprises of three parts, which are preprocessing part to reduce image noises, motion detection part by using TV-L1 optical flow algorithm, and performance measure part. The last part will analyze the results of the object detection part in term of the bounding boxes, which are compared with the given ground truth. The proposed method is tested on Fall Down Detection (FDD) dataset and compared with Gunnar-Farneback optical flow by measuring intersection over union (IoU) of the output with respect to the ground truth bounding box. The experimental results show that the proposed method achieves an average IoU of 0.92524.

<title>Abstract</title> <bold>Objectives</bold> Tooth numbering plays a significant role in dental care. The aim of this study was to evaluate the Mask Region-based Convolutional Neural Network (Mask R-CNN) model for tooth number detection, classification, and segmentation from smartphone photographs. <bold>Materials and Methods</bold> This retrospective study utilized the data collected in Finland during 2022. A total of 448 individuals, aged 13–78 years, participated in this study. Participants uploaded dental photographs taken by smartphone. The dataset consisted of upper and lower occlusal, right and left lateral and frontal views. Teeth were labelled using polygon annotation for the FDI World Dental Federation (FDI) notation. The final number of images and tooth polygons were 1,272 and 14,736 respectively. The Mask R-CNN model was used for tooth number recognition. Performance of trained models was measured based on the bounding box and class prediction for 28 tooth numbers, using PASCAL performance metric’s open toolset. The mean average precision (mAP) was calculated for each class with an intersection over union (IoU) threshold of 0.5. External validation was conducted using open Mendeley dataset from India, consisting of 40 preliminary selected images. <bold>Results</bold> The tooth number recognition model demonstrated an overall mAP of 88.2%, with a standard deviation of 0.0075 on 10-fold cross validation, which indicates stable performance of the model. The external validation showed excellent performance of the model, with a mAP of 84.1%. <bold>Conclusions</bold> The Mask R-CNN model developed in this study demonstrated excellent performance and good generalizability. Clinical Relevance: The developed mHealth system facilitates remote dental screening in real-world community settings.

Lumen segmentation using a Mask R-CNN in carotid arteries with stenotic atherosclerotic plaque