DeforT: Deformable transformer for visual tracking

In object detection algorithms, the bounding box regression (BBR) loss directly influences the model's accuracy in predicting object positions. Initially, we conducted simulation experiments on the proposed intersection over union-based loss. Through in-depth analysis, we discovered that the width and height components of the distance term could hinder BBR. To address this, we proposed the AIoU-v1 loss, which decouples the width and height components of the distance term, thereby preventing the suppression of BBR loss by the distance term. Additionally, issues such as the imbalance between sample quantity and sample quality in the dataset, as well as labeling errors, can adversely affect BBR. To tackle these dataset problems, we designed an adaptive dynamic non-monotonic focusing mechanism with strong robustness and wide applicability. Finally, we proposed a post-processing algorithm that combines fusion and non-maximum suppression, resulting in more accurate bounding boxes during the post-processing stage. Our source code and data are available at https://github.com/Wenhua-Zheng/AIOU.

Object detection is a fundamental task in computer vision. Bounding box regression (BBR) losses are critical to detector performance. However, evaluation measures that rely on the Intersection over Union (IoU) between the predicted and ground truth boxes are highly sensitive to positional deviations, which can hinder optimization. To alleviate this issue, we propose ATDIoU, a novel arctangent-differential loss for bounding-box regression. ATDIoU computes distance similarity between a predicted and a ground truth box by modeling the distances between their corresponding vertices as a two-dimensional arctangent differential distribution (ATD). This arctangent differential-based design mitigates bounding box drift and reduces sensitivity to localization errors. As a result, it guides the model to learn target positions more effectively. We evaluate ATDIoU by integrating it into YOLOv6 and conducting experiments on PASCAL VOC and VisDrone2019. The results demonstrate that ATDIoU yields improvements of 1.4% and 0.7% in mean average precision (mAP) relative to MPDIoU.

Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$ by introducing a generalized version as both a new loss and a new metric. By incorporating this generalized $IoU$ ($GIoU$) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, $IoU$ based, and new, $GIoU$ based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.

Automatic spondylolisthesis grading from MRIs across modalities using faster adversarial recognition network.

Advanced Siamese visual object tracking architectures are jointly trained using pair-wise input images to perform target classification and bounding box regression. They have achieved promising results in recent benchmarks and competitions. However, the existing methods suffer from two limitations: First, though the Siamese structure can estimate the target state in an instance frame, provided the target appearance does not deviate too much from the template, the detection of the target in an image cannot be guaranteed in the presence of severe appearance variations. Second, despite the classification and regression tasks sharing the same output from the backbone network, their specific modules and loss functions are invariably designed independently, without promoting any interaction. Yet, in a general tracking task, the centre classification and bounding box regression tasks are collaboratively working to estimate the final target location. To address the above issues, it is essential to perform target-agnostic detection so as to promote cross-task interactions in a Siamese-based tracking framework. In this work, we endow a novel network with a target-agnostic object detection module to complement the direct target inference, and to avoid or minimise the misalignment of the key cues of potential template-instance matches. To unify the multi-task learning formulation, we develop a cross-task interaction module to ensure consistent supervision of the classification and regression branches, improving the synergy of different branches. To eliminate potential inconsistencies that may arise within a multi-task architecture, we assign adaptive labels, rather than fixed hard labels, to supervise the network training more effectively. The experimental results obtained on several benchmarks, i.e., OTB100, UAV123, VOT2018, VOT2019, and LaSOT, demonstrate the effectiveness of the advanced target detection module, as well as the cross-task interaction, exhibiting superior tracking performance as compared with the state-of-the-art tracking methods.

Surface defect detection is essential for ensuring the quality and safety of steel products. While Transformer-based methods have achieved state-of-the-art performance, they face several limitations, including high computational costs due to the quadratic complexity of the attention mechanism, inadequate detection accuracy for small-scale defects due to substantial downsampling, inconsistencies between classification scores and localization confidence, and feature resolution loss caused by simple upsampling and downsampling strategies. To address these challenges, we propose the HCT-Det model, which incorporates a window-based self-attention residual (WSA-R) block structure. This structure combines window-based self-attention (WSA) blocks to reduce computational overhead and parallel residual convolutional (Res) blocks to enhance local feature continuity. The model's backbone generates three cross-scale features as encoder inputs, which undergo Intra-Scale Feature Interaction (ISFI) and Cross-Scale Feature Interaction (CSFI) to improve detection accuracy for targets of various sizes. A Soft IoU-Aware mechanism ensures alignment between classification scores and intersection-over-union (IoU) metrics during training. Additionally, Hybrid Downsampling (HDownsample) and Hybrid Upsampling (HUpsample) modules minimize feature degradation. Our experiments demonstrate that HCT-Det achieved a mean average precision (mAP@0.5) of 0.795 on the NEU-DET dataset and 0.733 on the GC10-DET dataset, outperforming other state-of-the-art approaches. These results highlight the model's effectiveness in improving computational efficiency and detection accuracy for steel surface defect detection.

IMSiam: IoU-aware Matching-adaptive Siamese network for object tracking

PurposeApplication of a deep learning model visualization plugin for rapid and accurate automatic quantification and classification of lumbar disc herniation (LDH) types on axial T2-weighted MRIs.MethodsRetrospective analysis of 2500 patients, with the training set comprising data from 2120 patients (25,554 images), an internal test set covering data from 80 patients (784 images), and an external test set including data from 300 patients (3285 images). To enhance implementation, this study categorized normal and bulging discs as a grade without significant abnormalities, defining the region and severity grades of LDH based on the relationship between the disc and the spinal canal. The automated detection training and validation process employed the YOLOv8 object detection model for target area localization, the YOLOv8-seg segmentation model for disc recognition, and the YOLOv8-pose keypoint detection model for positioning. Finally, the stability of the detection results was verified using metrics such as Intersection over Union (IoU), mean error (ME), precision (P), F1 score (F1), Kappa coefficient (kappa), and 95% confidence interval (95%CI).ResultsThe segmentation model achieved an mAP50:95 of 98.12% and an IoU of 98.36% in the training set, while the keypoint detection model achieved an mAP50:95 of 93.58% with a mean error (ME) of 0.208 mm. For the internal and external test sets, the segmentation model’s IoU was 97.58 and 97.49%, respectively, while the keypoint model’s ME was 0.219 mm and 0.221 mm, respectively. In the quantification validation of the extent of LDH, P, F1, and kappa were measured. For LDH classification (18 categories), the internal and external test sets showed P = 81.21% and 74.50%, F1 = 81.26% and 74.42%, and kappa = 0.75 (95%CI 0.68, 0.82, p = 0.00) and 0.69 (95%CI 0.65, 0.73, p = 0.00), respectively. For the severity grades of LDH (four categories), the internal and external test sets showed P = 92.51% and 90.07%, F1 = 92.36% and 89.66%, and kappa = 0.88 (95%CI 0.80, 0.96, p = 0.00) and 0.85 (95%CI 0.81, 0.89, p = 0.00), respectively. For the regions of LDH (eight categories), the internal and external test sets showed P = 83.34% and 77.87%, F1 = 83.85% and 78.21%, and kappa = 0.77 (95%CI 0.70, 0.85, p = 0.00) and 0.71 (95%CI 0.67, 0.75, p = 0.00), respectively.ConclusionThe automated aided diagnostic model achieved high performance in detecting and classifying LDH and demonstrated substantial consistency with expert classification.

YOLOX-SwinT algorithm improves the accuracy of AO/OTA classification of intertrochanteric fractures by orthopedic trauma surgeons

Majority of the present day applications in computer vision use Convolutional neural networks (CNN) as the preferred engine of computation. For hardware acceleration of CNNs, z-first storage architectures like NVDLA have gained prominence which show better resource utilization and effective throughput for 3-D convolution operation. However, these architectures have low effective throughput for 2-D operations such as Depthwise Convolution (DWC) and Pooling due to poor resource utilization. The current trend in CNNs show that DWC has gained prominence in compact mobile-first CNNs like MnasNet, EfficientNet etc., which have classification accuracies comparable to state-of-the-art complex CNNs. In this work, an accelerator for accelerating 2-D operations based on z-first storage architectures is proposed. Proposed accelerator delivers power efficiency improvement of at least 1.46× and 1.89× for DWC and Pooling respectively over the baseline architecture, while improving effective throughput by 4×. Proposed IFM reuse scheme also reduces IFM reads from memory at least by a factor of 1.72×.

Individual measurement of traits of interest is of great importance for breeding and management decisions in animal production systems. However, measurements are often taken manually, which is laborious and also stressful for the animals. Therefore, the development of fast, precise and indirect measurement methods is paramount. An appealing way for such a task is through computer vision systems (CVS). Hence, the objectives of the current work were: 1) Devise a CVS for autonomous measurement of Nile tilapia body area, length, height, and eccentricity; and 2) Evaluation of linear models for prediction of body weight (BW). The pixels of 822 RGB images of live fish were labeled into background, fish fins or body using the “MTurk” crowdsourcing service. This dataset was then split into training (60% of data) and testing sets for the development of Deep Learning Networks for image segmentation into the three pixel categories. The networks differed in input image size (10 to 40% of original size) and number of encoder/decoder layer stacks (1 to 5). An independent dataset with 831 images was used for validation of the linear predictive models. The results for intersection over union (IoU) show that a network with input of 20% of the original size and 4 encoder/decoder stacks achieved the best results, with IoU on the test dataset of 99, 90 and 64% for background, fish body and fin, respectively (Figure 1). Predicted segmentation from this network on randomly selected images is shown in Figure 2. From the linear models evaluated, the one considering only area as input showed predicted R2 of 0.92 for fish BW (Table 1). In conclusion, the devised CVS was able to correctly separate fish body from background and fins, and a linear model using fish body area as input provided good predictive quality of Nile tilapia BW.

Multi-class classification of thyroid nodules from automatic segmented ultrasound images: Hybrid ResNet based UNet convolutional neural network approach

In this work, we present an algorithm for regression and classification tasks on big datasets using augmented tree models. Partitioning a big dataset using a tree model permits us to apply a divide and conquer strategy to classification and regression tasks. Experiments conducted as part of this study illustrate that such an approach has an important benefit. Methods associated with good accuracies on learning tasks on big datasets such as ensemble tree methods or neural networks produce models that are not interpretable. The models produced by the proposed algorithm are interpretable while being as accurate as ensemble methods such as random forests or gradient boosted trees. Model interpretation can be performed at coarse and fine granularity. This permits us to extract insights that characterize the entire dataset or a particular subset of the data. Models that are accurate and interpretable are highly desirable in many application settings. The partitions created by the algorithm also permit a divide and conquer approach to model analysis. Analysis of performance by partition helped identify problems such as possible data errors and model overfitting.

Transfer learning is a method for improving generalization performance by training a model for a different task first and then additionally training the pre-learned weights for the target task. However, transferability—the ease with which source task can be effectively transferred to which target task—is often unknown. Existing works proposed methods of measuring the transferability between classification tasks using images and discrete labels, but it cannot be applied to regression tasks. In this work, we investigate transferability among classification and regression tasks, and propose a method for predicting transferability by extending the optimal transport theory. Our transferability prediction model also can be applied to subjective tasks (e.g., aesthetics and memorability), which are usually regression tasks. We show that the appropriate source (pre-training) tasks can be predicted for the chosen target task without conducting actual pre-training and transferring trials. Experimental results demonstrated high prediction accuracy (correlation coefficient of ρ=0.791\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\rho =0.791$$\\end{document}) and a speed improvement of approximately 300 times compared with the above-mentioned greedy approach.

<sec>This study aims to predict the thermodynamic stability of rare-earth compounds by using machine learning (ML) models, providing crucial data support for designing advanced materials and facilitating the discovery of new rare-earth compounds.</sec><sec>In terms of methods, this study is based on a dataset consisting of 280569 compounds. The formation energies of these compounds are calculated by density functional theory (DFT). A system consisting of 145 feature descriptors is constructed, covering stoichiometric properties, statistical properties of elements, electronic structure properties, and properties of ionic compounds, comprehensively describing the characteristics of rare-earth compounds. Two ML models, i.e. random forest (RF) and neural network (NN), are selected to perform classification and regression tasks respectively. The 5-fold cross-validation is used to improve the reliability of the models. The min-max scaling technique is used for preprocessing data, and an ensemble learning architecture is constructed to address the limitations of single model.</sec><sec>In the classification task, the RF and NN algorithms perform remarkably well. With 5-fold cross-validation, the accuracy reaches approximately 0.97, and the F1 score is around 0.98, enabling the precise classification of compounds into stable or unstable categories. In the regression task, the mean absolute errors (MAEs) of the formation energy predictions by the RF and NN models are 0.055 eV/atom and 0.071 eV/atom, respectively. This indicates that the model predictions are highly accurate and can replace complete DFT calculations to a certain extent. In the predictive analysis of system outside the test set, six representative components are selected from the material project database, covering binary, ternary, and quaternary systems. The prediction errors of all compositions are controlled within 0.5 eV/atom, with an error percentage of lower than 25%, indicating that the model has strong ability of extrapolation and prediction. When predicting the binary phase diagrams of rare-earth compounds La-Al and Ce-H by using the trained models, the convex hull phase diagrams constructed through the ensemble learning architecture, which combines the prediction results of the RF and NN models, are highly consistent with those constructed from the open quantum materials database. The models successfully capture several metastable phases that are not present in multiple databases. Moreover, the convex hull distances of the predicted phases are mostly less than 0.1 eV/atom, with the maximum not exceeding 0.2 eV/atom.</sec><sec>In conclusion, this study successfully uses ML models to predict the thermodynamic stability of rare-earth compounds. The constructed models demonstrate strong capabilities in classification and regression tasks. The ensemble learning architecture effectively improves the model performance, providing a promising tool for discovering materials in the field of rare-earth science, contributing to the research and development of new rare-earth compounds, and designing advanced materials.</sec>