Siamese Network for Underwater Multiple Object Tracking

Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT

Purpose:The integration of a surgical robotic instrument tracking module within optical microscopes holds the potential to advance microsurgery practices, as it facilitates automated camera movements, thereby augmenting the surgeon’s capability in executing surgical procedures.Methods:In the present work, an innovative detection backbone based on spatial attention module is implemented to enhance the detection accuracy of small objects within the image. Additionally, we have introduced a robust data association technique, capable to re-track surgical instrument, mainly based on the knowledge of the dual-instrument robotics system, Intersection over Union metric and Kalman filter.Results:The effectiveness of this pipeline was evaluated through testing on a dataset comprising ten manually annotated videos of anastomosis procedures involving either animal or phantom vessels, exploiting the Symani®Surgical System—a dedicated robotic platform designed for microsurgery. The multiple object tracking precision (MOTP) and the multiple object tracking accuracy (MOTA) are used to evaluate the performance of the proposed approach, and a new metric is computed to demonstrate the efficacy in stabilizing the tracking result along the video frames. An average MOTP of 74±0.06% and a MOTA of 99±0.03% over the test videos were found.Conclusion:These results confirm the potential of the proposed approach in enhancing precision and reliability in microsurgical instrument tracking. Thus, the integration of attention mechanisms and a tailored data association module could be a solid base for automatizing the motion of optical microscopes.

Analysis-by-synthesis: Pedestrian tracking with crowd simulation models in a multi-camera video network

Aerial imagery surveys are commonly used in marine mammal research to determine population size, distribution and habitat use. Analysis of aerial photos involves hours of manually identifying individuals present in each image and converting raw counts into useable biological statistics. Our research proposes the use of deep learning algorithms to increase the efficiency of the marine mammal research workflow. To test the feasibility of this proposal, the existing YOLOv4 convolutional neural network model was trained to detect belugas, kayaks and motorized boats in oblique drone imagery, collected from a stationary tethered system. Automated computer-based object detection achieved the following precision and recall, respectively, for each class: beluga = 74%/72%; boat = 97%/99%; and kayak = 96%/96%. We then tested the performance of computer vision tracking of belugas and occupied watercraft in drone videos using the DeepSORT tracking algorithm, which achieved a multiple-object tracking accuracy (MOTA) ranging from 37% to 88% and multiple object tracking precision (MOTP) between 63% and 86%. Results from this research indicate that deep learning technology can detect and track features more consistently than human annotators, allowing for larger datasets to be processed within a fraction of the time while avoiding discrepancies introduced by labeling fatigue or multiple human annotators.

We present a novel adaptive host-chip modular architecture for video acquisition to optimize an overall objective task constrained under a given bit rate. The chip is a high resolution imaging sensor such as gigapixel focal plane array (FPA) with low computational power deployed on the field remotely, while the host is a server with high computational power. The communication channel data bandwidth between the chip and host is constrained to accommodate transfer of all captured data from the chip. The host performs objective task specific computations and also intelligently guides the chip to optimize (compress) the data sent to host. This proposed system is modular and highly versatile in terms of flexibility in re-orienting the objective task. In this work, object tracking is the objective task. While our architecture supports any form of compression/distortion, in this paper we use quadtree (QT)-segmented video frames. We use Viterbi (Dynamic Programming) algorithm to minimize the area normalized weighted rate-distortion allocation of resources. The host receives only these degraded frames for analysis. An object detector is used to detect objects, and a Kalman Filter based tracker is used to track those objects. Evaluation of system performance is done in terms of Multiple Object Tracking Accuracy (MOTA) metric. In this proposed novel architecture, performance gains in MOTA is obtained by twice training the object detector with different system generated distortions as a novel 2-step process. Additionally, object detector is assisted by tracker to upscore the region proposals in the detector to further improve the performance.

Everyone “knows” that compressing a video will degrade the accuracy of object tracking. Yet, a literature search on this topic reveals that there is very little documented evidence for this presumed fact. Part of the reason is that, until recently, there were no object tracking datasets for uncompressed video, which made studying the effects of compression on tracking accuracy difficult. In this paper, using a recently published dataset that contains tracking annotations for uncompressed videos, we examined the degradation of tracking accuracy due to video compression using rigorous statistical methods. Specifically, we examined the impact of quantization parameter (QP) and motion search range (MSR) on Multiple Object Tracking Accuracy (MOTA). The results show that QP impacts MOTA at the 95% confidence level, while there is insufficient evidence to claim that MSR impacts MOTA. Moreover, regression analysis allows us to derive a quantitative relationship between MOTA and QP for the specific tracker used in the experiments.

Multiple objects tracking is a challenging task. This article presents an algorithm which can detect and track multiple objects, and update target model automatically. The contributions of this paper as follow: Firstly,we also use color histogram(CH) and histogram of orientated gradients(HOG) to represent the objects, model update is realized by kalman filter and gaussian model; secondly we use Gaussian Mixture Model(GMM) and Bhattacharyya distance to detect object appearance. Particle filter with combined features and model update mechanism can improve tracking results. Experiments on video sequences demonstrate that the method presented in this paper can realize multiple objects detection and tracking.

The paper proposes a novel approach to the objects localization in the working area of a modular reconfigurable robot (MRR), which implies the installation of stationary monitoring points (SMP), consisting of detachable robot’s modules and in- stalled by robot itself. This approach is based on the architecture of the MRR control system previously proposed by the authors and a new method for comparing information about the speed and position obtained from various sensors. The key steps of the approach are following. Upon arriving in the target area, the MRR places SMPs, which consist of a power source, a computing device, a wireless transceiver and a sensor, detached from the robot. Then SMPs monitor the working area using different types of sensors (cameras, rangefinders, etc.), perform segmentation of the measured data and transfer this information to the robot. Further a sensor fusion is performed using a novel object tracking method, which makes it possible to localize target objects even in those cases when they are not visible by some of the SMPs. One of the key advantages of the new approach is a possibility of implementation in the distributed architecture of a MRR. The simulation results show that proposed method has Multiple Object Tracking Accuracy (MOTA) metric of 86 %, which is higher than the most of its analogues, while the estimated dynamic object localization error in a 8x7 m working area using 2 cameras and 1 rangefinder does not exceed 10 cm.

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed.

This paper introduces a joint learning architecture (JLA) for multiple object tracking (MOT) and multiple object forecasting (MOF) in which the goal is to predict tracked objects’ current and future locations simultaneously. MOF is a recent formulation of trajectory forecasting where the full object bounding boxes are predicted rather than trajectories alone. Existing works separate multiple object tracking and multiple object forecasting. Such an approach can propagate errors in tracking to forecasting. We propose a joint learning architecture for multiple object tracking and forecasting (MOTF). Our approach reduces the chances of propagating tracking errors to the forecasting module. In addition, we show, through a new data association step, that forecasting predictions can be used for tracking objects during occlusion. We adapt an existing MOT method to simultaneously predict current and future object locations and confirm that JLA benefits both the MOT and MOF tasks.

In this study, we propose a multiple vehicle tracking method using multiple hypotheses and the appearance model. The multiple hypotheses are associated with multiple tracks using track-to-multiple hypotheses association method. A target state is estimated using the maximum a posteriori probability estimation method. The posterior probability is proportional to the product of a priori probability and the likelihood that is calculated using similarities of multiple hypotheses and the appearance model. The posterior probability density function is estimated using the Markov chain Monte Carlo particle filter. An optimal posterior target state is determined using a sample with the maximum a posteriori probability. Our experimental results show that the proposed method can improve multiple objects tracking precision as well as multiple object tracking accuracy.

In the tracking‐by‐detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking‐by‐detection paradigm. In this study, object detection accuracy has been increased by employing a faster region‐based convolutional neural network (Faster R‐CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter‐based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow‐based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R‐CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state‐of‐the‐art tracking methods demonstrate competitive tracking performance of the proposed MOT framework.

By associating the positions and features of object detection results, tracking by detection framework have performed well in multiple-object tracking (MOT) task. But models of this framework focus on using the consistency of objects' bounding-box and appearance to improve data association ability, ignoring the detection model could also be optimized by temporal information. In this article, we propose a novel MOT model named PreTracker. It associates the heatmaps of the targets in the previous frames to predict the heatmap of the targets in the current frame with an additional heatmap prediction module (HPM) during detection. After that, it combines the predicted heatmap with semantic features to enhance the detection capability, thus improving the tracking performance. After validation on the MOT17 validation dataset, our model obtains a 2.1 % improvement in multiple object tracking accuracy (MOTA) and a 1.7% improvement in ID F1 score (IDF1) based on FairMOT.

Object detection and tracking has always been one of the important research directions in computer vision. The purpose is to determine whether the object is contained in the input image and enclose the object with a bounding box. However, most object detection and tracking methods are applied to daytime objects, and the processing of nighttime objects is imprecise. In this paper, a spectral-spatial feature enhancement algorithm for nighttime object detection and tracking is proposed, which is inspired by symmetrical neural networks. The proposed method consists of the following steps. First, preprocessing is performed on unlabeled nighttime images, including low-light enhancement, object detection, and dynamic programming. Second, object features for daytime and nighttime times are extracted and modulated with a domain-adaptive structure. Third, the Siamese network can make full use of daytime and nighttime object features, which is trained as a tracker by the above images. Fourth, the test set is subjected to feature enhancement and then input to the tracker to obtain the final detection and tracking results. The feature enhancement step includes low-light enhancement and Gabor filtering. The spatial-spectral features of the target are fully extracted in this step. The NAT2021 dataset is used in the experiments. Six methods are employed as comparisons. Multiple judgment indicators were used to analyze the research results. The experimental results show that the method achieves excellent detection and tracking performance.

A robust method for multi object tracking in autonomous ship navigation systems