Automatic Object Tracking and Segmentation Using Unsupervised SiamMask

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

Object detection and tracking has always been one of the important research directions in computer vision. The purpose is to determine whether the object is contained in the input image and enclose the object with a bounding box. However, most object detection and tracking methods are applied to daytime objects, and the processing of nighttime objects is imprecise. In this paper, a spectral-spatial feature enhancement algorithm for nighttime object detection and tracking is proposed, which is inspired by symmetrical neural networks. The proposed method consists of the following steps. First, preprocessing is performed on unlabeled nighttime images, including low-light enhancement, object detection, and dynamic programming. Second, object features for daytime and nighttime times are extracted and modulated with a domain-adaptive structure. Third, the Siamese network can make full use of daytime and nighttime object features, which is trained as a tracker by the above images. Fourth, the test set is subjected to feature enhancement and then input to the tracker to obtain the final detection and tracking results. The feature enhancement step includes low-light enhancement and Gabor filtering. The spatial-spectral features of the target are fully extracted in this step. The NAT2021 dataset is used in the experiments. Six methods are employed as comparisons. Multiple judgment indicators were used to analyze the research results. The experimental results show that the method achieves excellent detection and tracking performance.

The use of unmanned aerial vehicles (UAVs) that can operate autonomously in dynamic and dangerous operational environments are becoming increasingly common. In such operations, object detection, classification and tracking can often be one of the main goals. In recent years there has been an increased focus on embedded hardware that is both small and powerful, making UAV on-board data processing more viable. Being able to process the video feed on-board the UAV calls for fast and robust real-time algorithms for object identification and tracking. This paper discusses the development and implementation of a machine vision system for a low-cost fixed-wing UAV with a total flying weight of less than 4kg. The machine vision system incorporates the use of a thermal imaging camera and on-board processing power to perform real-time object detection, classification and tracking of objects in the ocean surface. The system is tested on thermal video data from a test flight, and is found to be able to detect 99;6% of objects of interest located in the ocean surface. Of the detected objects, only 5% were false positives. Furthermore, it classifies 93; 3% of the object types it is trained to classify correctly. The classifier is highly agile, allowing the user to quickly define which object characteristics that should be considered during classification, and what types of objects to classify. Finally, the system is found to successfully track 85% of the object types it is actively searching for in a real-time simulation test.

As an important part of intelligent vehicle, environmental perception system mainly refers to the detection of the surrounding environment of the vehicle by the sensors attached on the vehicle. In order to ensure the accuracy and stability of the intelligent vehicle environmental perception system, it is necessary to use intelligent vehicle sensors to detect and track objects in the passable area. In this paper, an object detection and tracking algorithm based on the LiDAR and camera information fusion is proposed. The algorithm uses the point cloud data clustering method of LiDAR to detect the objects in the passable area and project them onto the image to determine the tracking objects. After the objects are determined, the algorithm uses color information to track objects in the image sequence. Since the object tracking algorithm based on image is easily affected by light, shadow and background interference, the algorithm uses LiDAR point cloud to modify the tracking results. This paper uses KITTI data set to verify and test this algorithm and experiments show that the target area detection overlap of the proposed target detection and tracking algorithm is 83.10% on average and the tracking success rate is 80.57%. Compared with particle filtering algorithm, the average region overlap increased by 29.47% and the tracking success rate increased by 19.96%.

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.

A review of object tracking methods: From general field to autonomous vehicles

Automatic object detection and tracking has been widely applied in the video surveillance systems for homeland security and data fusion in the remote sensing and airborne imagery. The typical applications include human motion analysis and the vehicle detection. Here we implement object detection and tracking under shape graphs of interesting objects integrating local contextual information (corner/point features, etc) of the objects. On the top layer, shapes/sketches provide a discrimination measure to describe the global status of the interesting objects. This kind of information is very useful to improve the object tracking performance for occlusion. The shape can be modeled as a graph or hyper graph through its local geometric features. On the bottom layer, local geometric features are used to capture local properties of objects and perform correspondence estimation of high-level shapes. The local features provide a way to conquer inaccurate object segmentation and extraction. The experiments were implemented on human face tracking and vehicle detection and tracking.

Most conventional object tracking algorithms are implemented on general-purpose processors in software due to its great flexibility. However, the real-time performance is hard to achieve due to the inherent characteristics of the sequential processing of these processors. To tackle this issue, a reconfigurable system-on-chip (rSoC) platform with microprocessors and FPGAs is applied in this paper. To simplify the hardware/software interface, a Belief–Desire–Intention (BDI)-based multi-agent architecture is proposed as the unified framework. Then an agent-based task graph and two heuristic partitioning methods are proposed to partition the hardware and software on an rSoC platform. Compared to the module-based architecture, this BDI-based multi-agent architecture provides more efficiency, flexibility, autonomy, and scalability for the real-time tracking systems. A particle swarm optimization (PSO)-based object detection and tracking algorithm is applied to evaluate the proposed architecture. Extensive experimental results of object tracking demonstrate that the proposed architecture is efficient and highly robust with real-time performance.

The paper presents review of key aspects associated with object detection and tracking algorithms in sensing systems for automotive applications covering active safety, advanced driver assistance and autonomous driving systems. The role of discussed systems is to establish location and monitor relative movement of objects and environment with main focus on identification of vulnerable road users. The object detection algorithms allow to classify and differentiate both static and dynamic objects in the vehicle’s surrounding. The analysis will be focused on selection of most suitable platform for development of optimized and safe vulnerable road user detection system.

Object tracking in video is a challenging problem in several applications such as video surveillance, video compression, video retrieval, and video editing. Tracking an object in a video is not easy due to loss of information caused by illumination changing in a scene, occlusions with other objects, similar target appearances, and inaccurate tracker responses. In this paper, we present a novel object detection and tracking algorithm via structured output prediction classifier. Given an initial bounding box with its position, we first divide it into sub-blocks with a predefined size. Next, we extract the features from each sub-blocks with Haar-like features method. And then we learn those features with a structured output prediction classifier. We treat the sub-blocks obtained from the initial bounding box as positive samples and then randomly choose negative samples from search windows defined by the specific area around the bounding box. After that, we obtain prediction scores for each sub-blocks both from positive and negative samples. We construct a region-graph with sub-blocks as nodes and classifier’s score as weight to detect the target object in each frame. Our experimental results show that the proposed method outperforms state-of-the-art object tracking algorithms.KeywordsObject DetectionObject TrackingSupport Vector Machine

Object detection and tracking under complex environment are challenging because of the disturbances induced by background clutter, illumination changes, occlusions and other factors. The bulk of traditional algorithms basically rely on hand‐crafted features, which are not sufficiently robust to a complex environment. Moreover, the processes of detection and tracking are separated, which leads to the overall efficiency not high. In this study, a novel local probability model (LPM)‐based mean shift (MS) algorithm is proposed to integrate object detection and tracking. The main contributions include: (i) a new framework based on the combination of LPM and MS is established for the integration of object tracking and detection. (ii) For object detection, the training and prediction of LPM are built by stacked denoising autoencoders based deep learning. (iii) For object tracking, an MS tracking algorithm leveraging LPM is modified to improve the tracking efficiency under a complex environment. Experimental results demonstrate that the proposed method is superior to the colour histograms based MS and histograms of oriented gradients based MS in terms of robustness and tracking accuracy.

The rapid advancement in the field of deep learning and high performance computing has highly augmented the scope of video-based vehicle counting system. In this paper, the authors deploy several state-of-the-art object detection and tracking algorithms to detect and track different classes of vehicles in their regions of interest (ROI). The goal of correctly detecting and tracking vehicles’ in their ROI is to obtain an accurate vehicle count. Multiple combinations of object detection models coupled with different tracking systems are applied to access the best vehicle counting framework. The models’ addresses challenges associated to different weather conditions, occlusion and low-light settings and efficiently extracts vehicle information and trajectories through its computationally rich training and feedback cycles. The automatic vehicle counts resulting from all the model combinations are validated and compared against the manually counted ground truths of over 9 h’ traffic video data obtained from the Louisiana Department of Transportation and Development. Experimental results demonstrate that the combination of CenterNet and Deep SORT, and YOLOv4 and Deep SORT produced the best overall counting percentage for all vehicles.

Automatic segmentation and tracking of biological objects from dynamic microscopy data is of great interest for quantitative biology. A successful framework for this task are active contours, curves that iteratively minimize a cost function, which contains both data-attachment terms and regularization constraints reflecting prior knowledge on the contour geometry. However the choice of these latter terms and of their weights is largely arbitrary, thus requiring time-consuming empirical parameter tuning and leading to sub-optimal results. Here, we report on a first attempt to use regularization terms based on known biophysical properties of cellular membranes. The present study is restricted to 2D images and cells with a simple to skeletal cortex underlying the membrane. We describe our new active contour model and its implementation, and show a first application to real biological images. The obtained segmentation is slightly better than standard active contours, however the main advantage lies in the self-consistent and automated determination of the weights of regularization terms. This encouraging result will lead us to extend the approach to 3D and more complex cells.

Object tracking is an important basis for the autonomous navigation of unmanned surface vehicles. However, several problems still must be addressed for a wide applicating of object tracking in unmanned surface vehicles. First, if multiple objects of the same classification exist in the same field of view, then stable extraction of an object is difficult. Second, in an environment with a complex background and large changes in object shape, the tracking accuracy is low, and object tracking errors and tracking loss can easily occur. Third, much time is required to detect a high-resolution real-time video stream, not meeting the delay requirement of the photoelectric servo stable tracking. To resolve these problems, this paper proposes an object detection-tracking algorithm based on a radar-photoelectric system. The algorithm combines an object detection algorithm with an object tracking algorithm and involves the following steps. First, a first-frame object extraction algorithm is used to extract the tracking object from the first frame. Second, a region of interest (ROI)-prediction algorithm is used to predict ROIs and detect objects in these ROIs. This algorithm can effectively solve the above problems in marine tests. When multiple objects of the same classification exist in the same field of view, the algorithm can extract the radar-guided object stably. When faced with a complex background and a large change in object shape, the algorithm substantially improves the accuracy and robustness of object tracking. Compared with the conventional object detection algorithm, the time consumption of this algorithm is reduced by 25.8%.