Agent-based reconfigurable architecture for real-time object tracking

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

Object detection and tracking has always been one of the important research directions in computer vision. The purpose is to determine whether the object is contained in the input image and enclose the object with a bounding box. However, most object detection and tracking methods are applied to daytime objects, and the processing of nighttime objects is imprecise. In this paper, a spectral-spatial feature enhancement algorithm for nighttime object detection and tracking is proposed, which is inspired by symmetrical neural networks. The proposed method consists of the following steps. First, preprocessing is performed on unlabeled nighttime images, including low-light enhancement, object detection, and dynamic programming. Second, object features for daytime and nighttime times are extracted and modulated with a domain-adaptive structure. Third, the Siamese network can make full use of daytime and nighttime object features, which is trained as a tracker by the above images. Fourth, the test set is subjected to feature enhancement and then input to the tracker to obtain the final detection and tracking results. The feature enhancement step includes low-light enhancement and Gabor filtering. The spatial-spectral features of the target are fully extracted in this step. The NAT2021 dataset is used in the experiments. Six methods are employed as comparisons. Multiple judgment indicators were used to analyze the research results. The experimental results show that the method achieves excellent detection and tracking performance.

A review of object tracking methods: From general field to autonomous vehicles

As an important part of intelligent vehicle, environmental perception system mainly refers to the detection of the surrounding environment of the vehicle by the sensors attached on the vehicle. In order to ensure the accuracy and stability of the intelligent vehicle environmental perception system, it is necessary to use intelligent vehicle sensors to detect and track objects in the passable area. In this paper, an object detection and tracking algorithm based on the LiDAR and camera information fusion is proposed. The algorithm uses the point cloud data clustering method of LiDAR to detect the objects in the passable area and project them onto the image to determine the tracking objects. After the objects are determined, the algorithm uses color information to track objects in the image sequence. Since the object tracking algorithm based on image is easily affected by light, shadow and background interference, the algorithm uses LiDAR point cloud to modify the tracking results. This paper uses KITTI data set to verify and test this algorithm and experiments show that the target area detection overlap of the proposed target detection and tracking algorithm is 83.10% on average and the tracking success rate is 80.57%. Compared with particle filtering algorithm, the average region overlap increased by 29.47% and the tracking success rate increased by 19.96%.

Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT

Event cameras utilize novel imaging sensors patterned after visual pathways in the brain that are responsive to low contrast, transient events. Specifically, the pixels of dynamic vision sensors (DVS) react independently and asynchronously to changes in light intensity, creating a stream of time-stamped events encoding the pixels’ (x, y) location in the sensor array and the sign of the brightness change. In contrast with conventional cameras that sample every pixel at a fixed rate, DVS pixels produce output only when the change in intensity has surpassed a set threshold, which leads to reduced power consumption in scenes with relatively little motion. Furthermore, compared to conventional CMOS imaging pixels, DVS pixels have extremely high dynamic range, low latency, and low motion blur. Taken together, these characteristics make event cameras uniquely qualified for persistent surveillance. In particular, we have been investigating their use in port surveillance applications. Such an application of DVS presents the need for automated pattern recognition and object tracking algorithms which can process event data. Due to the fundamentally different nature of the output relative to conventional frame-based cameras, traditional methods of machine learning for computer vision cannot be directly applied. Anticipating this need, this work details data collection and collation efforts to facilitate development of object detection and tracking algorithms in this modality. We have assembled a maritime dataset capturing several moving objects including sail boats, motor boats, large ships, etc.; as well as incidentally captured objects. The data was collected with lenses of various focal lengths and aperture settings to provide data variability and avoid unwanted bias to specific sensor parameters. In addition, the captured data was recorded with the camera in both static and dynamic states. These different states can be used to mimic potential behavior and help understand how this movement can affect the algorithms being developed for automated ship detection and tracking. We will describe the data captured, effects of hardware settings and lenses, as well as how lighting conditions and sensor movement contributed to the quality of the event data recorded. Finally, we will detail future data collection efforts.

In this paper we address the basic limitation of SiamMask - the state of the art single object tracking and segmentation algorithm. SiamMask requires semi-supervision in that it needs a bounding box to be drawn manually around the object that has to be tracked. This is however not always possible or feasible, and slows down the pipeline even in the best case. We overcome this limitation by using state-of-the-art object detection algorithms: Detectron2 and YOLO to automatically detect the object and then track using SiamMask. The main purpose of this study is to devise an efficient technique for an end-to-end object detection and tracking, which can then be used in other applications like self-driving cars, etc. We compared different approaches using current state-of-the-art tools for time and detection efficiency. One of the secondary aim was to test how the two approaches perform on different types of datasets. We note that YOLO gives better and more meaningful detection of objects in the scene. However, Detectron2 gives a higher detection speed than YOLO, making the overall detection and tracking process faster.

Object tracking in video is a challenging problem in several applications such as video surveillance, video compression, video retrieval, and video editing. Tracking an object in a video is not easy due to loss of information caused by illumination changing in a scene, occlusions with other objects, similar target appearances, and inaccurate tracker responses. In this paper, we present a novel object detection and tracking algorithm via structured output prediction classifier. Given an initial bounding box with its position, we first divide it into sub-blocks with a predefined size. Next, we extract the features from each sub-blocks with Haar-like features method. And then we learn those features with a structured output prediction classifier. We treat the sub-blocks obtained from the initial bounding box as positive samples and then randomly choose negative samples from search windows defined by the specific area around the bounding box. After that, we obtain prediction scores for each sub-blocks both from positive and negative samples. We construct a region-graph with sub-blocks as nodes and classifier’s score as weight to detect the target object in each frame. Our experimental results show that the proposed method outperforms state-of-the-art object tracking algorithms.KeywordsObject DetectionObject TrackingSupport Vector Machine

The Kalman filter has long been regarded as the optimal solution to many applications in computer vision for example the tracking objects, prediction and correction tasks. Its use in the analysis of visual motion has been documented frequently, we can use in computer vision and open cv in different applications in reality for example robotics, military image and video, medical applications, security in public and privacy society, etc. In this paper, we investigate the implementation of a Matlab code for a Kalman Filter using three algorithm for tracking and detection objects in video sequences (block-matching (Motion Estimation) and Camshift Meanshift (localization, detection and tracking object)). The Kalman filter is presented in three steps: prediction, estimation (correction) and update. The first step is a prediction for the parameters of the tracking and detection objects. The second step is a correction and estimation of the prediction parameters. The important application in Kalman filter is the localization and tracking mono-objects and multi-objects are given in results. This works presents the extension of an integrated modeling and simulation tool for the tracking and detection objects in computer vision described at different models of algorithms in implementation systems.

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

Object tracking is an important basis for the autonomous navigation of unmanned surface vehicles. However, several problems still must be addressed for a wide applicating of object tracking in unmanned surface vehicles. First, if multiple objects of the same classification exist in the same field of view, then stable extraction of an object is difficult. Second, in an environment with a complex background and large changes in object shape, the tracking accuracy is low, and object tracking errors and tracking loss can easily occur. Third, much time is required to detect a high-resolution real-time video stream, not meeting the delay requirement of the photoelectric servo stable tracking. To resolve these problems, this paper proposes an object detection-tracking algorithm based on a radar-photoelectric system. The algorithm combines an object detection algorithm with an object tracking algorithm and involves the following steps. First, a first-frame object extraction algorithm is used to extract the tracking object from the first frame. Second, a region of interest (ROI)-prediction algorithm is used to predict ROIs and detect objects in these ROIs. This algorithm can effectively solve the above problems in marine tests. When multiple objects of the same classification exist in the same field of view, the algorithm can extract the radar-guided object stably. When faced with a complex background and a large change in object shape, the algorithm substantially improves the accuracy and robustness of object tracking. Compared with the conventional object detection algorithm, the time consumption of this algorithm is reduced by 25.8%.

3D tracking of objects and hands in an object manipulation scenario is a very interesting computer vision problem with a wide variety of applications ranging from consumer electronics to robotics and medicine. Recent advances in this research topic allow for 3D tracking of complex scenarios involving bi-manual manipulation of several rigid objects using commodity hardware and with high accuracy. The problem with these approaches is that they treat tracking as a search problem whose dimensionality increases with the number of objects in the scene. This fact typically limits the number of the tracked objects and/or the processing framerate. In this paper we present a method that utilizes simple low level motion cues for dynamically assigning computational resources to parts of the scene where they are actually required. In a series of experiments, we show that this simple idea improves tracking performance dramatically at a cost of only a minor degradation of tracking accuracy. The works that are most related to ours are the approaches by Kyriazis and Argyros on top-down 3D tracking of multiple active objects from RGBD input [1, 2]. The methodological part of our contribution can be briefly described as an extra processing node in the pipeline of [2] which it extends. The tracking approach in [2], the Ensemble of Collaborative Trackers (ECT), regards a set of semi-independent trackers. Each tracker is associated with a distinct object in the scene. For an object to be tracked, a separate optimization problem is solved, one for each frame and each object. Each optimization problem is numerically solved using a black box optimizer, i.e. a variant of the Particle Swarm Optimization (PSO) algorithm. PSO treats the objective function as an oracle and queries it on purposefully evolved “guesses” in the search space, in order to find the optimum. The harder the problem is, the more guesses (budget, which is the product of PSO particles and PSO generations that need to be computed) are required for adequate accuracy to be achieved. For example, tracking the pose and the articulation of the hand amounts to solving a 27-parameter optimization problem. This is much harder than solving for the 3D pose of a rigid object (6 parameters). ECT [2] assigns a fixed amount of computational resources to each tracking sub-problem which depends on this notion of complexity and which is empirically estimated. In this work, we quantify at run time, how hard the tracking of an object should be, not only based on its intrinsic complexity, but also based on its observed dynamics. An object that appears static in the recent temporal window requires less resources compared to an object whose state is more dynamic. Thus, change detection is performed on the image space of color intensities and depth measurements of the RGBD input. In more detail, for the next tracking frame, and the ith tracker, a value mi is computed, which takes the value of 0 if the corresponding tracked object appears to be relatively static, and takes the value of 1 otherwise. For mi to be 1, any of the following need to be true: • the mean value of the pixel-wise differences between the observations of object i and the back-projection of the last estimated configuration of object i is high enough, • the kinetic energy of object i, as computed from so far tracked velocities, suggests a moving object, • the amount of missing depth measurements changes substantially, from one frame to the next, which might me attributed to change in the slant of object i, or • any of the above was satisfied in the recent past (damping). Budget has the trivial minimum value of 0. Moreover, as it has been shown in [3, 4], a PSO budget of 64 particles and 64 generations suffices to track a hand, even in interaction with another hand. No more budget should be required to track simpler structures such as rigid objects. The proposed dynamic budget allocation policy assigns a minimum budget Bmin of 64 particles running for 4 generations to the objects that are static and a maximum budget Bmax that never exceeds the aforementioned (a) Pouring pancake mix (b) Waiting for the mix to cook

Moving target detection and tracking algorithm as the core issue of computer vision and human-computer interaction is the first step of intelligent video surveillance system. Through comparing temporal difference method and background subtraction, a moving object detection and tracking algorithm based on background subtraction under static background is proposed, in order to quickly and accurately detect and identify the moving object in the intelligent monitoring system. In this algorithm, firstly, we use background acquisition method to receive the background image, then use the current frame image and the received background image to perform background subtraction in order to extract foreground object information and receive the difference image; secondly, we use threshold segmentation and morphology image processing to process the difference image in order to eliminate noises and receive the clear binary moving object image; finally, we use the centroid tracking method to track and mark the moving object. Experimental results show that the algorithm can effectively and quickly detect and track moving object from video sequence under static background. This algorithm is easily realized and has good real-time and robust, which is automated and self triggered for background updating. The algorithm can be used in driver assistance systems, motion capture, virtual reality and other fields.

The rapid advancement in the field of deep learning and high performance computing has highly augmented the scope of video-based vehicle counting system. In this paper, the authors deploy several state-of-the-art object detection and tracking algorithms to detect and track different classes of vehicles in their regions of interest (ROI). The goal of correctly detecting and tracking vehicles’ in their ROI is to obtain an accurate vehicle count. Multiple combinations of object detection models coupled with different tracking systems are applied to access the best vehicle counting framework. The models’ addresses challenges associated to different weather conditions, occlusion and low-light settings and efficiently extracts vehicle information and trajectories through its computationally rich training and feedback cycles. The automatic vehicle counts resulting from all the model combinations are validated and compared against the manually counted ground truths of over 9 h’ traffic video data obtained from the Louisiana Department of Transportation and Development. Experimental results demonstrate that the combination of CenterNet and Deep SORT, and YOLOv4 and Deep SORT produced the best overall counting percentage for all vehicles.