A lightweight 3D object tracking method in complex scenes

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed.

Six degrees-of-freedom (6D) object pose estimation plays an important role in pattern recognition of fields such as robotics and augmented reality. However, there are issues with low accuracy and real-time performance of 6D object pose estimation in complex scenes. To address these challenges, in this article, RFF-PoseNet (a 6D object pose estimation network based on robust feature fusion) is proposed for complex scenes. Firstly, a more lightweight Ghost module is used to replace the convolutional blocks in the feature extraction network. Then, a pyramid pooling module is added to the semantic label branch of PoseCNN to fuse the features of different pooling layers and enhance the network’s ability to capture information about objects in complex scenes and the correlations between contextual information. Finally, a pose regression and optimization module is utilized to further improve object pose estimation in complex scenes. Simulation experiments conducted on the YCB-Video and Occlusion LineMOD datasets show that the RFF-PoseNet algorithm can strengthen the correlation of features between different levels and the recognition ability of unclear targets, thereby achieving excellent accuracy and real-time performance, as well as strong robustness.

Particle filter (PF) based object tracking methods have been widely used in computer vision. However, traditional particle filter trackers cannot effectively distinguish the target from the background in complex scenes since they only exploit appearance information of observation to determine the target region. In this paper, we present an adaptive particle filter based on energy field (EPF), which makes good use of moving information of previous frames adaptively to track the target. Besides, we present the mechanism of result rectification to ensure the target region is accurate. Experiment results on several challenging video sequences have verified that the adaptive EPF method is compared very robust and effective with the traditional particle filter in many complicated scenes.KeywordsTrackingparticle filterprobabilistic approximationimage sequence analysisdynamic scenes

Discriminative correlation filters show excellent performance in object tracking. However, in complex scenes, the apparent characteristics of the tracked target are variable, which makes it easy to pollute the model and cause the model drift. In this paper, considering that the secondary peak has a greater impact on the model update, we propose a method for detecting the primary and secondary peaks of the response map. Secondly, a novel confidence function which uses the adaptive update discriminant mechanism is proposed, which yield good robustness. Thirdly, we propose a robust tracker with correlation filters, which uses hand-crafted features and can improve model drift in complex scenes. Finally, in order to cope with the current trackers' multi-feature response merge, we propose a simple exponential adaptive merge approach. Extensive experiments are performed on OTB2013, OTB100 and TC128 datasets. Our approach performs superiorly against several state-of-the-art trackers while runs at speed in real time.

Accurate recognition of circular marks is crucial for calibration, object tracking, and three-dimensional reconstruction in videogrammetry. However, most existing studies were designed under single or relatively simple scenes. When the existing algorithms are applied to more complex scenarios, it will result in higher false detection and miss-detection rate. In this article, we present a high-precision recognition method based on a novel deep learning model, Circular-MarkNet (CMNet) to solve this problem. The proposed network consists of three main steps: first, circular marks are detected using the improved YOLOv4 model to narrow the search region of the circular contour; the contour of the circular marks is then extracted based on the saliency object detection model BASNet; and finally, least square fitting (LSF) is used to calculate the central pixel coordinate of the identified contour on the saliency map. The proposed method was tested under three complex scenarios with different characteristics and disturbances. The experimental results demonstrated that: (1) the proposed CMNet can effectively recognize of circular marks within complex scenes, which reveals the superiority and generalization ability of the proposed method; (2) the improved YOLOv4 can significantly enhance the detection accuracy of circular marks, which is crucial to the subsequent saliency courter detection and circle center identification; (3) Circular-MarkNet achieved the best performance, with an RMSE of 0.0713 pixel, compared to the state-of-the-art methods.

The event camera, a new bio-inspired vision sensor with low latency and high temporal resolution, has brought great potential and demonstrated a promising application in machine vision and artificial intelligence. Corner detection is a key step of object motion estimation and tracking. However, most existing event-based corner detectors, such as G-eHarris and Arc*, lead to a huge number of redundant or wrong corners, and cannot strike a balance between the accuracy and real-time performance, especially in complex scenes with high texture that require higher computational costs. To address these issues, we propose an asynchronous corner detection method: a double threshold filter with Sigmoid eHarris (DTFS-eHarris) and an asynchronous corner tracker. The main contributions are that a double threshold filter is designed to reduce the redundant events and the improved Sigmoid function is utilized to represent the Surface of Active Events (Sigmoid*-SAE). We selected four scenes—shapes, dynamic, poster and boxes—from the public event camera dataset DAVIS240C to compare with the existing state-of-the-art hybrid method; our method has shown more than a 10% reduction in false positive rate and a 5% and 20% improvement in accuracy and throughput, respectively. The evaluations indicate that DTFS-eHarris shows a significant improvement, especially in complex scenes. Thus, it is anticipated to enhance the real-time performance and feature detection accuracy for future robotic applications.

We consider the problem of tracking multiple interacting objects in 3D, using RGBD input and by considering a hypothesize-and-test approach. Due to their interaction, objects to be tracked are expected to occlude each other in the field of view of the camera observing them. A naive approach would be to employ a Set of Independent Trackers (SIT) and to assign one tracker to each object. This approach scales well with the number of objects but fails as occlusions become stronger due to their disjoint consideration. The solution representing the current state of the art employs a single Joint Tracker (JT) that accounts for all objects simultaneously. This directly resolves ambiguities due to occlusions but has a computational complexity that grows geometrically with the number of tracked objects. We propose a middle ground, namely an Ensemble of Collaborative Trackers (ECT), that combines best traits from both worlds to deliver a practical and accurate solution to the multi-object 3D tracking problem. We present quantitative and qualitative experiments with several synthetic and real world sequences of diverse complexity. Experiments demonstrate that ECT manages to track far more complex scenes than JT at a computational time that is only slightly larger than that of SIT.

Research on object tracking has been an active field because of its fundamental roles in surveillance and monitoring. In this paper, a new adaptive algorithm for fast target tracking based on hierarchical block matching is proposed. The improvement of the algorithm lies in three aspects. First, the hierarchical pyramid method is applied to enhance the search efficiency of the local features. In addition, at different pyramid level, an adaptive searching strategy is adopted according to the number of candidate features. Second, since image blocks make full use of local characters of features, the structure similarity method, associated with the relation coefficient in statistics, can be exploited to design a new matching criteria, which provides more precise matching results, especially in the complex scenes such as the change of illumination and partial occlusion. Lastly, traditional algorithms often track the object through adjacent frames, whereas our algorithm will search two frames with a variable interval on the basis of matching results. The intervals are obtained by using the bisection method and thus the searching time can be greatly saved. Simulation results show that the proposed block matching method has better performances in the searching time and accuracy in contrast to the classic Mean-Shift target tracking algorithm and Particle Filter target tracking algorithm under the complex circumstances such as illumination variation and partial occlusion.

In vision-based robotic systems the robust tracking of scene features is a key element of grasping, navigation and interpretation tasks. The stability of feature initialisation and tracking is strongly influenced by ambient conditions, like lighting and background, and their changes over time. This work presents how robustness can be increased especially in complex scenes by reacting to a measurement of the scene content. Element candidates are proposed, to indicate the scene complexity remaining after running a method. Local cue integration and global topological constraints are applied to select the best feature set. Experiments show in particular the success of the approach to disambiguate features in complex scenes.

Under the influence of COVID-19, detection and identification of moving targets are very important for personnel management. A lot of research work has improved the accuracy and robustness of the moving target tracking method, but the recognition accuracy of the traditional target tracking method in complex scenes (lighting changes, background interference, posture changes and other factors) is not satisfactory. In this paper, in view of the limitations of single feature representation of target objects, the method of fusion of HSV color features and edge direction features is used to identify and detect moving targets. In each frame of the tracking process, the weight of each feature is adjusted adaptively according to the proposed fusion strategy, and the position of the target is located by using the method of double template matching. Experiments show that the proposed tracking algorithm based on multi feature fusion can meet the requirements of moving target recognition in complex scenes. The method proposed in this paper has a certain reference value for personnel management under the influence of COVID-19.

In this paper we propose a pixel-wise visual tracking method using a novel tri-model representation. The newly proposed tri-model is composed of three models, which each model learns the target object, the background, and other non-target moving objects online. The proposed method performs tracking by simultaneous estimation of the holistic position of the target object and the pixel-wise labels. By utilizing the information in the background and the foreground models as well as the target model, our method obtains robust results even under background clutters and partial occlusions in complex scenes. Furthermore, our method is able to give pixel-wise results, and uses them in the learning process to prevent drifting. The method is extensively tested against seven representative trackers both quantitatively and qualitatively showing promising results.

ECO++: Adaptive deep feature fusion target tracking method in complex scene

BackgroundInspection of livestock farms using surveillance cameras is emerging as a means of early detection of transboundary animal disease such as African swine fever (ASF). Object tracking, a developing technology derived from object detection aims to the consistent identification of individual objects in farms.ObjectivesThis study was conducted as a preliminary investigation for practical application to livestock farms. With the use of a high-performance artificial intelligence (AI)-based 3D depth camera, the aim is to establish a pathway for utilizing AI models to perform advanced object tracking.MethodsMultiple crossovers by two humans will be simulated to investigate the potential of object tracking. Inspection of consistent identification will be the evidence of object tracking after crossing over. Two AI models, a fast model and an accurate model, were tested and compared with regard to their object tracking performance in 3D. Finally, the recording of pig pen was also processed with aforementioned AI model to test the possibility of 3D object detection.ResultsBoth AI successfully processed and provided a 3D bounding box, identification number, and distance away from camera for each individual human. The accurate detection model had better evidence than the fast detection model on 3D object tracking and showed the potential application onto pigs as a livestock.ConclusionsPreparing a custom dataset to train AI models in an appropriate farm is required for proper 3D object detection to operate object tracking for pigs at an ideal level. This will allow the farm to smoothly transit traditional methods to ASF-preventing precision livestock farming.

A fast active contour algorithm for object tracking in complex background

3D tracking of objects and hands in an object manipulation scenario is a very interesting computer vision problem with a wide variety of applications ranging from consumer electronics to robotics and medicine. Recent advances in this research topic allow for 3D tracking of complex scenarios involving bi-manual manipulation of several rigid objects using commodity hardware and with high accuracy. The problem with these approaches is that they treat tracking as a search problem whose dimensionality increases with the number of objects in the scene. This fact typically limits the number of the tracked objects and/or the processing framerate. In this paper we present a method that utilizes simple low level motion cues for dynamically assigning computational resources to parts of the scene where they are actually required. In a series of experiments, we show that this simple idea improves tracking performance dramatically at a cost of only a minor degradation of tracking accuracy. The works that are most related to ours are the approaches by Kyriazis and Argyros on top-down 3D tracking of multiple active objects from RGBD input [1, 2]. The methodological part of our contribution can be briefly described as an extra processing node in the pipeline of [2] which it extends. The tracking approach in [2], the Ensemble of Collaborative Trackers (ECT), regards a set of semi-independent trackers. Each tracker is associated with a distinct object in the scene. For an object to be tracked, a separate optimization problem is solved, one for each frame and each object. Each optimization problem is numerically solved using a black box optimizer, i.e. a variant of the Particle Swarm Optimization (PSO) algorithm. PSO treats the objective function as an oracle and queries it on purposefully evolved “guesses” in the search space, in order to find the optimum. The harder the problem is, the more guesses (budget, which is the product of PSO particles and PSO generations that need to be computed) are required for adequate accuracy to be achieved. For example, tracking the pose and the articulation of the hand amounts to solving a 27-parameter optimization problem. This is much harder than solving for the 3D pose of a rigid object (6 parameters). ECT [2] assigns a fixed amount of computational resources to each tracking sub-problem which depends on this notion of complexity and which is empirically estimated. In this work, we quantify at run time, how hard the tracking of an object should be, not only based on its intrinsic complexity, but also based on its observed dynamics. An object that appears static in the recent temporal window requires less resources compared to an object whose state is more dynamic. Thus, change detection is performed on the image space of color intensities and depth measurements of the RGBD input. In more detail, for the next tracking frame, and the ith tracker, a value mi is computed, which takes the value of 0 if the corresponding tracked object appears to be relatively static, and takes the value of 1 otherwise. For mi to be 1, any of the following need to be true: • the mean value of the pixel-wise differences between the observations of object i and the back-projection of the last estimated configuration of object i is high enough, • the kinetic energy of object i, as computed from so far tracked velocities, suggests a moving object, • the amount of missing depth measurements changes substantially, from one frame to the next, which might me attributed to change in the slant of object i, or • any of the above was satisfied in the recent past (damping). Budget has the trivial minimum value of 0. Moreover, as it has been shown in [3, 4], a PSO budget of 64 particles and 64 generations suffices to track a hand, even in interaction with another hand. No more budget should be required to track simpler structures such as rigid objects. The proposed dynamic budget allocation policy assigns a minimum budget Bmin of 64 particles running for 4 generations to the objects that are static and a maximum budget Bmax that never exceeds the aforementioned (a) Pouring pancake mix (b) Waiting for the mix to cook