Image processing techniques for object tracking in video surveillance- A survey

This paper presents a principled way for dealing with occlusions in visual tracking which is a long-standing issue in computer vision but largely remains unsolved. As the major innovation, we develop a learning-based jump-diffusion process to jointly track object locations and estimate their visibility statuses over time. Our method employs in particular a set of jump dynamics to change object's visibility statuses and a set of diffusion dynamics to track objects in videos. Different from the traditional jump-diffusion process that stochastically generates dynamics, we utilize deep policy functions to determine the best dynamic at the present step and learn the optimal policies from raw videos using reinforcement learning methods.Our method is capable of tracking objects with severe occlusions in crowded scenes and thus recovers the complete trajectories of objects that undergo multiple interactions with others. We evaluate the proposed method on challenging video sequences and compare it to alternative methods. Significant improvements are obtained particularly for the videos including frequent interactions or occlusions.

Network video surveillance has been a popular security application for many years. Target tracking in a cluttered environment remains one of the challenging problems of video surveillance. The task of target tracking is a key component of video surveillance and monitoring systems. It provides input to high-level processing such as recognition, access control or re-identification or is used to initialize the analysis and classification of human activities. Intelligent and automated security surveillance systems have become an active research area in recent time due to an increasing demand for such systems in public areas such as airports, underground stations and mass events. In this context, tracking of stationary foreground regions is one of the most critical requirements for surveillance systems based on the tracking of abandoned or stolen objects or parked vehicles. Object tracking based techniques are the most popular choice to detect stationary foreground objects because they work reasonably well when the camera is stationary and the change in ambient lighting is gradual, and they also represent the most popular choice to separate foreground objects from the current frame. In this paper, we did the literature survey on different technique and finally carried out our methodology for the same situation.

Computer vision-based motion target detection and tracking, which is widely used in video surveillance, human-computer interaction, range interpretation, and other fields, is one of the current research hotspots in the field of computer vision. In engineering scenarios, the two are inseparable and need to work together to accomplish specific tasks. The related research is progressing rapidly, but there is still room for improving its timeliness, accuracy, and automation. In this paper, we summarize and classify some classical target detection methods, analyze the basic principles of convolutional neural networks, and analyze the classical detection algorithms based on region suggestion and deep regression networks. After that, we improve the SSD algorithm for the shortage of low-level feature convolution layers, which has insufficient feature extraction and leads to poor detection of small targets. For the motion target tracking problem, this paper studies the motion target tracking method based on support vector machine and proposes the tracking method of support vector regression and the corresponding online support vector regression solution method based on the analysis of support vector tracking method and structural support vector tracking method. In this paper, we propose a tracking method that fuses structural support vector machines and correlation filtering. The structure is based on the idea of Inception, which adds and replaces some feature convolution layers of the original network while maintaining the original lightweight backbone. The final experiments on the VOC data set demonstrate that the improved algorithm improves the average detection accuracy by 2.6% compared to the original algorithm and basically maintains the real-time speed as well. Experimental simulations on a subset of VOC data (human set) show a significant improvement in AP values and more effective and reliable detection tracking of moving targets. The stability and accuracy of motion target detection and tracking are improved by setting parameters, such as confidence level; the effectiveness and continuity of detection and tracking are judged by setting the interframe centroid distance.

When entering into the realm of Computer Vision, the first thing which comes in to mind is Visual tracking. Visual tracking by far comes into one of the most actively investigated research areas because of the fact that it has an extensive collection of applications in areas such as activity recognition, surveillance, motion analysis and as well as human computer interaction. Some serious challenges of this area which still create hindrance in achieving 100% accuracy are abrupt appearance and pose changes of an object along with its background blockage due to blockages called occlusion, illumination and lighting variances and changes in scale of target object in the frames. Moreover, diverse algorithms had been proposed for the resolution of said issue. Now in such cases, if we study the statistical analysis of correlation between two frames in a certain video, it can be efficiently utilized to get the most exact location of the targeted object. The algorithms in existence today do not completely exploit a strong spatio-temporal relationship that very often occurs between the two successive frames in a video sequence. Recent advances in correlation-based tracking systems have been proposed to address the problem in successive frames. In this thesis a very simple yet quite speedy and robust algorithm that in actual brings all the relevant information used for Visual Tracking. Two of the Models proposed are the “Locality Sensitive Histogram” and “Discriminative Scale Tracking Method”. These are robust enough to the variations which are based on appearance which are normally presented by blockage, pose, illumination and lighting variations alike. A scheme is proposed called scale adaptation which is very much clever to adapt variations of targeted scale in the most efficient manner. The Discriminative Scale Tracking Method is used for detection as well as scale change ultimately resulting in an effective tracking method in the end. Various different experiments with the best algorithms have demonstrated on challenging sequences that the suggested methodology attains promising results as far as robustness, accuracy, and speed is concerned.

Recently, the correlation filter (CF)-based methods have achieved great success in the field of object tracking. In most of these methods, the CF utilizes L 2 norm as the regularization, which does not pay attention to the stability and robustness of the feature. However, there may exist some unstable points in the image because the object in the video may have different appearance changes. We propose a tracking method based on a structured robust correlation filter (SRCF), which employs the L 2,1 norm as the regularization. The robust CF can not only retain the accuracy from the regression formulation but also take into account the stability of the image region to improve the robustness of the appearance model. The alternating direction method of multipliers algorithm is used to solve the L 2,1 optimization problem in SRCF. Moreover, the multilayer convolutional features are adopted to further improve the representation accuracy. The proposed method is evaluated in several benchmark datasets, and the results demonstrate that it can achieve comparable performance with respect to the state-of-the-art tracking methods.

Visual tracking has attracted a significant attention in the last few decades. The recent surge in the number of publications on tracking-related problems have made it almost impossible to follow the developments in the field. One of the reasons is that there is a lack of commonly accepted annotated data-sets and standardized evaluation protocols that would allow objective comparison of different tracking methods. To address this issue, the Visual Object Tracking (VOT) workshop was organized in conjunction with ICCV2013. Researchers from academia as well as industry were invited to participate in the first VOT2013 challenge which aimed at single-object visual trackers that do not apply pre-learned models of object appearance (model-free). Presented here is the VOT2013 benchmark dataset for evaluation of single-object visual trackers as well as the results obtained by the trackers competing in the challenge. In contrast to related attempts in tracker benchmarking, the dataset is labeled per-frame by visual attributes that indicate occlusion, illumination change, motion change, size change and camera motion, offering a more systematic comparison of the trackers. Furthermore, we have designed an automated system for performing and evaluating the experiments. We present the evaluation protocol of the VOT2013 challenge and the results of a comparison of 27 trackers on the benchmark dataset. The dataset, the evaluation tools and the tracker rankings are publicly available from the challenge website (http://votchallenge.net).

The utilization of video surveillance systems is becoming common and is expected to more widespread as societies become more complex and the population continues to grow. The implementation of these video surveillance systems has provided valuable information and assistance in monitoring large areas. These systems typically use human operators to determine human behavior and to manually track people or objects of interest over an array of cameras. With the application of computers with video surveillance, real time surveillance of large public areas, people and their activities has been made possible for monitoring and security. The main contribution of this paper is to show the effect of color space on tracking methods in video surveillance. Results from evaluations on different tracking methods have indicated that YCbCr and HSV color spaces have better tracking ability compared to grayscale and RGB color spaces. In addition, the results from evaluations have also indicated that data from selected layers in some color spaces can be used for the purpose of tracking namely the Cb and Cr layers from the YCbCr color space and the H layer from the HSV color space.

Real-time contour tracking in video surveillance suffers the problem of low speed and influence of shadows and noises. In this paper, a novel approach of video tracking is proposed, which employs a faster level-set-based method and a foreground extracting method free of noises. Compared with traditional level-set-based methods, our method avoids the high computational complexity of solving partial differential equations. In addition to the fast tracking method, we analyze the color information in the RGB color space so that the shadows and noises are eliminated in foreground extraction. It is performed without any manual thresholds’ tuning. Furthermore, we make use of the foreground extraction process to get the coarse contour of the object, which is also performed automatically and significantly reduces the time spent in the curve evolution step. With this approach, real-time surveillance system can be implemented. Experiments show that our approach outperforms previous methods for indoor environments.

There are mainly two components in almost every visual object tracking algorithm, which are the object presentation and the searching mechanism. In the literature, metaheuristic algorithms have been recently, widely used as searching method in visual tracking. In fact, the more discriminative is their combined object presentation the better is their precision. Salient background information is often inside the first detected region of the object, which may decrease considerably the tracking performance. In this work, we combine a powerful metaheuristic searching method, whale optimization algorithm (WOA), with the discriminative presentation named as corrected background weighted histogram (CBWH) to improve the tracking accuracy in difficult environments. WOA algorithm is used because of its strong searching ability, which enable it avoid being trapped in local optimum, and also its lowest computation cost compared to other metaheuristic algorithms. The WOA sensitivity parameters are studied experimentally to be adjusted in visual tracking. For a fair assessment of the proposed approach, its performance has been evaluated on 20 video sequences, from the famous object tracking benchmark OTB15, and compared with six (6) other state-of-art trackers. The evaluations were carried quantitatively and qualitatively on challenging situations and have provided satisfying results. The experimental results shows that WOA combined with CBWH can lead to further improvement, and better success rate in tracking.

Most methods for multiple object tracking in video represent the state of multi-object in a high dimensional joint state space. This leads to high computational complexity. This paper presents a method using the probability hypothesis density (PHD) filter to estimate the state of multiple objects in video. The method operates on the single object state space instead of the joint state space. A PHD recursion for visual observations with color measurements is proposed. Our method can track varying number of objects.

Target tracking is an important issue in computer vision for its widely applications in video surveillance, human-computer interaction, robotic navigation, image compression and so on. There exist challenging problems such as occlusion, pose change, illumination change etc. in real videos. The two-stage sparse coding method for target tracking is proposed in this paper. The proposed method is based on Bayesian network and sparse coding with dynamic dictionary. In order to overcome the visual drift, the two-stage method is applied with target and dictionary updating to realize tracking accurately and quickly. Some public and popular challenging sequences are used to demonstrate the effectiveness of the proposed method. The experiment results show that the proposed method has better performance compared with other state-of-the-art methods.

The inverse synthetic aperture lidar (ISAL) have attracted increasing attention for its merits including small visual tracking which is considered as one of the important research topics in the field of computer vision due to its key role in versatile applications, such as precision guidance, intelligent video surveillance, human-computer interaction, robot navigation and public safety. The basic idea for implementing visual tracking is composed of finding the target object in a video or sequence of images, then determining its exact position in the next successive frames and finally generating the corresponding trajectory of this object. Visual tracking, however, is still a challenging problem in practice while taking into account the abrupt appearance changes of the target objects induced by their non-rigid transformation, the sophisticated lighting variation, the obstruction by the block or similar objects in the background and the camera jitter. Motivated by the successful applications in target detection and recognition in recent years, plenty of deep learning models have been integrated in the visual tracking and better performance over traditional methods was achieved in a series of data evaluations, which opens a new door in the field of visual tracking. In this paper, the overview and progress on visual tracking were summarized. The current challenges and corresponding solving approaches in this field are introduced firstly and in particular, several novel and mainstream visual tracking algorithms based on the deep learning are specially described and analyzed in details, including their basic ideas, advantages and disadvantages and future prospect.

This study addresses the challenges of selecting a suitable visual tracking method for real-time mobile robot applications, particularly in scenarios where the target is moving on the ground. The primary research problem addressed is the need for a flexible, computationally efficient tracking method that does not rely on pre-existing labelled datasets, as is often required by deep learning approaches. Unsupervised methods can overcome this problem by utilizing object motion information in each image frame without prior training. With many unsupervised tracking methods available, choosing an appropriate algorithm that can perform efficiently under dynamic conditions becomes a critical problem. The study compares the performance of three unsupervised visual tracking methods: particle filter, optical flow, and channel and spatial reliability tracker (CSRT) under various tracking conditions. The dataset used includes challenges such as moving target variations, changes in object scale, viewpoint changes, suboptimal lighting, image blurring, partial occlusions, and abrupt movements. Evaluation criteria include tracking accuracy, resistance to occlusion, and computational efficiency. The particle filter with ORB and a constant velocity model achieves a root mean square error (RMSE) of 36.47 pixels at 13 frames per second (fps). Optical flow performs best with an RMSE of 10.79 pixels at 30 fps, while CSRT shows an RMSE of 252.35 pixels at 4 fps. These findings highlight the effectiveness of optical flow for real-time applications, making it a promising solution for mobile robot visual tracking in challenging situations.

Visual tracking is an important area in computer vision. Based on the Siamese network, current tracking methods employ the self‐attention block in convolutional networks to extract semantic features containing the image structure information of an object. However, spatial continuity is a point of contradiction between two seemingly unrelated challenges, that is, occlusion and similar distractor, in tracking methods. At the same time, it is a spatially discontinuous task to locate a target reappearing after occlusion accurately. The prediction of bounding boxes should be constrained by spatial continuity to prevent them from jumping into similar distractors. This study proposes a novel tracking method for introducing spatial continuity in visual tracking called M‐CoTransT; the novel tracking method is developed through the confidence‐based adaptive Markov motion model (M‐model) and a novel correlation‐based feature fusion network (CoTransT). In particular, the M‐model provides confidence for the nodes of the Markov motion model to estimate the motion state continuity. It also predicts a more accurate search region for CoTransT, which then adds a cross‐correlation branch into the self‐attention tracking network to enhance the continuity of target appearance in the feature space. Extensive experiments on five challenging datasets (LaSOT, GOT‐10k, TrackingNet, OTB‐2015 and UAV123) demonstrated the effectiveness of the proposed M‐CoTransT in visual tracking.

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed.