Multi-Channel Features Spatio-Temporal Context Learning for Visual Tracking

This paper gives an overview of the various potential applications involved in human motion tracking. Also, a review of some relevant algorithms in this area are summarised based on three main components: the human target extraction, the features extraction, and the motion model. In addition, a proposed generative and discriminative human tracking method is presented. This proposed method is based on the hybridisation of Kalman filter (KF) and the support vector machines (SVMs). Numerous experiments illustrate the effectiveness of the proposed method against several state of the art trackers. These experiments are applied on several real world image sequences with various challenges that make the tracker vulnerable which are: partial and full occlusions, illumination changes, scale variation, out-of-plane, rotation, motion blur, low resolution, deformation, fast motion and background clutter.

Object tracking is still an intriguing task as the target undergoes significant appearance changes due to illumination, fast motion, occlusion and shape deformation. Background clutter and numerous other environmental factors are other major constraints which remain a riveting challenge to develop a robust and effective tracking algorithm. In the present study, an adaptive Spatio-temporal context (STC)-based algorithm for online tracking is proposed by combining the context-aware formulation, Kalman filter, and adaptive model learning rate. For the enhancement of seminal STC-based tracking performance, different contributions were made in the proposed study. Firstly, a context-aware formulation was incorporated in the STC framework to make it computationally less expensive while achieving better performance. Afterwards, accurate tracking was made by employing the Kalman filter when the target undergoes occlusion. Finally, an adaptive update scheme was incorporated in the model to make it more robust by coping with the changes of the environment. The state of an object in the tracking process depends on the maximum value of the response map between consecutive frames. Then, Kalman filter prediction can be updated as an object position in the next frame. The average difference between consecutive frames is used to update the target model adaptively. Experimental results on image sequences taken from Template Color (TC)-128, OTB2013, and OTB2015 datasets indicate that the proposed algorithm performs better than various algorithms, both qualitatively and quantitatively.

We propose a novel method for the illumination robust template matching in unconstrained environments, named illumination robust template matching (IRTM). IRTM is based on the patch diversity of the Nearest-Neighbor. A core component is to calculate the similarity between patch pairs. Without illumination change, the Euclidean distance of appearance and geometric features jointly measure the similarity well. However, the illumination change will significantly affect the confidence of measure. In this paper, based on the relative pixel values that do not change with illumination, we define an illumination-robust distance function. With experiments, IRTM is shown to be a well-performing similarity measure against illumination changes, non-rigid geometric deformations, as well as background clutter, and occlusions.

Target tracking is a popular but challenging problem in computer vision field. Due to many aggravating factors such as position transformation, illumination, occlusion, it is difficult to achieve robust target tracking. According to the above constraints, an improved target tracking algorithm based on spatio-temporal context (STC) under occlusions is proposed. On the basis of STC, the proposed method introduces a novel mechanism for dealing with occlusion, the scale update, and the learning rate update to reduce the error update of the model and restrain error accumulation. As a consequence, the tracking performance can be improved efficiently. Extensive experimental results show that our algorithm outperforms the original STC algorithm and some other state-of-the-art algorithms.

Vehicle tracking, significant in the computer vision using machine learning method, allows the vehicle to comprehend its immediate environment and therefore, enhances the intelligence of the vehicles and the safety of vehicle occupants. We propose a novel tracking algorithm that can work robustly under challenging circumstances such as road scene where several kinds of appearance and motion changes of a tracking object occur. Our algorithm is based on the perceptual hashing algorithm (PHA) and the color, low-frequency and rotation information are considered. By means of PHA, our tracker generates a single identification at each frame. The sliding windows produce a series of candidates between consecutive frames so that the new position of tracking object can be updated by comparing the binary code of candidates and identification. In the experiment, the quantitative and qualitative results are expressed by center location error(CLE) and VOC overlap ratio(VOR). Compared to the advanced tracker at present, PHA tracker shows its robustness when confronting violent changes of noise, illumination, background clutter and part occlusion, which demonstrates its state-of-the-art performance in the field of dynamic vehicle tracking.

The inverse synthetic aperture lidar (ISAL) have attracted increasing attention for its merits including small visual tracking which is considered as one of the important research topics in the field of computer vision due to its key role in versatile applications, such as precision guidance, intelligent video surveillance, human-computer interaction, robot navigation and public safety. The basic idea for implementing visual tracking is composed of finding the target object in a video or sequence of images, then determining its exact position in the next successive frames and finally generating the corresponding trajectory of this object. Visual tracking, however, is still a challenging problem in practice while taking into account the abrupt appearance changes of the target objects induced by their non-rigid transformation, the sophisticated lighting variation, the obstruction by the block or similar objects in the background and the camera jitter. Motivated by the successful applications in target detection and recognition in recent years, plenty of deep learning models have been integrated in the visual tracking and better performance over traditional methods was achieved in a series of data evaluations, which opens a new door in the field of visual tracking. In this paper, the overview and progress on visual tracking were summarized. The current challenges and corresponding solving approaches in this field are introduced firstly and in particular, several novel and mainstream visual tracking algorithms based on the deep learning are specially described and analyzed in details, including their basic ideas, advantages and disadvantages and future prospect.

In this paper, we propose a visual tracking method based on spatio-temporal context model, to solve the fast and stable target tracking problem with large scale changes, which is suitable for unmanned rotorcrafts high precision fixed-pointed landing using vision-based navigation. Based on the Bayesian outline, we express the spatio-temporal connections between the interest object and local context. The local context represents the statistical correlation between the target features and its adjacent areas. Then, by integrating this spatio-temporal context model and focusing properties of biological vision system, we compute a confidence map and obtain the best target location by maximizing an object location likelihood function. This paper presents a rapid model learning and object detection technique via Fast Fourier Transform (FFT), and proposes a multiscale estimation method to resolve the large scale changes problem during fix-pointed landing. Experimental results show that the proposed algorithm for visual tracking has respectable performance during partial occlusion, scale changes, lighting changes, deformation, motion blur and background clutter. In terms of efficiency, accuracy and robustness, our method meets the requirements of embedded image processing system in UAVs.

We present a new descriptor-sequence model for action recognition that enhances discriminative power in the spatio-temporal context, while maintaining robustness against background clutter as well as variability in inter-/intra-person behavior. We extend the framework of Dense Trajectories based activity recognition (Wang et al., 2011) and introduce a pool of dynamic Bayesian networks (e.g., multiple HMMs) with histogram descriptors as codebooks of composite action categories represented at respective key points. The entire codebooks bound with spatio-temporal interest points constitute intermediate feature representation as basis for generic action categories. This representation scheme is intended to serve as visual code-sentences which subsume a rich vocabulary of basis action categories. Through extensive experiments using KTH, UCF Sports, and Hollywood2 datasets, we demonstrate some improvements over the state-of-the-art methods.

Lane and road recognition are essential for self-driving where GPS solution is inaccurate due to the signal block or multipath in an urban environment. Vision based lane or road recognition algorithms have been studied extensively, but they are not robust to changes in weather or illumination due to the characteristic of the sensor. Lidar is a sensor for measuring distance, but it also contains intensity information. The road mark on the road is made to look good with headlight at night by using a special paint with good reflection on the light. With this feature, road marking can be detected with lidar even in the case of changes in illumination due to the rain or shadow. In this paper, we propose equipping autonomous cars with sensor fusion algorithms intended to operate in a different weather conditions. The proposed algorithm was applied to the self-driving car EureCar (KAIST) in order to test its feasibility for real-time use.

Video object segmentation and tracking are two essential building blocks of smart surveillance systems. However, there are several issues that need to be resolved. Threshold decision is a difficult problem for video object segmentation with a multi-background model. In addition, some conditions make robust video object tracking difficult. These conditions include nonrigid object motion, target appearance variations due to changes in illumination, and background clutter. In this paper, a video object segmentation and tracking framework is proposed for smart cameras in visual surveillance networks with two major contributions. First, we propose a robust threshold decision algorithm for video object segmentation with a multi-background model. Second, we propose a video object tracking framework based on a particle filter with the likelihood function composed of diffusion distance for measuring color histogram similarity and motion clue from video object segmentation. The proposed framework can track nonrigid moving objects under drastic changes in illumination and background clutter. Experimental results show that the presented algorithms perform well for several challenging sequences, and our proposed methods are effective for the aforementioned issues.

We have examined visual functions, including colour vision, in a patient with bilateral cortical lesions involving mainly the fusiform and lingual gyri, areas known to be involved in the central processing of chromatic stimuli. The patient has near normal (6/9) acuity, and his responses to tests of binocular function and spatial vision are normal, as are his discrimination of changes in target speed and surface lightness. He does, however, exhibit minor losses in the upper visual field, mild prosopagnosia and topographical agnosia, all conditions commonly associated with cerebral achromatopsia. Colour matches and spectral response data establish that his cone photoreceptors have normal spectral characteristics and his spectral sensitivity measured against a white background reveals normal postreceptoral chromatic function. The patient's colour discrimination for differences in wavelength, hue or saturation is, however, impaired and his colour naming is significantly disturbed, particularly for blues and greens. We have determined the areas of the chromaticity chart that correspond to his naming categories for surface colours, and show that changes in illuminant cause him to alter the names of surface colours in a manner consistent with the changes in their chromaticities. Other subjects with normal or congenital red-green deficient colour vision make many fewer name changes under changes in illuminant. We conclude that the patient's colour constancy is impaired as a consequence of abnormal central processing of colour vision.

With image analysis as the core for multitarget detection and intelligent tracking, mostly applying the Faster R-CNN or YOLO framework, the MOTA score for multitarget tracking is low in the face of complex working environments. Therefore, further research into computer vision techniques is carried out to design new multitarget detection and intelligent tracking methods. Based on the small-aperture imaging model, the principle of lens distortion was analyzed, and a camera calibration and image calibration scheme was designed to obtain effective environmental images. The attention mechanism is introduced to optimise the structure of deep learning networks, and a computer vision detection algorithm based on this is applied to complete regional multitarget detection. The distance between each target and the body is then measured in combination with binocular vision principles. Finally, the spatiotemporal context algorithm is applied to perform simulation calculations to obtain the multitarget intelligent tracking results. The experimental results show that the mean MOTA score of the proposed technique is 0.87 in the night environment, which is 24.14% and 28.374% better than the neural network-based and machine vision-based tracking methods, respectively; in the daytime environment, the mean MOTA score of the multitarget tracking results of the technique is 0.94, which is 28.72%, and the mean MOTA score of 0.94 for the multitarget tracking results in the daytime environment was 28.72% and 22.34% higher than the other two methods.

In order to realize robust visual tracking in natural environments, a novel algorithm based on adaptive appearance model is proposed. The model can adapt to changes in object appearance over time. A mixture of three Gaussian distributions models the value of each pixel. An online Expectation Maximization (EM) algorithm is developed to update the parameters of the Gaussians. The observation model in the particle filter is designed based on the adaptive appearance model. Numerous experimental results demonstrate that our proposed algorithm can track objects well under illumination change, large pose variation, and partial or full occlusion.Keywordsvisual trackingadaptive appearance modelExpectation Maximization algorithmparticle filter

In this paper, we propose a novel background subtraction method which enables reliable detection of foreground objects in a long surveillance video stream. Recently, although much progress has been made in the field of background subtraction, there are still challenging scenarios (e.g., high frequency motion of dynamic texture, non-stationary motion of camera, abrupt changes of illumination, etc.) in the long surveillance videos in which even state-of-the-art methods are often prone to fail. To cope with these challenging scenarios effectively, in the proposed method, a background model is initialized in a low-dimensional subspace and then updated periodically based on a novel recursive on-line ${(2{\rm D})}^2{\rm PCA}$ algorithm developed in this paper. Moreover, a threshold map is also updated in a scene-adaptive manner for labeling each pixel in a scene either foreground or background independently. Based on this on-line framework, the background of a surveillance video stream is reconstructed over time, thereby facilitating the detection of foreground objects reliably. In extensive experiments, we demonstrate that the proposed background subtraction method can cope with the aforementioned challenging scenarios more favorably than the state-of-the-art methods.

In this paper, a novel ‘co-superpixel’ generation method is proposed via the graph matching. The co-superpixel can capture the common semantic information in coupled images. Therefore, it is significant for various applications in visual pattern recognition. Specifically, we first introduce a superpixel correspondence method based on the graph matching. The main property is that it has the ability to capture the consistent intermediate-level semantic information in coupled images, which can represent the region-based similarity rather than the conventional similarity based on low-level vision features. Second, a new co-superpixel generation method is proposed by the superpixel-merging incorporated with the graph matching cost and the adjacent superpixel appearance similarity in coupled images simultaneously. Furthermore, we extend the proposed co-superpixel method to tackle the object matching problem. The experimental results show that the object matching can be effectively addressed by the co-superpixel. The proposed method is effective for challenging cases in which object appearance changes, deformation and background clutter.