Adaptive pyramid mean shift for global real-time visual tracking

Mean shift algorithm has grained great success in object tracking domain due to its ease of implementation, real time response and robust tracking performance, however, the fixed kernel bandwidth may cause tracking failure for size changing objects. A novel object tracking algorithm for FLIR imagery is proposed based on mean shift with adaptive bandwidth. The scale invariant feature transform is employed to compute the affine model between the successive frames. Then, the scale and orientation of the kernel can be estimated by the gained parameters. Experiment results verify the effectives and robustness of this extraction algorithm which can improve the tracking performance efficiently.

In this paper, we improve the real-time object tracking algorithm of Yang [1] which uses a symmetric similarity function between spatially smoothed kernel-density estimates of the model and the target distributions. This spatial smoothed process applied on the centre points of the probability density functions increases not only computational complexity but also noise sensitivity. After reducing background information and using a new simple similarity function between the model and the target distributions, the proposed algorithm successfully coped with camera motion, partial occlusions and clutter with lower computational complexity. For tracking object, the mean shift algorithm is used iteratively. The simplicity of the new similarity function leads to an efficient and robust nonparametric tracking algorithm. The mathematical results obtained about the performance of the proposed algorithm on several image sequences are represented comparatively in a tabular format.

Open-vocabulary Video Instance Segmentation addresses the challenging task of detecting, segmenting, and tracking objects in videos, including categories not encountered during training. However, existing approaches often overlook rich temporal cues from preceding frames, limiting their ability to leverage causal context for robust open-world generalization. To bridge this gap, we propose CPOVIS, a novel framework that introduces causal prompts-dynamically propagated visual and taxonomy prompts from historical frames-to enhance temporal reasoning and semantic consistency. Built upon a Mask2Former architecture with a CLIP backbone, CPOVIS integrates three core innovations: (1) PromptCLIP, which aligns cross-modal embeddings while preserving open-vocabulary capabilities; (2) a Visual Prompt Injector that propagates object-level features to maintain spatial-temporal coherence; and (3) a Taxonomy Prompt Infuser that leverages hierarchical semantic relationships to stabilize unseen category recognition. Furthermore, we introduce a contrastive learning strategy to disentangle object representations across frames and adapt the Segment Anything Model (SAM2) to boost open-vocabulary segmentation and tracking capacity in open-vocabulary video scenarios. Extensive experiments on seven challenging open- and closed-vocabulary video segmentation benchmarks demonstrate CPOVIS's state-of-the-art performance, outperforming existing methods by significant margins. Our findings highlight the critical role of causal prompt propagation in advancing video understanding in open-world scenarios.

The classical mean shift (MS) algorithm is the best color-based method for object tracking. However, in the real environment it presents some limitations, especially under the presence of noise, objects with partial and full occlusions in complex environments. In order to deal with these problems, this paper proposes a reliable object tracking algorithm using corrected background-weighted histogram (CBWH) and the Kalman filter (KF) based on the MS method. The experimental results show that the proposed method is superior to the traditional MS tracking in the following aspects: 1) it provides consistent object tracking throughout the video; 2) it is not influenced by the objects with partial and full occlusions; 3) it is less prone to the background clutter.

The work presented in this paper describes a novel algorithm for aut omatic video object tracking based on a process of subtraction of successive frames, where the prediction of the direction of movement of the object being tracked is carried out by analyzing the changing areas generated as result of the object’s motion, specifically in regions of interest defined inside the object being tracked in both the current and the next frame. Simultaneously, it is initiated a minimization process which seeks to determine the location of the object being tracked in the next frame usi ng a function which measures the grade of dissimilarity between the region of interest defined inside the object being tracked in the current frame and a moving region in a next frame. This moving region is displaced in the direction of the object’s motion predicted on the process of subtraction of successive frames. Finally, the location of the moving region of interest in the next frame that minimizes the proposed function of dissimilarity corresponds to the predicted location of the object being trackedin the next frame. On the other hand, it is also designed a testing platform which is used to create virtual scenarios that allow us to assess the performance of the proposed algorithm. These virtual scenarios are exposed to heavily cluttered conditions w here areas which surround the object being tracked present a high variability. The results obtained with the proposed algorithm show that the tracking process was successfully carried out in a set of virtual scenarios under different challenging conditions .

Video object segmentation and tracking are two essential building blocks of smart surveillance systems. However, there are several issues that need to be resolved. Threshold decision is a difficult problem for video object segmentation with a multi-background model. In addition, some conditions make robust video object tracking difficult. These conditions include nonrigid object motion, target appearance variations due to changes in illumination, and background clutter. In this paper, a video object segmentation and tracking framework is proposed for smart cameras in visual surveillance networks with two major contributions. First, we propose a robust threshold decision algorithm for video object segmentation with a multi-background model. Second, we propose a video object tracking framework based on a particle filter with the likelihood function composed of diffusion distance for measuring color histogram similarity and motion clue from video object segmentation. The proposed framework can track nonrigid moving objects under drastic changes in illumination and background clutter. Experimental results show that the presented algorithms perform well for several challenging sequences, and our proposed methods are effective for the aforementioned issues.

Object detection and tracking under complex environment are challenging because of the disturbances induced by background clutter, illumination changes, occlusions and other factors. The bulk of traditional algorithms basically rely on hand‐crafted features, which are not sufficiently robust to a complex environment. Moreover, the processes of detection and tracking are separated, which leads to the overall efficiency not high. In this study, a novel local probability model (LPM)‐based mean shift (MS) algorithm is proposed to integrate object detection and tracking. The main contributions include: (i) a new framework based on the combination of LPM and MS is established for the integration of object tracking and detection. (ii) For object detection, the training and prediction of LPM are built by stacked denoising autoencoders based deep learning. (iii) For object tracking, an MS tracking algorithm leveraging LPM is modified to improve the tracking efficiency under a complex environment. Experimental results demonstrate that the proposed method is superior to the colour histograms based MS and histograms of oriented gradients based MS in terms of robustness and tracking accuracy.

Object tracking is the main task in the field of computer vision. This chapter proposes and compares a new method for recognition and tracking by using CamShift and color detection. All the presented algorithms were programmed with Python programming language supported by OpenCV libraries, and executed with a credit card–size computer board called Raspberry Pi with attached external camera. A camera mimics and uses real-time digital videos for object recognition and tracking. The chapter also proposes a color detection algorithm in very low light conditions is proposed for the enhancement of target tracking. Mean shift is a nonparametric feature-space analysis technique for locating the maxima of a density function. The CamShift algorithm is dependent on an adaptation of mean shift algorithm. Histogram equalization is a nonlinear technique for adjusting the contrast of an image using its histogram.

This paper presents efficient object tracking in video sequences using multiple features by embedding mean shift into particle filters. When clutter background and occlusions are present. Particle filtering is used because it is very robust and performs well for non-linear and non-Gaussian dynamic state estimation problems. The image features, such as shape, texture, color, contours, and random motion appearance can be used to track the moving object(s) in videos. We proposed a smart video surveillance system with real-time as well as offline stored video database for moving object detection, classification and tracking capabilities. The graphical user interface of the proposed system using MATLAB 2012b is implemented that operates on both color and gray scale video images from a stationary camera. In this method, we proposed real time moving object detection and tracking system using static webcam that can processes 800*600, 640*480, and 320*240 resolution video sequences for capturing live scene as well as stored database of segmented videos. Issues related with object detection and tracking is jointly employed using mean shift and particle filters. Tracker will estimate the dynamic shape and random appearance of objects. Tracking requires location and shape of object in every segmented frame. Rectangular bounding box is used to improve the estimate of its shape and appearance. Target object may experience partial occlusions, intersection with other objects with similar color distributions, abrupt motion speed changes, and cluttered background. Test result shows improvement in terms of robustness, tracking drifts, accuracy of tracked bounding box to occlusions.

A sufficient condition for the convergence of the mean shift algorithm with Gaussian kernel

In order to solve the problems of motion blur and fast motion, a new robust object tracking algorithm using the Kernelized Correlation Filters (KCF) and the Mean Shift (MS) algorithm, called KCFMS is presented in this paper. The object tracking process can be described as: First, we give the initial position and size of the object and use the Mean Shift algorithm to obtain the position of the object. Second, the Kernelized Correlation Filtering algorithm is used to obtain the position of the object in the same frame. Third, we use the cross update strategy to update the object models. In order to improve the tracking speed as much as possible, our object tracking algorithm works only over one layer. This hybrid algorithm has a good tracking effect on the target fast motion and motion blur. We present extensive experimental results on a number of challenging sequences in terms of efficiency, accuracy and robustness.

Siamese network-based visual tracking algorithms have achieved excellent performance in recent years, but challenges such as fast target motion, shape and scale variations have made the tracking extremely difficult. The regression of anchor-free tracking has low computational complexity, strong real-time performance, and is suitable for visual tracking. Based on the anchor-free siamese tracking framework, this paper firstly introduces balance factors and modulation coefficients into the cross-entropy loss function to solve the classification inaccuracy caused by the imbalance between positive and negative samples as well as the imbalance between hard and easy samples during the training process, so that the model focuses more on the positive samples and the hard samples that make the major contribution to the training. Secondly, the intersection over union (IoU) loss function of the regression branch is improved, not only focusing on the IoU between the predicted box and the ground truth box, but also considering the aspect ratios of the two boxes and the minimum bounding box area that accommodate the two, which guides the generation of more accurate regression offsets. The overall loss of classification and regression is iteratively minimized and improves the accuracy and robustness of visual tracking. Experiments on four public datasets, OTB2015, VOT2016, UAV123 and GOT-10k, show that the proposed algorithm achieves the state-of-the-art performance.

Segmentation is a low-level processing aimed at the partition of an image in disjoint regions, each one homogeneous with respect to some properties like intensity, texture, shape, etc. Such a task is needed in many high-level processing and applications in such diverse fields as remote-sensing, medical imaging, video coding, and industrial automation, just to name a few. In this work of thesis, we first deal with the problem of color based image segmentation, that is based on some form of homogeneity in the color/spectral properties of the image, and then move to the more complex task of texture based segmentation, where the aim is to recognize complex structures in the image which are typically non homogeneous in terms of spectral properties. Concerning color based segmentation, we focus on a specific class of Markov Random Field (MRF) probability models, namely the Tree-Structured Markov Random Fields (TS-MRF), recently proposed to provide a hierarchical multiscale description of images. Based on such a model, the unsupervised image segmentation is carried out by means of a sequence of nested class splits, where each class is modeled as a local binary MRF. As a first contribution, we first extend the model to generic tree structures removing the unnecessary binary constraint, and then devise a new TS-MRF unsupervised segmentation technique which improves upon the original algorithm by selecting a better tree structure and eliminating spurious classes. Such results are obtained by using the Mean-Shift procedure to estimate the number of pdf modes at each node, and to obtain a more reliable initial clustering for subsequent MRF optimization. To this end, we devise a new reliable and fast clustering algorithm based on the Mean-Shift technique, which makes use of a variable-bandwidth strategy based on the k-Nearest Neighbors (k-NN) technique, and is implemented with a speed-up strategy that cuts significantly the computational complexity. Experimental results on synthetic data and real life images from the remote sensing domain prove the potential of the proposed method. Turning to texture based segmentation, we present a novel multiscale texture model, and a related algorithm for the unsupervised segmentation of color images. Elementary textures are characterized by their spatial interactions with neighboring regions along selected directions. Such interactions are modeled in turn by means of a set of Markov chains, one for each direction, whose parameters are collected in a feature vector that synthetically describes the texture. Based on the feature vectors, the texture are then recursively merged, giving rise to larger and more complex textures, which appear at different scales of observation: accordingly, the model is named Hierarchical Multiple Markov Chain (H-MMC). The Texture Fragmentation and Reconstruction (TFR) algorithm, addresses the unsupervised segmentation problem based on the H-MMC model. The ``fragmentation'' step allows one to find the elementary textures of the model, while the ``reconstruction'' step defines the hierarchical image segmentation based on a probabilistic measure (texture score) which takes into account both region scale and inter-region interactions. The performance of the proposed method was assessed through the Prague segmentation benchmark, based on mosaics of real natural textures, and also tested on real-world natural images from the Berkeley dataset. Finally, an application of the TFR algorithm for the unsupervised segmentation of multiresolution remotely sensed images is proposed, based on the significant presence of textural content in last generation high-resolution satellite images. The obtained results further assess the potential of the proposed method for such real life applications.

A great interest has been fueled in object tracking solutions, mainly because of factors such as an increase in the number of powerful computers, the availability of high-quality video cameras, and the need for automation in video analysis. Tracking plays a vital role in various application domains, such as traffic monitoring, object recognition, autonomous surveillance systems, defence etc. This paper conducts a concise review on some of the few important techniques that are being used for robust tracking of moving objects in videos, completely along with their merits and demerits. Also, a thorough performance analysis of popular techniques – Mean Shift, CamShift, Optical Flow, Kalman Filter and SURF – has been performed, and the results have been interpreted and illustrated.

Urban transportation systems encounter significant challenges in extracting meaningful traffic patterns from extensive historical datasets, a critical aspect of smart city initiatives. This paper addresses the challenge of analyzing and understanding these patterns by employing various clustering techniques on hourly urban traffic flow data. The principal aim is to develop a model that can effectively analyze temporal patterns in urban traffic, uncovering underlying trends and factors influencing traffic flow, which are essential for optimizing smart city infrastructure. To achieve this, we applied DBSCAN, K-Means, Affinity Propagation, Mean Shift, and Gaussian Mixture clustering techniques to the traffic dataset of Aarhus, Denmark's second-largest city. The performance of these clustering methods was evaluated using the Silhouette Score and Dunn Index, with DBSCAN emerging as the most effective algorithm in terms of cluster quality and computational efficiency. The study also compares the training times of the algorithms, revealing that DBSCAN, K-Means, and Gaussian Mixture offer faster training times, while Affinity Propagation and Mean Shift are more computationally intensive. The results demonstrate that DBSCAN not only provides superior clustering performance but also operates efficiently, making it an ideal choice for analyzing urban traffic patterns in large datasets. This research emphasizes the importance of selecting appropriate clustering techniques for effective traffic analysis and management within smart city frameworks, thereby contributing to more efficient urban planning and infrastructure development.