MVR: Multimodule visual reasoner-assisted small object tracking with a remote sensing-based UAV

Appearance change of moving targets is a challenging problem in visual tracking. In this paper, we present a novel visual object tracking algorithm based on the observation dependent hidden Markov model (OD-HMM) framework. The observation dependency is computed by structure complexity coefficients (SCC) which is defined to predict the target appearance change. Unlike conventional methods addressing the appearance change problem by investigating different online appearance models, we handle this problem by addressing the fundamental reason of motion -related appearance change during visual tracking. Based on the analysis of motion-related appearance change, we investigate the relationship between the structure of the object surface and the appearance stability. The appearance of complex structural regions is easier to change compared with that of smooth structural regions with object moving. Based on this, we define SCC to predict the appearance stability of moving objects. Different from the standard HMM-based tracking algorithms where observations between different frames are assumed to be independent, we consider the observation dependency between consecutive frames with the information provided by SCC. Moreover , we present a novel outlier removing method in appearance model updating which helps to avoid error accumulation. Experimental results on challenging video sequences demonstrate that the proposed visual tracking algorithm with OD-HMM and SCC achieves better performance than existing related tracking algorithms.

As a fundamental task in computer vision, visual object tracking has received much attention in recent years. Most studies focus on short-term visual tracking which addresses shorter videos and always-visible targets. However, long-term visual tracking is much closer to practical applications with more complicated challenges. There exists a longer duration such as minute-level or even hour-level in the long-term tracking task, and the task also needs to handle more frequent target disappearance and reappearance. In this paper, we provide a thorough review of long-term tracking, summarizing long-term tracking algorithms from two perspectives: framework architectures and utilization of intermediate tracking results. Then we provide a detailed description of existing benchmarks and corresponding evaluation protocols. Furthermore, we conduct extensive experiments and analyse the performance of trackers on six benchmarks: VOTLT2018, VOTLT2019 (2020/2021), OxUvA, LaSOT, TLP and the long-term subset of VTUAV-V. Finally, we discuss the future prospects from multiple perspectives, including algorithm design and benchmark construction. To our knowledge, this is the first comprehensive survey for long-term visual object tracking. The relevant content is available at https://github.com/wangdong-dut/Long-term-Visual-Tracking.

Visual tracking has attracted a significant attention in the last few decades. The recent surge in the number of publications on tracking-related problems have made it almost impossible to follow the developments in the field. One of the reasons is that there is a lack of commonly accepted annotated data-sets and standardized evaluation protocols that would allow objective comparison of different tracking methods. To address this issue, the Visual Object Tracking (VOT) workshop was organized in conjunction with ICCV2013. Researchers from academia as well as industry were invited to participate in the first VOT2013 challenge which aimed at single-object visual trackers that do not apply pre-learned models of object appearance (model-free). Presented here is the VOT2013 benchmark dataset for evaluation of single-object visual trackers as well as the results obtained by the trackers competing in the challenge. In contrast to related attempts in tracker benchmarking, the dataset is labeled per-frame by visual attributes that indicate occlusion, illumination change, motion change, size change and camera motion, offering a more systematic comparison of the trackers. Furthermore, we have designed an automated system for performing and evaluating the experiments. We present the evaluation protocol of the VOT2013 challenge and the results of a comparison of 27 trackers on the benchmark dataset. The dataset, the evaluation tools and the tracker rankings are publicly available from the challenge website (http://votchallenge.net).

Attention shake siamese network with auxiliary relocation branch for visual object tracking

Chapter 10 - Long-term deep object tracking

Visual object tracking technology is one of the key issues in computer vision. In this paper, we propose a visual object tracking algorithm based on cross-modality featuredeep learning using Gaussian-Bernoulli deep Boltzmann machines (DBM) with RGB-D sensors. First, a cross-modality featurelearning network based on aGaussian-Bernoulli DBM is constructed, which can extract cross-modality features of the samples in RGB-D video data. Second, the cross-modality features of the samples are input into the logistic regression classifier, andthe observation likelihood model is established according to the confidence score of the classifier. Finally, the object tracking results over RGB-D data are obtained using aBayesian maximum a posteriori (MAP) probability estimation algorithm. The experimental results show that the proposed method has strong robustness to abnormal changes (e.g., occlusion, rotation, illumination change, etc.). The algorithm can steadily track multiple targets and has higher accuracy.

Subject of the research – visual object tracking using various image hashing algorithms for real-time tracking tasks. The goal of this study is to evaluate the tracking success and processing speed of existing and new hashing algorithms for object tracking and to identify the most suitable algorithms to be used under limited computational resources. The objectives of the research include: developing and implementing object tracking based on the aHash, dHash, pHash, mHash, LHash, and LDHash algorithms; comparing the processing speed and accuracy of these methods on the video sequences "OccludedFace2," "David," and "Sylvester"; determining the tracking success rate (TSR) and frames per second (FPS) metrics for each algorithm; analyzing the impact of the search window size, search strategy, and type of hashing on tracking quality, and providing recommendations for their use. The study also explores the trade-off between accuracy and processing speed for each algorithm considering the constraints of limited computational resources. The methods of this study involve testing and evaluating the accuracy and speed of image hashing algorithms on different test video sequences, as well as the use of metrics to determine object similarity using the Hamming distance. The results demonstrate that the aHash and mHash algorithms demonstrate the best accuracy indicators for all hash window sizes, aHash has a higher processing speed, and mHash offers better robustness to changes in lighting and object position. The dHash and pHash algorithms were less effective than the aHash and mHash algorithms due to their sensitivity to changes in scale and rotation. However, perceptual hashing-based methods, such as pHash, are more robust to contrast and blurring. Conclusions. The best hashing algorithms for real-time object-tracking tasks are aHash and mHash. This study underscores the significance of selecting suitable hashing algorithms and search strategies tailored to specific application scenarios and offers possibilities for further optimization.

Object detection and tracking are vital in computer vision and visual surveillance, allowing for the detection, recognition, and subsequent tracking of objects within images or video sequences. These tasks underpin surveillance systems, facilitating automatic video annotation, identification of significant events, and detection of abnormal activities. However, detecting and tracking small objects introduce significant challenges within computer vision due to their subtle appearance and limited distinguishing features, which results in a scarcity of crucial information. This deficit complicates the tracking process, often leading to diminished efficiency and accuracy. To shed light on the intricacies of small object detection and tracking, we undertook a comprehensive review of the existing methods in this area, categorizing them from various perspectives. We also presented an overview of available datasets specifically curated for small object detection and tracking, aiming to inform and benefit future research in this domain. We further delineated the most widely used evaluation metrics for assessing the performance of small object detection and tracking techniques. Finally, we examined the present challenges within this field and discussed prospective future trends. By tackling these issues and leveraging upcoming trends, we aim to push forward the boundaries in small object detection and tracking, thereby augmenting the functionality of surveillance systems and broadening their real-world applicability.

Visual object tracking is a critical problem in the field of computer vision. The visual object tracker methods can be divided into Correlation Filters (CF) and non-correlation filters trackers. The main advantage of CF-based trackers is that they have an accepted real-time tracking response. In this article, we will focus on CF-based trackers, due to their key role in online applications such as an Unmanned Aerial Vehicle (UAV), through two contributions. In the first contribution, we proposed a set of new video sequences to address two uncovered issues of the existing standard datasets. The first issue is to create two video sequence that is difficult to be tracked by a human being for the movement of the Amoeba under the microscope; these two proposed video sequences include a new feature that combined background clutter and occlusion features in a unique way; we called it hard-to-follow-by-human. The second issue is to increase the difficulty of the existing sequences by increasing the displacement of the tracked object. Then, we proposed a thorough, practical evaluation of eight CF-base trackers, with the top performance, on the existing sequence features such as out-of-view, background clutters, and fast motion. The evaluation utilized the well-known OTB-2013 dataset as well as the proposed video sequences. The overall assessment of the eight trackers on the standard evaluation metrics, e.g., precision and success rates, revealed that the Large Displacement Estimation of Similarity transformation (LDES) tracker is the best CF-based tracker among the trackers of comparison. On the contrary, with a deeper analysis, the results of the proposed video sequences show an average performance of the LDES tracker among the other trackers. The eight trackers failed to capture the moving objects in every frame of the proposed Amoeba movement video sequences while the same trackers managed to capture the object in almost every frame of the sequences of the standard dataset. These results outline the need to improve the CF-based object trackers to be able to process sequences with the proposed feature (i.e., hard-to-follow-by-human).

Visual object tracking has always been a research hotspot in the field of computer vision. Recently, some excellent techniques in video object detection can be used to improve the performance of visual object trackers. Therefore, we propose a new Siamese-like network tracker by leveraging the superior representation ability of the recent visual object detection algorithm MLP-Mixer. Specifically, the MLP-Mixer based backbone network is used for feature extraction, which can enhance the feature representation. Furthermore, to connect the target template and the search region, we adopt cross-attention instead of conventional correlation operation to avoid getting stuck in local optimization when predicting target location. Finally, in order to demonstrate the effectiveness of the proposed method, we conduct extensive experiments on four tracking benchmarks, including OTB2015, VOT2016, VOT201S and GOT-10k datasets. Experimental results show that our proposed SiamL-MLP achieves competitive performance on the OTB2015, VOT2016, VOT201S and GOT-10k datasets. Our tracker runs at approximatively 40 FPS on GPU. Code will be available at https://github.com/SilvesterSun/SiamL-MLP.

This work deals with visual object tracking. The well known discriminant correlation filter (DCF) based approach is improved by multi-layer CNN features, spatial reliability (through a foreground mask) and conditionally model updating strategy. In the training stage, by calculating a foreground mask using the color histograms, for each chosen CNN layer, a correlation filter is trained under the foreground constraint to construct a weak tracker. In next frame, the tracking position is from the weighting of weak trackers, for which the weights are computed by Hedge method. The response peak and oscillation are both considered to estimate the confidence criteria. The model and weight of each weak tracker are updated only when the tracking is high-confident. We analyze and evaluate our system on OTB-13 dataset, and show that our approach performs superiorly against several state-of-the-art methods.

Visual object tracking is a classical, but still open research problem in computer vision, with many real world applications. The problem is challenging due to several factors, such as illumination variation, occlusions, camera motion and appearance changes. Such problems can be alleviated by constructing robust, discriminative and computationally efficient visual features. Recently, biologically-inspired channel representations [9] have shown to provide promising results in many applications ranging from autonomous driving to visual tracking.

Visual tracking has recently gained a great advance with the use of the convolutional neural network (CNN). Usually, existing CNN‐based trackers exploit the features from a single layer or a certain combination of multiple layers. However, these features only characterise an object from an invariable aspect and cannot adapt to scene variation, which limits the performance of such trackers. To overcome this limitation, the authors study the problem from a new perspective and propose a novel convolutional layer selection method. To obtain robust appearance representation, they investigate the advantages of features extracted from different convolutional layers. To determine the correctness of the tracking prediction and updated model, they design a verification mechanism based on historical retrospect, which can estimate the deviation for each layer by bidirectionally locating the target. Meanwhile, the deviation works as the layer‐wise selection criteria. Extensive evaluations on the OTB‐2013, visual object tracking (VOT)‐2016 and VOT‐2017 benchmarks demonstrate that the proposed tracker performs favourably against several state‐of‐the‐art trackers.

Due to the effects of deformation, scale changes and background interference, visual tracking has always been one of the challenging problems. In recent years, kernelized correlation filter (KCF) has attracted much attention. However, the tracking accuracy of KCF tracker is poor when the object fast motion, heavy occlusion and scale variation appear. Herein, this reason is analyzed that the KCF tracker generates candidate patches of object according to its location in the last frame. By establishing grey prediction model of object motion, we produce the candidate patches around the predicted location. Moreover, we apply an adaptive strategy to update model and employ a simple mechanism for scale estimation. We conduct our tracker on some tracking benchmarks, including OTB2013, VOT2016 and UAV123. Experimental results demonstrate the proposed tracking method has good and stable results.

In recent years, visual object tracking has become a very active research field which is mainly divided into the correlation filter-based tracking and deep learning (e.g., deep convolutional neural network and Siamese neural network) based tracking. For target tracking algorithms based on deep learning, a large amount of computation is required, usually deployed on expensive graphics cards. However, for the rich monitoring devices in the Internet of Things, it is difficult to capture all the moving targets in each device in real time, so it is necessary to perform hierarchical processing and use tracking based on correlation filtering in insensitive areas to alleviate the local computing pressure. In sensitive areas, upload the video stream to a cloud computing platform with a faster computing speed to perform an algorithm based on deep features. In this paper, we mainly focus on the correlation filter-based tracking. In the correlation filter-based tracking, the discriminative scale space tracker (DSST) is one of the most popular and typical ones which is successfully applied to many application fields. However, there are still some improvements that need to be further studied for DSST. One is that the algorithms do not consider the target rotation on purpose. The other is that it is a very heavy computational load to extract the histogram of oriented gradient (HOG) features from too many patches centered at the target position in order to ensure the scale estimation accuracy. To address these two problems, we introduce the alterable patch number for target scale tracking and the space searching for target rotation tracking into the standard DSST tracking method and propose a visual object multimodality tracker based on correlation filters (MTCF) to simultaneously cope with translation, scale, and rotation in plane for the tracked target and to obtain the target information of position, scale, and attitude angle at the same time. Finally, in Visual Tracker Benchmark data set, the experiments are performed on the proposed algorithms to show their effectiveness in multimodality tracking.