SiamL-MLP: Siamese-like Network for Object Tracking Based on MLP-Mixer

During past few years’ significant usage of video monitoring has been observed in several places. Video monitoring has always helped in improvement of security specific aspects. These systems provide major helping hand in monitoring through multiple object detection methods. However, there are certain challenges related to weather conditions which needs to be addressed. In view of this in this paper, we study multiple object detection in surveillance videos through hybrid deep gaussian mixture ensembles. This method integrates multiple deep gaussian mixture components to form ensembles which helps to develop multiple object detection solutions in surveillance videos. The computational system consists of ensemble of three major deep gaussian mixture components. The ensemble pipeline is formed with augmenting, bagging and stacking methods. The ensemble consists of hybrid object detection paradigm with steps background analysis, video pre-processing, data integration, object detection, foreground detection and post-processing. The computational pipeline is successfully experimented with ViSOR and CDnet 2014 benchmarked datasets. All results are validated with accuracy, precision, sensitivity or recall, specificity, F1-Score and RMSE metrics. Several comparative studies are performed with state-of-the-art methods as well as baselines. The experimental results demonstrate superiority of this method in comparison with other methods. Ablation studies have also been performed here with superior results. This system presents robustness in multiple object detection for real life challenges such as sudden illumination variation, shadow presence, long term occlusion and formation of dynamic backgrounds. It provides cost-effective, more profitable, efficient and sustainable real time multiple object detection solutions

Given an object model and a black-box measure of similarity between the model and candidate targets, we consider visual object tracking as a numerical optimization problem. During normal tracking conditions when the object is visible from frame to frame, local optimization is used to track the local mode of the similarity measure in a parameter space of translation, rotation and scale. However, when the object becomes partially or totally occluded, such local tracking is prone to failure, especially when common prediction techniques like the Kalman filter do not provide a good estimate of object parameters in future frames. To recover from these inevitable tracking failures, we consider object detection as a global optimization problem and solve it via Adaptive Simulated Annealing (ASA), a method that avoids becoming trapped at local modes and is much faster than exhaustive search. As a Monte Carlo approach, ASA stochastically samples the parameter space, in contrast to local deterministic search. We apply cluster analysis on the sampled parameter space to redetect the object and renew the local tracker. Our numerical hybrid local and global mode-seeking tracker is validated on challenging airborne videos with heavy occlusion and large camera motions. Our approach outperforms state-of-the-art trackers on the VIVID benchmark datasets.

Visual object tracking is a critical problem in the field of computer vision. The visual object tracker methods can be divided into Correlation Filters (CF) and non-correlation filters trackers. The main advantage of CF-based trackers is that they have an accepted real-time tracking response. In this article, we will focus on CF-based trackers, due to their key role in online applications such as an Unmanned Aerial Vehicle (UAV), through two contributions. In the first contribution, we proposed a set of new video sequences to address two uncovered issues of the existing standard datasets. The first issue is to create two video sequence that is difficult to be tracked by a human being for the movement of the Amoeba under the microscope; these two proposed video sequences include a new feature that combined background clutter and occlusion features in a unique way; we called it hard-to-follow-by-human. The second issue is to increase the difficulty of the existing sequences by increasing the displacement of the tracked object. Then, we proposed a thorough, practical evaluation of eight CF-base trackers, with the top performance, on the existing sequence features such as out-of-view, background clutters, and fast motion. The evaluation utilized the well-known OTB-2013 dataset as well as the proposed video sequences. The overall assessment of the eight trackers on the standard evaluation metrics, e.g., precision and success rates, revealed that the Large Displacement Estimation of Similarity transformation (LDES) tracker is the best CF-based tracker among the trackers of comparison. On the contrary, with a deeper analysis, the results of the proposed video sequences show an average performance of the LDES tracker among the other trackers. The eight trackers failed to capture the moving objects in every frame of the proposed Amoeba movement video sequences while the same trackers managed to capture the object in almost every frame of the sequences of the standard dataset. These results outline the need to improve the CF-based object trackers to be able to process sequences with the proposed feature (i.e., hard-to-follow-by-human).

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

Soft Computing based object detection and tracking approaches: State-of-the-Art survey

In video object detection, the deterioration of an object’s appearance in a single frame brings challenges for recognition; therefore, it is natural to exploit temporal information to boost the robustness of video object detection. Existing methods usually utilize temporal information to enhance features, often ignoring the information in label assignments. Label assignment, which assigns labels to anchors for training, is an essential part of object detection. It is also challenged in video object detection and can be improved by temporal information. In this work, a temporal-guided label assignment framework is proposed for the learning task of a region proposal network (RPN). Specifically, we propose a feature instructing module (FIM) to establish the relation model among labels through feature similarity in the temporal dimension. The proposed video object detection framework was evaluated on the ImageNet VID benchmark. Without any additional inference cost, our work obtained a 0.8 mean average precision (mAP(%)) improvement over the baseline and achieved a mAP(%) of 82.0. The result was on par with the state-of-the-art accuracy without using any post-processing methods.

Salient object detection in video has attracted enormous research efforts for its wide applicability. But there are still some issues in restraining the disturbance of background, which make it difficult to detect salient object in complex scenarios. Inspired by the hypothesis of center prior in image domain, we novelly introduced the concept of dynamic attention center in video. The distance between specific regions and this center is used as a weighting term to restrain the influence of background disturbance and obtain more accurate spatial and temporal saliency maps. Besides, we develop a dynamic fusion method to combine the temporal and spatial saliency map, leading to higher spatiotemporal consistency. The experiments on Freiburg-Berkeley Motion Segmentation Dataset show that our method outperforms several state-of-art methods on subjective visual perception and objective measurements.

Object detection in videos is widely used in entertainment, robotics, surveillance etc. A major challenge in effective object detection is observed when there is occlusion, bad illumination or cluttered background. Besides, cameras do not provide any mechanism for detecting moving objects after capture. Researchers have proposed different methods for object detection in video frames, ranging from traditional to deep learning approaches. However, using the right method in the right situation for efficient and accurate detection is a concern. Even though deep learning has shown high accuracy in detection, the training and testing time and cost is a concern. In our research, we explore the best motion models for tracking objects in a video sequence. The characteristics of each model was analyzed and the results were compared using different types of videos. Our observations determine that depending on the quality of input video, traditional approaches show high accuracy in detection, comparable to state-of-the art methods.

Siamese network based trackers have achieved significant progress in visual object tracking. For the sake of speed, they mainly rely on offline training to learn a mono-level feature correlation between a target template and a search region. During the tracking period, they use a fixed strategy to infer target positions over sequences regardless of target states. However, such approaches are vulnerable in case of long-term challenges e.g. large variance, presence of distractors, fast motion, or target disappearing and the like. In this paper, we propose a new tracking framework, referred to as SiamX, by exploiting cross-level Siamese features to learn robust correlations between the target template and search regions, and also adaptive inference strategies to prevent tracking loss and realize fast target re-localization. Extensive experiments on four benchmarks including VOT-2019, LaSOT, GOT-10k, and TrackingNet show our method significantly enhances the tracker's ability to resist variance and interference, and achieve state-of-the-art results at around 50 FPS.

<p>The ultimate goal of this study is to afford enhanced video object detection and tracking by eliminating the limitations which are existing nowadays. Although high performance ratio for video object detection and tracking is achieved in the earlier work it takes more time for computation. Consequently we are in need to propose a novel video object detection and tracking technique so as to minimize the computational complexity. Our proposed technique covers five stages they are preprocessing, segmentation, feature extraction, background subtraction and hole filling. Originally the video clip in the database is split into frames. Then preprocessing is performed so as to get rid of noise, an adaptive median filter is used in this stage to eliminate the noise. The preprocessed image then undergoes segmentation by means of modified region growing algorithm. The segmented image is subjected to feature extraction phase so as to extract the multi features from the segmented image and the background image, the feature value thus obtained are compared so as to attain optimal value, consequently a foreground image is attained in this stage. The foreground image is then subjected to morphological operations of erosion and dilation so as to fill the holes and to get the object accurately as these foreground image contains holes and discontinuities. Thus the moving object is tracked in this stage. This method will be employed in MATLAB platform and the outcomes will be studied and compared with the existing techniques so as to reveal the performance of the novel video object detection and tracking technique. </p>

Visual tracking has attracted a significant attention in the last few decades. The recent surge in the number of publications on tracking-related problems have made it almost impossible to follow the developments in the field. One of the reasons is that there is a lack of commonly accepted annotated data-sets and standardized evaluation protocols that would allow objective comparison of different tracking methods. To address this issue, the Visual Object Tracking (VOT) workshop was organized in conjunction with ICCV2013. Researchers from academia as well as industry were invited to participate in the first VOT2013 challenge which aimed at single-object visual trackers that do not apply pre-learned models of object appearance (model-free). Presented here is the VOT2013 benchmark dataset for evaluation of single-object visual trackers as well as the results obtained by the trackers competing in the challenge. In contrast to related attempts in tracker benchmarking, the dataset is labeled per-frame by visual attributes that indicate occlusion, illumination change, motion change, size change and camera motion, offering a more systematic comparison of the trackers. Furthermore, we have designed an automated system for performing and evaluating the experiments. We present the evaluation protocol of the VOT2013 challenge and the results of a comparison of 27 trackers on the benchmark dataset. The dataset, the evaluation tools and the tracker rankings are publicly available from the challenge website (http://votchallenge.net).

Due to the effects of deformation, scale changes and background interference, visual tracking has always been one of the challenging problems. In recent years, kernelized correlation filter (KCF) has attracted much attention. However, the tracking accuracy of KCF tracker is poor when the object fast motion, heavy occlusion and scale variation appear. Herein, this reason is analyzed that the KCF tracker generates candidate patches of object according to its location in the last frame. By establishing grey prediction model of object motion, we produce the candidate patches around the predicted location. Moreover, we apply an adaptive strategy to update model and employ a simple mechanism for scale estimation. We conduct our tracker on some tracking benchmarks, including OTB2013, VOT2016 and UAV123. Experimental results demonstrate the proposed tracking method has good and stable results.

Chaotic particle filter for visual object tracking

Road objects (such as pedestrians and vehicles) detection is a very important step to enhance road safety and achieve autonomous driving. Many on-vehicle sensors, such as radars, lidars and ultrasonic sensors, are used to detect surrounding objects. However, cameras are widely used sensors for road objects detection for the rich information they provide and their inexpensive prices with compared to other sensors. Machine learning and computer vision algorithms are utilized to classify objects in the collected images and videos. There are many computer vision algorithms proposed for image and video object detection, e.g. logistic regression and SVM with feature extraction. However, Convolutional Neural Network (CNN) al-gorithms showed a high detection accuracy compared to other approaches. This research implements You Only Look Once (YOLO) algorithm that uses Draknet-53 CNN to detect four classes: pedestrians, vehicles, trucks and cyclists. The model is trained using Kitti images dataset which is collected from public roads using vehicle’s front looking camera. The algorithm is tested, and detection results are presented.

In recent years, visual object tracking has become a very active research field which is mainly divided into the correlation filter-based tracking and deep learning (e.g., deep convolutional neural network and Siamese neural network) based tracking. For target tracking algorithms based on deep learning, a large amount of computation is required, usually deployed on expensive graphics cards. However, for the rich monitoring devices in the Internet of Things, it is difficult to capture all the moving targets in each device in real time, so it is necessary to perform hierarchical processing and use tracking based on correlation filtering in insensitive areas to alleviate the local computing pressure. In sensitive areas, upload the video stream to a cloud computing platform with a faster computing speed to perform an algorithm based on deep features. In this paper, we mainly focus on the correlation filter-based tracking. In the correlation filter-based tracking, the discriminative scale space tracker (DSST) is one of the most popular and typical ones which is successfully applied to many application fields. However, there are still some improvements that need to be further studied for DSST. One is that the algorithms do not consider the target rotation on purpose. The other is that it is a very heavy computational load to extract the histogram of oriented gradient (HOG) features from too many patches centered at the target position in order to ensure the scale estimation accuracy. To address these two problems, we introduce the alterable patch number for target scale tracking and the space searching for target rotation tracking into the standard DSST tracking method and propose a visual object multimodality tracker based on correlation filters (MTCF) to simultaneously cope with translation, scale, and rotation in plane for the tracked target and to obtain the target information of position, scale, and attitude angle at the same time. Finally, in Visual Tracker Benchmark data set, the experiments are performed on the proposed algorithms to show their effectiveness in multimodality tracking.