Layer-based object detection and tracking with graph matching

Object detection and tracking under complex environment are challenging because of the disturbances induced by background clutter, illumination changes, occlusions and other factors. The bulk of traditional algorithms basically rely on hand‐crafted features, which are not sufficiently robust to a complex environment. Moreover, the processes of detection and tracking are separated, which leads to the overall efficiency not high. In this study, a novel local probability model (LPM)‐based mean shift (MS) algorithm is proposed to integrate object detection and tracking. The main contributions include: (i) a new framework based on the combination of LPM and MS is established for the integration of object tracking and detection. (ii) For object detection, the training and prediction of LPM are built by stacked denoising autoencoders based deep learning. (iii) For object tracking, an MS tracking algorithm leveraging LPM is modified to improve the tracking efficiency under a complex environment. Experimental results demonstrate that the proposed method is superior to the colour histograms based MS and histograms of oriented gradients based MS in terms of robustness and tracking accuracy.

In this paper we address the basic limitation of SiamMask - the state of the art single object tracking and segmentation algorithm. SiamMask requires semi-supervision in that it needs a bounding box to be drawn manually around the object that has to be tracked. This is however not always possible or feasible, and slows down the pipeline even in the best case. We overcome this limitation by using state-of-the-art object detection algorithms: Detectron2 and YOLO to automatically detect the object and then track using SiamMask. The main purpose of this study is to devise an efficient technique for an end-to-end object detection and tracking, which can then be used in other applications like self-driving cars, etc. We compared different approaches using current state-of-the-art tools for time and detection efficiency. One of the secondary aim was to test how the two approaches perform on different types of datasets. We note that YOLO gives better and more meaningful detection of objects in the scene. However, Detectron2 gives a higher detection speed than YOLO, making the overall detection and tracking process faster.

BackgroundInspection of livestock farms using surveillance cameras is emerging as a means of early detection of transboundary animal disease such as African swine fever (ASF). Object tracking, a developing technology derived from object detection aims to the consistent identification of individual objects in farms.ObjectivesThis study was conducted as a preliminary investigation for practical application to livestock farms. With the use of a high-performance artificial intelligence (AI)-based 3D depth camera, the aim is to establish a pathway for utilizing AI models to perform advanced object tracking.MethodsMultiple crossovers by two humans will be simulated to investigate the potential of object tracking. Inspection of consistent identification will be the evidence of object tracking after crossing over. Two AI models, a fast model and an accurate model, were tested and compared with regard to their object tracking performance in 3D. Finally, the recording of pig pen was also processed with aforementioned AI model to test the possibility of 3D object detection.ResultsBoth AI successfully processed and provided a 3D bounding box, identification number, and distance away from camera for each individual human. The accurate detection model had better evidence than the fast detection model on 3D object tracking and showed the potential application onto pigs as a livestock.ConclusionsPreparing a custom dataset to train AI models in an appropriate farm is required for proper 3D object detection to operate object tracking for pigs at an ideal level. This will allow the farm to smoothly transit traditional methods to ASF-preventing precision livestock farming.

Decision letter: Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Although many mesh generation methods have been developed, difficulties still exist in generating high-quality structured hex-meshes for specific local geometric features such as holes and columns in three-dimensional models. In this paper, a mesh generation approach is proposed to overcome the problem of meshing local features having regular structure with high quality. Firstly, analyze the feature's geometry information and customize its mesh process. Secondly, introduce parametric design into the optimization of mesh customization. Scaled Jacobian and aspect ratio are taken as indexes of the objective functions. Thirdly, an optimized module of the feature is set up by integrating the meshing scheme into program. By establishing a knowledge base of optimized modules corresponding to geometric features, a local feature can be automatically meshed according to its geometry information. Experiments in this paper reveal that the method is applicable and stable for generating high-quality meshes for local geometric features with regular structure.

Real time object tracking is an essential part of surveillance and robot applications. The performance of any object tracking system depends on its accuracy and its ability to deal with various sizes of objects, for better results the tracking should happen at a high speed. Object tracking can be defined as the process of segmenting the region of interest from the video sequence and keeping track of the motion in order to extract useful information. The basic fundamentals of video surveillance such as object detection, tracking and recognition using multiple cameras. Whenever an object enters a video frame, it is detected using background subtraction and the detected objects are tracked with a novel Bayesian Kalman filter with simplified Gaussian mixture (BKF-SGM). Object recognition algorithm is applied using Gait method which is employed to support multiple object tracking with its robust performance. Processing of image is done in MATLAB to get detection, tracking and recognized results.

The use of unmanned aerial vehicles (UAVs) that can operate autonomously in dynamic and dangerous operational environments are becoming increasingly common. In such operations, object detection, classification and tracking can often be one of the main goals. In recent years there has been an increased focus on embedded hardware that is both small and powerful, making UAV on-board data processing more viable. Being able to process the video feed on-board the UAV calls for fast and robust real-time algorithms for object identification and tracking. This paper discusses the development and implementation of a machine vision system for a low-cost fixed-wing UAV with a total flying weight of less than 4kg. The machine vision system incorporates the use of a thermal imaging camera and on-board processing power to perform real-time object detection, classification and tracking of objects in the ocean surface. The system is tested on thermal video data from a test flight, and is found to be able to detect 99;6% of objects of interest located in the ocean surface. Of the detected objects, only 5% were false positives. Furthermore, it classifies 93; 3% of the object types it is trained to classify correctly. The classifier is highly agile, allowing the user to quickly define which object characteristics that should be considered during classification, and what types of objects to classify. Finally, the system is found to successfully track 85% of the object types it is actively searching for in a real-time simulation test.

Object tracking is a dynamic optimization process based on the temporal information related to the previous frames. Proposing a method with higher precision in complex environments is a challenge for researchers in the field of study. In this research, efficient object detection and movement tracking video is proposed. Here we are considering the input video sequence is PETS and Hall monitor video. Kernel fuzzy c-means procedure still endures several downsides, such as reduced convergence rate, obtaining stuck in the local minima as well as at risk to initialization level of sensitivity. Ant colony optimization algorithm is a new population-based optimization technique that has actually been utilized effectively for resolving numerous complicated difficulties. This research suggested a new strategy called ACO–MKFCM to resolve MKFCM initialization issue making use of ant colony optimization (ACO) algorithm to locate optimum first cluster centres for the MKFCM, hence enhance all applications associated fuzzy clustering such as foreground segmentation in image processing. Initially, the background and foreground separation is done by hybridization of modified kernel fuzzy c means algorithm (MKFCM) with ant colony optimization. The recommended new technique is intellectual and also vibrant clustering method for splitting up of non-static object. Then object detection and tracking is done by gravitational search algorithm based deep belief neural network. The implementation will be in MATLAB. The performance of the suggested technique is appraised by means of precision, recall, F-measure, FPR, FNR, PWC, FAR, similarity, specificity, and accuracy. From the empirical effects, the future work outperforms than the state of art work. Here the proposed method attains maximum precision and recall value for both PETS and Hall monitor video when analyzed to the existent algorithm.

During the continual transitions from lab research to real-world applications of vision-based algorithms, there are significant challenges, e.g. the robustness to adapt to complex environments and the high demands of multi-task, multi-modal learning models. More specifically on visual simultaneous localization and mapping (visual SLAM) for mobile robots, two notable limitations are: (i) the drift issue during pose estimation, especially in dynamic environments, makes the positioning system unstable; (ii) the additional loop detection takes many computational resources for image retrieval and global feature geometry checking. This thesis explores the approaches of visual localization on unmanned ground vehicles by solving those limitations. The conventional feature extraction and matching pipeline for vision-based tasks involve three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. The first contribution of the thesis is adding a feature selection and anticipation stage to reduce tracking drift. We explore the noise model of image features to select a subset of all the observed image features with the best ”contribution” during data association and pose estimation across multiple frames. Conventional SLAM algorithms take a strong assumption of scene rigidity, which limits the application under challenging environments. The second part of the thesis addresses the tough issue in dynamic environments with moving objects. We presented GMC, namely the motion clustering approach, a lightweight dynamic object filtering method. It can distinguish moving objects from static landmarks. Based on the theory of motion coherence within a particular image area, GMC could segment dynamic objects in 3D space. We can provide an efficient and robust correspondence algorithm that can extract dynamic objects from a static background with the method. In this way, we propose a dynamic SLAM system that is real-time and free from expensive GPU processors. In contrast to GMC, the thesis’s third part turns to an object-aware learning-based model for more general dynamic scenarios. We use object detection and tracking as points, lines, planes, etc. We utilize semantic information and extract sparse image features simultaneously to keep track of dynamic objects. The static background and different dynamic objects are jointly optimized in a newly developed bundle adjustment sliding window. The estimated 3D bounding boxes can provide more robust camera tracking and better scene understanding, and better map merging. The fourth part of the thesis leverages the emerging feature learning framework. It proposes a unified self-supervised model called LGDNet to generate both global and local image feature descriptors end-to-end. Global feature descriptors embed the whole image into a compact representation, leading to easier scene comparison. On the other hand, local features focus more on the local region similarities of some salient parts for structure from motion. Our proposed method can directly extract features together with descriptors that encode both local maximum responses and global context information, avoiding duplicate calculations based on different feature extraction criteria.

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

In the tracking‐by‐detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking‐by‐detection paradigm. In this study, object detection accuracy has been increased by employing a faster region‐based convolutional neural network (Faster R‐CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter‐based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow‐based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R‐CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state‐of‐the‐art tracking methods demonstrate competitive tracking performance of the proposed MOT framework.

Low complexity object detection and tracking with inter-layer graph mapping and intra-layer graph refinement in H.264/SVC bitstreams

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.

The problem addressed here is the challenge of determining the best production strategy for multilayer reservoirs which varies from field to field as well as from well to well. Commingled production strategy is one of the most commonly used for several multilayer wells. However, there are some limitations to this method. Mostly, when very complex multilayer wells which contain different types of fluid in each productive layer are identified, the selection process of production approach is basically trial and error and later found to be a poor choice. One attempted solution to choosing the optimal production strategy is using numerical reservoir simulation. In this study, we used reservoir simulation to compare the performance of four different production strategies for different combinations of types of fluid present in a two-layer system. Strategy 1 in which the fluid is produced from two layers simultaneously is the best production approach when oil is present in the top and bottom layers. Strategy 2 in which the fluid is produced from one layer at a time starting with the bottom layer is the best production approach when reservoir fluid at the top layer is gas while fluids present in the bottom layer can vary from oil, gas&oil, gas&water and oil&water. Strategy 3 produces fluid from one layer at a time, starting with the bottom layer. When the production rate decreases to half of the original rate, the two zones are produced in a commingled fashion. Similar to strategy 1, strategy 3 is also the best production approach when oil is present in the top and bottom layers. Strategy 4 is similar to strategy 3 but starting with the top layer. This strategy is the best production approach when oil is present in the top layer and gas&water in the bottom layer. This approach can delay water production and sustain reservoir pressure.