Computer Vision for Object Recognition and Tracking Based on Raspberry Pi

The field of computer vision has undergone rapid development. Starting from recognition tasks such as classification, detection, and segmentation, the focus of visual analysis has gradually shifted towards learning spatiotemporal information. This thesis presents research on spatiotemporal learning, with a particular emphasis on motion estimation and visual recognition. First, we address the problem of video object tracking. Previous methods have primarily re-lied on learning improved appearance representations, while the spatiotemporal relationships of individual objects have been underexplored. We propose leveraging optical flow features to achieve higher generalization in semi-supervised video object segmentation, directly incorporating these features into both the target representation and the decoder network. Our experiments and analysis show that enriching feature representations with spatiotemporal information improves segmentation quality and generalization capability. Next, we investigate spatiotemporal learning in 3D for motion estimation, specifically scene flow estimation. Scene flow estimation as an important research topic in 3D computer vision is crucial for applications such as robotics, autonomous driving, embodied navigation, and tracking. We investigate the problem in different perspectives: 1. What is the best formulation for solving the problem and how to learn a better spatiotemporal feature representation? 2. Can we introduce uncertainty estimation to the task, which is of crucial importance for safety-critical downstream tasks? 3. How to scale the estimation to large-scale data, e.g., autonomous scenes, and leverage the temporal information without introducing much computation overheads? To answer these questions, we explore the use of transformers for improved feature representation, diffusion models for uncertainty estimation, and efficient feature learning methods for multi-frame, large-scale autonomous driving scenarios. Finally, we extend our research to joint visual segmentation, tracking, and open-vocabulary recognition in LiDAR sequences, particularly for autonomous scenes. In such environments, precise segmentation, tracking, and recognition of objects are essential for downstream analysis and control. Current human-annotated open-source datasets allow for reasonable tracking of traffic participants such as cars and pedestrians. However, we aim to advance beyond this towards segmenting and tracking any object in LiDAR data. To this end, we propose a pseudo-labeling engine that leverages the 2D visual foundation model SAM v2 and the vision-language model CLIP to automatically label LiDAR streams. We further introduce the SAL-4D model, capable of segmenting, tracking, and recognizing any object in a zero-shot manner. In summary, we explore the learning of spatiotemporal information in both 2D image and 3D point cloud domains. In the image domain, we demonstrate that spatiotemporal information improves video object segmentation quality and generalization. In the 3D point cloud domain, we show that spatiotemporal learning enables more accurate motion estimation and facilitates the first method for zero-shot segmentation, tracking, and open-vocabulary recognition of arbitrary objects.

The importance of the real-time object recognition and tracking field has been growing steadily due to rapid advancement in the computer vision applications industry. As is well known, the mean-shift algorithm is widely used in robust real-time object tracking systems. Since the mentioned algorithm is easy to implement and efficient in object tracking computation, many say it is suitable to be applied to real-time object tracking systems. However, one of the major drawbacks of this algorithm is that it always converges to a local mode, failing to perform well in a cluttered environment. In this paper, an Optical Flow-based algorithm which fits for real-time recognition of multiple moving objects is proposed. Also in the tests, the newly proposed method contributed to raising the similarity of multiple moving objects, the similarity was as high as 0.96, up 13.4% over that of the mean-shift algorithm. Meanwhile, the level of pixel errors from using the new method keenly decreased by more than 50% over that from applying the mean-shift algorithm. If the data processing speed in the video surveillance systems can be reduced further, owing to improved algorithms for faster moving object recognition and tracking functions, we will be able to expect much more efficient intelligent systems in this industrial arena.

The Kalman filter has long been regarded as the optimal solution to many applications in computer vision for example the tracking objects, prediction and correction tasks. Its use in the analysis of visual motion has been documented frequently, we can use in computer vision and open cv in different applications in reality for example robotics, military image and video, medical applications, security in public and privacy society, etc. In this paper, we investigate the implementation of a Matlab code for a Kalman Filter using three algorithm for tracking and detection objects in video sequences (block-matching (Motion Estimation) and Camshift Meanshift (localization, detection and tracking object)). The Kalman filter is presented in three steps: prediction, estimation (correction) and update. The first step is a prediction for the parameters of the tracking and detection objects. The second step is a correction and estimation of the prediction parameters. The important application in Kalman filter is the localization and tracking mono-objects and multi-objects are given in results. This works presents the extension of an integrated modeling and simulation tool for the tracking and detection objects in computer vision described at different models of algorithms in implementation systems.

A literature analysis has shown that object search, recognition, and tracking systems are becoming increasingly popular. However, such systems do not achieve high practical results in analyzing small moving living objects ranging from 8 to 14 mm. This article examines methods and tools for recognizing and tracking the class of small moving objects, such as ants. To fulfill those aims, a customized You Only Look Once Ants Recognition (YOLO_AR) Convolutional Neural Network (CNN) has been trained to recognize Messor Structor ants in the laboratory using the LabelImg object marker tool. The proposed model is an extension of the You Only Look Once v4 (Yolov4) 512 × 512 model with an additional Self Regularized Non–Monotonic (Mish) activation function. Additionally, the scalable solution for continuous object recognizing and tracking was implemented. This solution is based on the OpenDatacam system, with extended Object Tracking modules that allow for tracking and counting objects that have crossed the custom boundary line. During the study, the methods of the alignment algorithm for finding the trajectory of moving objects were modified. I discovered that the Hungarian algorithm showed better results in tracking small objects than the K–D dimensional tree (k-d tree) matching algorithm used in OpenDataCam. Remarkably, such an algorithm showed better results with the implemented YOLO_AR model due to the lack of False Positives (FP). Therefore, I provided a new tracker module with a Hungarian matching algorithm verified on the Multiple Object Tracking (MOT) benchmark. Furthermore, additional customization parameters for object recognition and tracking results parsing and filtering were added, like boundary angle threshold (BAT) and past frames trajectory prediction (PFTP). Experimental tests confirmed the results of the study on a mobile device. During the experiment, parameters such as the quality of recognition and tracking of moving objects, the PFTP and BAT, and the configuration parameters of the neural network and boundary line model were analyzed. The results showed an increased tracking accuracy with the proposed methods by 50%. The study results confirmed the relevance of the topic and the effectiveness of the implemented methods and tools.

Object detection and tracking under complex environment are challenging because of the disturbances induced by background clutter, illumination changes, occlusions and other factors. The bulk of traditional algorithms basically rely on hand‐crafted features, which are not sufficiently robust to a complex environment. Moreover, the processes of detection and tracking are separated, which leads to the overall efficiency not high. In this study, a novel local probability model (LPM)‐based mean shift (MS) algorithm is proposed to integrate object detection and tracking. The main contributions include: (i) a new framework based on the combination of LPM and MS is established for the integration of object tracking and detection. (ii) For object detection, the training and prediction of LPM are built by stacked denoising autoencoders based deep learning. (iii) For object tracking, an MS tracking algorithm leveraging LPM is modified to improve the tracking efficiency under a complex environment. Experimental results demonstrate that the proposed method is superior to the colour histograms based MS and histograms of oriented gradients based MS in terms of robustness and tracking accuracy.

Real time object tracking is an essential part of surveillance and robot applications. The performance of any object tracking system depends on its accuracy and its ability to deal with various sizes of objects, for better results the tracking should happen at a high speed. Object tracking can be defined as the process of segmenting the region of interest from the video sequence and keeping track of the motion in order to extract useful information. The basic fundamentals of video surveillance such as object detection, tracking and recognition using multiple cameras. Whenever an object enters a video frame, it is detected using background subtraction and the detected objects are tracked with a novel Bayesian Kalman filter with simplified Gaussian mixture (BKF-SGM). Object recognition algorithm is applied using Gait method which is employed to support multiple object tracking with its robust performance. Processing of image is done in MATLAB to get detection, tracking and recognized results.

Object recognition and tracking are two of the most dynamic research sub-areas that belong to the field of Computer Vision. Computer vision is one of the most active research fields that lies at the intersection of deep learning and machine vision. This paper presents an efficient ensemble algorithm for the recognition and tracking of fixed shape moving objects while accommodating the shift and scale invariances that the object may encounter. The first part uses the Maximum Average Correlation Height (MACH) filter for object recognition and determines the bounding box coordinates. In case the correlation based MACH filter fails, the algorithms switches to a much reliable but computationally complex feature based object recognition technique i.e., affine scale invariant feature transform (ASIFT). ASIFT is used to accommodate object shift and scale object variations. ASIFT extracts certain features from the object of interest, providing invariance in up to six affine parameters, namely translation (two parameters), zoom, rotation and two camera axis orientations. However, in this paper, only the shift and scale invariances are used. The second part of the algorithm demonstrates the use of particle filters based Approximate Proximal Gradient (APG) technique to periodically update the coordinates of the object encapsulated in the bounding box. At the end, a comparison of the proposed algorithm with other state-of-the-art tracking algorithms has been presented, which demonstrates the effectiveness of the proposed algorithm with respect to the minimization of tracking errors.

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed.

Image segmentation is the problem of partitioning an image into different subsets, where each subset may have a different characterization in terms of color, intensity, texture, and/or other features. Segmentation is a fundamental component of image processing, and plays a significant role in computer vision, object recognition, and object tracking. Active Contour Models (ACMs) constitute a powerful energy-based minimization framework for image segmentation, which relies on the concept of contour evolution. Starting from an initial guess, the contour is evolved with the aim of approximating better and better the actual object boundary. Handling complex images in an efficient, effective, and robust way is a real challenge, especially in the presence of intensity inhomogeneity, overlap between the foreground/background intensity distributions, objects characterized by many different intensities, and/or additive noise. In this thesis, to deal with these challenges, we propose a number of image segmentation models relying on variational level set methods and specific kinds of neural networks, to handle complex images in both supervised and unsupervised ways. Experimental results demonstrate the high accuracy of the segmentation results, obtained by the proposed models on various benchmark synthetic and real images compared with state-of-the-art active contour models.

This paper considers a task of visual object tracking with application in embedded systems. Specifically, Raspberry Pi is considered as target hardware platform. The main challenge in this problem setting is to balance between algorithm result quality and its runtime efficiency. To select the method, the main object tracking approaches were analyzed and compared. Based on this, Nanotrack method was picked for deeper analysis and experiments. Method results accuracy was analyzed by already available benchmark scores and by running test on handpicked cases from LaSOT dataset highlighting different challenges for object tracking methods such as occlusion, scale variability, etc. The runtime performance was measured using target hardware. Based on the obtained results it was concluded that selected method is suitable to perform real time object tracking in the target setting, and further development will be based on this method.

Multiple target tracking (MTT) or multiple object tracking (MOT), which is a key stage operation for many computer vision applications, has relied and it relies on detecting and identifying targets within videos. The objects of these videos may be, for instance, pedestrians, vehicles or animals regardless of the number of targets or their appearance. The main goal of multi object tracking is to follow paths or trajectories of multiple targets in a sequence. Multiple object tracking is a field of computer vision that has a wide range of applications, starting from video surveillance and human computer interaction to robotics. The aim of visual object tracking is to maintain objects’ identities. Developing an effective and accurate visual tracking system is very challenging due to the enormous number of problems, such as Illumination variation or background, the famous Clutters meaning the overlap between objects. Low resolution may be due to cameras problems, scale variation, occlusion, and change of targets’ positions within video. These challenges have a wide range of approaches and methods in this evolving research area with several algorithms in each subfield. This paper covers the crucial research area for Multiple Object Tracking (MOT) and this study will help researchers accomplish their scientific projects relying on the wide range of algorithms mentioned on this review.

Object detection and tracking are vital in computer vision and visual surveillance, allowing for the detection, recognition, and subsequent tracking of objects within images or video sequences. These tasks underpin surveillance systems, facilitating automatic video annotation, identification of significant events, and detection of abnormal activities. However, detecting and tracking small objects introduce significant challenges within computer vision due to their subtle appearance and limited distinguishing features, which results in a scarcity of crucial information. This deficit complicates the tracking process, often leading to diminished efficiency and accuracy. To shed light on the intricacies of small object detection and tracking, we undertook a comprehensive review of the existing methods in this area, categorizing them from various perspectives. We also presented an overview of available datasets specifically curated for small object detection and tracking, aiming to inform and benefit future research in this domain. We further delineated the most widely used evaluation metrics for assessing the performance of small object detection and tracking techniques. Finally, we examined the present challenges within this field and discussed prospective future trends. By tackling these issues and leveraging upcoming trends, we aim to push forward the boundaries in small object detection and tracking, thereby augmenting the functionality of surveillance systems and broadening their real-world applicability.

The classical mean shift (MS) algorithm is the best color-based method for object tracking. However, in the real environment it presents some limitations, especially under the presence of noise, objects with partial and full occlusions in complex environments. In order to deal with these problems, this paper proposes a reliable object tracking algorithm using corrected background-weighted histogram (CBWH) and the Kalman filter (KF) based on the MS method. The experimental results show that the proposed method is superior to the traditional MS tracking in the following aspects: 1) it provides consistent object tracking throughout the video; 2) it is not influenced by the objects with partial and full occlusions; 3) it is less prone to the background clutter.

Visual object tracking has become one of the most active research topics in computer vision, which has been growing in commercial development as well as academic research. Many visual trackers have been proposed in the last two decades. Recent studies of computer vision for dynamic scenes include motion detection, object classification, environment modeling, tracking of moving objects, understanding of object behaviors, object identification, and data fusion from multiple sensors. This paper provides an in-depth overview of recent object tracking research. Object tracking tasks in realistic scenario often face challenging problems such as camera motion, occlusion, illumination effect, clutter, and similar appearance. A variety of tracker techniques have been published, which combine multiple techniques to solve multiple visual tracking sub-problems. This paper also reviews the latest research trend in object tracking based on convolutional neural networks, which is receiving growing attention. Finally, the paper discusses the future challenges and research directions for the object tracking problems that still need extensive studies in coming years.

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.