A new framework for on-line object tracking based on SURF

Object tracking is a major technique in image processing and computer vision. In this paper, we propose a new robust feature-based tracking scheme by employing adaptive classifiers to match the detected keypoints in consecutive frames. The novelty of this paper is that the design of online boosting is combined with the invariance of local features so that the classifier-based descriptions are formed in association with the scale and rotation information. Furthermore, we introduce a sample weighting mechanism in the on-line classifier updating, for the subsequent tracking. Experimental results demonstrate the robustness and accuracy of our proposed technique.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

Video object segmentation and tracking are two essential building blocks of smart surveillance systems. However, there are several issues that need to be resolved. Threshold decision is a difficult problem for video object segmentation with a multi-background model. In addition, some conditions make robust video object tracking difficult. These conditions include nonrigid object motion, target appearance variations due to changes in illumination, and background clutter. In this paper, a video object segmentation and tracking framework is proposed for smart cameras in visual surveillance networks with two major contributions. First, we propose a robust threshold decision algorithm for video object segmentation with a multi-background model. Second, we propose a video object tracking framework based on a particle filter with the likelihood function composed of diffusion distance for measuring color histogram similarity and motion clue from video object segmentation. The proposed framework can track nonrigid moving objects under drastic changes in illumination and background clutter. Experimental results show that the presented algorithms perform well for several challenging sequences, and our proposed methods are effective for the aforementioned issues.

Recently, the popularity of user generated videos has highlighted efficient video indexing and browsing as an urgent problem. Points of interest (POI) detection is a technique to address this issue by establishing the implicit relationship among different media resources. The majority of existing studies detect POI by visual similarity, leveraging computer vision techniques. However, these methods suffer from high computational complexity when processing large-scale video sets and are challenging due to the sparse visual correlation among different consumer videos. The advent of geo-referenced videos provides an opportunity to detect POIs in an efficient manner. In this work, we first propose a probability model to formulate the capture intention distribution for video frames. Second, we detect the POIs by considering the contributions from multiple videos. Evaluations demonstrate that our algorithm successfully detects POIs with a reduced error distance from several dozens to a few meters.

Real-time object detection and tracking in video streams has become a fundamental component of modern intelligent systems, enabling applications in surveillance, autonomous driving, robotics, healthcare monitoring, and smart cities. The rapid advancement of deep learning techniques has significantly improved the accuracy and efficiency of detecting and tracking multiple objects under complex and dynamic environments. This study presents a deep learning–based framework for real-time object detection and tracking in continuous video streams, emphasizing both computational efficiency and detection robustness.The proposed system integrates a state-of-the-art convolutional neural network (CNN)-based object detector with a reliable multi-object tracking algorithm. The detection module identifies objects of interest in each frame, while the tracking component maintains consistent identities across consecutive frames by utilizing motion prediction and feature association strategies. To achieve real-time performance, the model is optimized through lightweight network architectures and parallel processing on GPU hardware. Data augmentation and transfer learning techniques are employed to enhance generalization and reduce training time. Performance evaluation is conducted on benchmark video datasets using standard metrics such as precision, recall, mean Average Precision (mAP), and tracking accuracy. Experimental results demonstrate that the proposed framework achieves high detection accuracy while maintaining low latency, making it suitable for real-time deployment. Furthermore, the system effectively handles challenges such as occlusion, scale variation, and illumination changes. This research presents a comprehensive study on real-time object detection and tracking in video streams using deep learning techniques. The proposed framework integrates YOLO-based object detection with Deep SORT multi-object tracking to achieve robust spatial localization and temporal identity preservation. The study evaluates performance using benchmark datasets including COCO, MOT17, and KITTI. Mathematical formulations of detection loss, Kalman filtering, and data association are presented. Experimental analysis using precision, recall, mAP, MOTA, ROC-AUC, and PR-AUC demonstrates that the system achieves high detection accuracy while maintaining real-time performance above 35 frames per second. The findings confirm that deep learning-based frameworks provide scalable and efficient solutions for surveillance, autonomous vehicles, and intelligent monitoring systems.

In this paper, we compare the results of five 3D interest point detection techniques to the interest points marked by human subjects. This comparison is used to quantitatively evaluate the interest point detection algorithms. We asked human subjects to look at a number of 3D models, and mark interest points on the models via a web-based interface. We propose a voting-based method to construct ground truth out of humans' selections of interest points. Evaluation measures, namely False Positive and False Negative Errors, are then defined based on the geodesic distance between the interest points detected by a particular algorithm and the human-generated ground truth.

Object detection and tracking is one of the key applications of wireless sensor networks (WSNs). The key issues associated with this application include network lifetime, object detection and localization accuracy. To ensure the high quality of the service, there should be a trade-off between energy efficiency and detection accuracy, which is challenging in a resource-constrained WSN. Most researchers have enhanced the application lifetime while achieving target detection accuracy at the cost of high node density. They neither considered the system cost nor the object localization accuracy. Some researchers focused on object detection accuracy while achieving energy efficiency by limiting the detection to a predefined target trajectory. In particular, some researchers only focused on node clustering and node scheduling for energy efficiency. In this study, we proposed a mobile object detection and tracking framework named the Energy Efficient Object Detection and Tracking Framework (EEODTF) for heterogeneous WSNs, which minimizes energy consumption during tracking while not affecting the object detection and localization accuracy. It focuses on achieving energy efficiency via node optimization, mobile node trajectory optimization, node clustering, data reporting optimization and detection optimization. We compared the performance of the EEODTF with the Energy Efficient Tracking and Localization of Object (EETLO) model and the Particle-Swarm-Optimization-based Energy Efficient Target Tracking Model (PSOEETTM). It was found that the EEODTF is more energy efficient than the EETLO and PSOEETTM models.

In this study, a novel interest point detection algorithm which combines image intensity variation and edge contour information is proposed. Firstly, the Canny edge contour detector is used to extract the edge map. Secondly, the imaginary parts of multi‐scale Gabor filters are applied to smooth input image and then the normalised information entropies at various scales can be acquired. Finally, multiplication of different normalised information entropies will be served as a new measure for interest point that can also be used for interest point detection. This method has two advantages: on the one hand, detection accuracy is greatly improved because combination information is adopted to extract interest points including contour shape information, grey variation of edge pixels and their neighbours; on the other hand, non‐interest points can be well inhabited due to multi‐scale product in detector is served as an interest point measure. Also, desirable noise robustness and time efficiency are validated through experiments. Compared with four other state‐of‐art methods, the proposed method shows excellent performance in terms of geometric transformations and localisation accuracy of repeated interest points.

In a goal-oriented Human Robot Collaborative (HRC) scenario, where the goal is to complete an assembly process, a robust object tracker might not necessarily fulfill its functional role due to the dynamic nature of HRC. Moreover, for an efficient HRC, the functional role of the object tacker should not only be limited to localizing and tracking objects for robotic manipulation. It should also help to determine the current state of the assembly process and verify if the chosen action has been successfully performed and thus to enable an uninterrupted completion of an HRC assembly process. We present a Context Enhanced Framework for Multi Object Tracking, that i) allows uninterrupted completion of an assembly process, ii) improves the overall functional accuracy of the object tracker from 49 percent to 96 percent, and iii) enables the object tracker to handle multiple instance of multiple objects in a HRC setting.

A recent trend is to combine multiple sensors ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , cameras, LiDARs and millimeter-wave Radars) to achieve robust multi-modal perception for autonomous systems such as self-driving vehicles. Although quite a few sensor fusion algorithms have been proposed, some of which are top-ranked on various leaderboards, a systematic study on how to integrate these three types of sensors to develop effective multi-modal 3D object detection and tracking is still missing. Towards this end, we first study the strengths and weaknesses of each data modality carefully, and then compare several different fusion strategies to maximize their utility. Finally, based upon the lessons learnt, we propose a simple yet effective multi-modal 3D object detection and tracking framework (namely EZFusion). As demonstrated by extensive experiments on the nuScenes dataset, without fancy network modules, our proposed EZFusion makes remarkable improvements over the LiDAR-only baseline, and achieves comparable performance with the state-of-the-art fusion-based methods.

A problem of interest point detection in camera images, i.e. detection of points with high information content is studied. An overview of existing techniques is given. A modified Harris-Laplace interest point detector is proposed and experimental results are given. Proposed detector generates a smaller number of detected points, resulting in less complex matching. The aim is to use interest point detectors, descriptors and matching for mobile robot navigation.

Several machine learning tasks require to represent the data using only a sparse set of interest points. An ideal detector is able to find the corresponding interest points even if the data undergo a transformation typical for a given domain. Since the task is of high practical interest in computer vision, many hand-crafted solutions were proposed. In this paper, we ask a fundamental question: can we learn such detectors from scratch? Since it is often unclear what points are interesting, human labelling cannot be used to find a truly unbiased solution. Therefore, the task requires an unsupervised formulation. We are the first to propose such a formulation: training a neural network to rank points in a transformation-invariant manner. Interest points are then extracted from the top/bottom quantiles of this ranking. We validate our approach on two tasks: standard RGB image interest point detection and challenging cross-modal interest point detection between RGB and depth images. We quantitatively show that our unsupervised method performs better or on-par with baselines.

With the development of the manufacturing technique of the microchip, the integral performance of the Field-Programmable Gate Array (FPGA) is becoming more and more adaptive for real-time embedded systems. Interest point detection is a prime step in many advanced computer vision applications such as 3-D reconstruction, 3-D object tracking and so on. Many image processing algorithms need to extract interest point in advance. We can easily implement these algorithms in software, but it can't meet with the request of a real-time embedded systems because of its low speed. In this paper, we achieved a hardware implementation of the Harris corner feature detector based on Zedboard platform which is a heterogeneous SoC composed of two ARM9 chips and a Xilinx xc7z020-clg484-1 chip. The hardware architecture is implemented with Verilog HDL and consists of 6 modules: CMOS register initialization, CMOS data fetcher, CMOS data graying, block RAM frame buffer, Harris Corner Detector(HCD), HDMI displayer. The implementation can process RGB565 video in 640×480 resolution. According to the simulation results, the hardware implementation can extract Harris corner at the speed of 154 frames per second. It can meet most requirements for real-time computer vision applications.

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian $$\nabla ^2_{norm} L$$?norm2L, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL and four new unsigned or signed Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL, $${\mathcal {D}}_{2,norm} L$$D2,normL and $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL is a truly affine covariant differential entity and the Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL and $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differential entities will be more robust under affine image deformations than local extrema of the Laplacian operator or the Hessian feature strength measures $${\mathcal {D}}_{2,norm} L$$D2,normL, $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL. It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL and the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris---Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.

Object tracking is still an intriguing task as the target undergoes significant appearance changes due to illumination, fast motion, occlusion and shape deformation. Background clutter and numerous other environmental factors are other major constraints which remain a riveting challenge to develop a robust and effective tracking algorithm. In the present study, an adaptive Spatio-temporal context (STC)-based algorithm for online tracking is proposed by combining the context-aware formulation, Kalman filter, and adaptive model learning rate. For the enhancement of seminal STC-based tracking performance, different contributions were made in the proposed study. Firstly, a context-aware formulation was incorporated in the STC framework to make it computationally less expensive while achieving better performance. Afterwards, accurate tracking was made by employing the Kalman filter when the target undergoes occlusion. Finally, an adaptive update scheme was incorporated in the model to make it more robust by coping with the changes of the environment. The state of an object in the tracking process depends on the maximum value of the response map between consecutive frames. Then, Kalman filter prediction can be updated as an object position in the next frame. The average difference between consecutive frames is used to update the target model adaptively. Experimental results on image sequences taken from Template Color (TC)-128, OTB2013, and OTB2015 datasets indicate that the proposed algorithm performs better than various algorithms, both qualitatively and quantitatively.