Visual Tracking via Locality Sensitive Histograms

This paper presents a novel locality sensitive histogram (LSH) algorithm for visual tracking. Unlike the conventional image histogram that counts the frequency of occurrence of each intensity value by adding ones to the corresponding bin, an LSH is computed at each pixel location, and a floating-point value is added to the corresponding bin for each occurrence of an intensity value. The floating-point value exponentially reduces with respect to the distance to the pixel location where the histogram is computed. An efficient algorithm is proposed that enables the LSHs to be computed in time linear in the image size and the number of bins. In addition, this efficient algorithm can be extended to exploit color images. A robust tracking framework based on the LSHs is proposed, which consists of two main components: a new feature for tracking that is robust to illumination change and a novel multiregion tracking algorithm that runs in real time even with hundreds of regions. Extensive experiments demonstrate that the proposed tracking framework outperforms the state-of-the-art methods in challenging scenarios, especially when the illumination changes dramatically. Evaluation using the latest benchmark shows that our algorithm is the top performer.

Recent researches on visual tracking have shown significant improvement in accuracy by handling the large uncertainties induced by appearance variation and abrupt motion. Most studies concentrate on random walk based Markov chain Monte Carlo(MCMC) tracking methods which have shown inefficiency in sampling from complex and high-dimensional distributions. This paper proposes an adaptive Hamiltonian Monte Carlo sampling based tracking method within the Bayesian filtering framework. In order to suppress the random walk behavior in Gibbs sampling stage, the ordered over-relaxation method is used to draw the momentum item for the joint state variable. An adaptive step-size based scheme is used to simulate the Hamiltonian dynamics in order to reduce the simulation error and improve acceptance rate of the proposed samples. Furthermore, in designing the appearnce model, we introduce the locality sensitive histogram (LSH) to deal with appearance changes induced by illumination change. The proposed tracking method is compared with several state-of-the-art trackers using different quantitative measures: success rate and abruption capture rate. Extensive experimental results have shown its superiority to several other trackers.

When entering into the realm of Computer Vision, the first thing which comes in to mind is Visual tracking. Visual tracking by far comes into one of the most actively investigated research areas because of the fact that it has an extensive collection of applications in areas such as activity recognition, surveillance, motion analysis and as well as human computer interaction. Some serious challenges of this area which still create hindrance in achieving 100% accuracy are abrupt appearance and pose changes of an object along with its background blockage due to blockages called occlusion, illumination and lighting variances and changes in scale of target object in the frames. Moreover, diverse algorithms had been proposed for the resolution of said issue. Now in such cases, if we study the statistical analysis of correlation between two frames in a certain video, it can be efficiently utilized to get the most exact location of the targeted object. The algorithms in existence today do not completely exploit a strong spatio-temporal relationship that very often occurs between the two successive frames in a video sequence. Recent advances in correlation-based tracking systems have been proposed to address the problem in successive frames. In this thesis a very simple yet quite speedy and robust algorithm that in actual brings all the relevant information used for Visual Tracking. Two of the Models proposed are the “Locality Sensitive Histogram” and “Discriminative Scale Tracking Method”. These are robust enough to the variations which are based on appearance which are normally presented by blockage, pose, illumination and lighting variations alike. A scheme is proposed called scale adaptation which is very much clever to adapt variations of targeted scale in the most efficient manner. The Discriminative Scale Tracking Method is used for detection as well as scale change ultimately resulting in an effective tracking method in the end. Various different experiments with the best algorithms have demonstrated on challenging sequences that the suggested methodology attains promising results as far as robustness, accuracy, and speed is concerned.

Video object segmentation and tracking are two essential building blocks of smart surveillance systems. However, there are several issues that need to be resolved. Threshold decision is a difficult problem for video object segmentation with a multi-background model. In addition, some conditions make robust video object tracking difficult. These conditions include nonrigid object motion, target appearance variations due to changes in illumination, and background clutter. In this paper, a video object segmentation and tracking framework is proposed for smart cameras in visual surveillance networks with two major contributions. First, we propose a robust threshold decision algorithm for video object segmentation with a multi-background model. Second, we propose a video object tracking framework based on a particle filter with the likelihood function composed of diffusion distance for measuring color histogram similarity and motion clue from video object segmentation. The proposed framework can track nonrigid moving objects under drastic changes in illumination and background clutter. Experimental results show that the presented algorithms perform well for several challenging sequences, and our proposed methods are effective for the aforementioned issues.

Feature points and object edges are two kinds of primitives which are frequently used in target tracking algorithms. Feature points can be easily localized in an image. Their correspondences between images can be detected accurately. They can adapt to wide baseline transformations. However, feature points are not so stable that they are fragile to changes in illumination and viewpoint. On the contrary, object edges are stable under a very wide range of illumination and viewpoint changes. Unfortunately, edge-based algorithms often fail in the presence of highly textured targets and clutter which produce too many irrelevant edges. We found that both edge-based and point-based tracking have failure modes which are complementary. Based on this analysis, we propose a novel tracking algorithm which fuses point and edge features. Our tracking algorithm uses feature points matching to track object first, and then uses the transformation parameters archived in the first step to initialize the edge tracking. By this means, our algorithm alleviates the disturbance of irrelevant edges. Then, we use the texture boundary detection algorithm to find the precise object boundary. Texture boundary detection is different from the conventional gradient-based edge detection which can directly compute the most probable location of a texture boundary on the search line. Therefore, it is very fast and can be incorporated into a real-time tracking algorithm. Experimental results show that our tracking algorithm has outstanding tracking accuracy and robustness.

One of the fundamental operations in computer vision applications is change detection, in which moving foreground objects are segmented from a static background. A common approach for change detection is the comparison of an image frame with the stored background model using a matching algorithm, a process known as background subtraction. However, such techniques fail in environments with dynamic backgrounds, illumination changes, or shadow and camera jitters. This study focuses on effectively detecting scene changes in complex environments. To this end, we proposed a new colour descriptor named local colour difference pattern (LCDP) that is insusceptible to shadow and is able to capture both colour and texture features at a pixel location. Furthermore, a scene change detection framework was proposed to handle dynamic scenes based on sample consensus that integrates LCDP and a novel spatial model fusion mechanism. Experiments using the CDnet benchmark dataset demonstrated the effectiveness of the proposed approach to change detection in complex environments.

Identifying and tracking objects in surveillance videos is an important means of mining information during surveillance. Currently, most object‐tracking methods rely only on image features, which cannot accurately express the motion of the tracked object in real geographical scenes and are easily influenced by occlusion and surrounding objects having similar features. However, tracked objects, such as pedestrians and vehicles, usually move in geographical space with fixed patterns of motion, and the motion in a short time is constrained by geographical space and time, the motion trajectory is predictable, and the range of motion is limited. Therefore, based on the SiamFC object tracking framework, this study introduces geographical spatiotemporal constraints into the tracking framework and proposes the GeoSiamFC method. The objective of this is to: (1) construct the mapping relationship between geographical space and image space to solve the problem that the pixel movement within the image after perspective imaging cannot accurately express the motion of the tracked object in a real geographical scene; (2) add candidate search areas according to the predicted trajectory location to correct the tracking errors caused by the occlusion of the object; and (3) focus on the search for the range of motion of the mapped pixel within the image space according to the limited geographical range of motion of the tracked objects in a certain time to reduce the interference of similar objects within the search area. In this study, separate experiments were conducted on a common test dataset using multiple methods to deal with challenges such as occlusion and illumination changes. In addition, a robust test dataset with noise addition and luminance adjustment based on the common test dataset was used. The results show that GeoSiamFC outperforms other object‐tracking methods in the common test dataset with a precision score of 0.995 and a success score of 0.756 compared with most other object‐tracking algorithms under the base condition of using only shallow networks. Moreover, GeoSiamFC maintained the highest precision score (0.970) and high success score (0.734) in the more challenging robust test dataset as well. The tracking speed of 59 frames per second far exceeds the real‐time requirement of 25 FPS. Geographical spatiotemporal constraints were considered to improve tracker performance while providing real‐time feedback on the motion trajectory of the target in geographical space. Thus, the proposed method is suitable for real‐time tracking of the motion trajectory of a target in real geographical scenes in various surveillance videos.

We present an algorithm for robust and real-time visual tracking under challenging illumination conditions characterized by poor lighting as well as sudden and drastic changes in illumination. Robustness is achieved by adapting illumination-invariant binary descriptors to dense image alignment using the Lucas and Kanade algorithm. The proposed adaptation preserves the Hamming distance under least-squares minimization, thus preserving the photometric invariance properties of binary descriptors. Due to the compactness of the descriptor, the algorithm runs in excess of 400 fps on laptops and 100 fps on mobile devices.

Real-time object detection and tracking in video streams has become a fundamental component of modern intelligent systems, enabling applications in surveillance, autonomous driving, robotics, healthcare monitoring, and smart cities. The rapid advancement of deep learning techniques has significantly improved the accuracy and efficiency of detecting and tracking multiple objects under complex and dynamic environments. This study presents a deep learning–based framework for real-time object detection and tracking in continuous video streams, emphasizing both computational efficiency and detection robustness.The proposed system integrates a state-of-the-art convolutional neural network (CNN)-based object detector with a reliable multi-object tracking algorithm. The detection module identifies objects of interest in each frame, while the tracking component maintains consistent identities across consecutive frames by utilizing motion prediction and feature association strategies. To achieve real-time performance, the model is optimized through lightweight network architectures and parallel processing on GPU hardware. Data augmentation and transfer learning techniques are employed to enhance generalization and reduce training time. Performance evaluation is conducted on benchmark video datasets using standard metrics such as precision, recall, mean Average Precision (mAP), and tracking accuracy. Experimental results demonstrate that the proposed framework achieves high detection accuracy while maintaining low latency, making it suitable for real-time deployment. Furthermore, the system effectively handles challenges such as occlusion, scale variation, and illumination changes. This research presents a comprehensive study on real-time object detection and tracking in video streams using deep learning techniques. The proposed framework integrates YOLO-based object detection with Deep SORT multi-object tracking to achieve robust spatial localization and temporal identity preservation. The study evaluates performance using benchmark datasets including COCO, MOT17, and KITTI. Mathematical formulations of detection loss, Kalman filtering, and data association are presented. Experimental analysis using precision, recall, mAP, MOTA, ROC-AUC, and PR-AUC demonstrates that the system achieves high detection accuracy while maintaining real-time performance above 35 frames per second. The findings confirm that deep learning-based frameworks provide scalable and efficient solutions for surveillance, autonomous vehicles, and intelligent monitoring systems.

Recently, face recognition became a key element in social cognition which is used in various applications including human–robot interaction (HRI), pedestrian identification, and surveillance systems. Deep convolutional neural networks (CNNs) have achieved notable progress in recognizing faces. However, achieving accurate and real-time face recognition is still a challenging problem, especially in unconstrained environments due to occlusion, lighting conditions, and the diversity in head poses. In this paper, we present a robust face recognition and tracking framework in unconstrained settings. We developed our framework based on lightweight CNNs for all face recognition stages, including face detection, alignment and feature extraction, to achieve higher accuracies in these challenging circumstances while maintaining the real-time capabilities required for HRI systems. To maintain the accuracy, a single-shot multi-level face localization in the wild (RetinaFace) is utilized for face detection, and additive angular margin loss (ArcFace) is employed for recognition. For further enhancement, we introduce a face tracking algorithm that combines the information from tracked faces with the recognized identity to use in the further frames. This tracking algorithm improves the overall processing time and accuracy. The proposed system performance is tested in real-time experiments applied in an HRI study. Our proposed framework achieves real-time capabilities with an average of 99%, 95%, and 97% precision, recall, and F-score respectively. In addition, we implemented our system as a modular ROS package that makes it straightforward for integration in different real-world HRI systems.

COVID-19 is the repugnant but the most searched word since its outbreak in November 2019 across the globe. The world has to battle with it until an effective solution is developed. Due to the advancement in mobile and sensor technology, it is possible to come up with Internet of things-based healthcare systems. These novel healthcare systems can be proactive and preventive rather than traditional reactive healthcare systems. This article proposes a real-time IoT-enabled framework for the detection and prediction of COVID-19 suspects in early stages, by collecting symptomatic data and analyzing the nature of the virus in a better manner. The framework computes the presence of COVID-19 virus by mining the health parameters collected in real time from sensors and other IoT devices. The framework is comprised of four main components: user system or data collection center, data analytic center, diagnostic system, and cloud system. To point out and detect the COVID-19 suspected in real time, this work proposes the five machine learning techniques, namely support vector machine (SVM), decision tree, naïve Bayes, logistic regression, and neural network. In our proposed framework, the real and primary dataset collected from SKIMS, Srinagar, is used to validate our work. The experiment on the primary dataset was conducted using different machine learning techniques on selected symptoms. The efficiency of algorithms is calculated by computing the results of performance metrics such as accuracy, precision, recall, F1 score, root-mean-square error, and area under the curve score. The employed machine learning techniques have shown the accuracy of above 95% on the primary symptomatic data. Based on the experiment conducted, the proposed framework would be effective in the early identification and prediction of COVID-19 suspect realizing the nature of the disease in better way.

Due to advancement in digital multimedia technologies, huge amount of video contents in such arenas as sports, surveillance, news, etc., is becoming widely available. Due to factors like presence of inactive frames, inter-view dependencies and frequent illumination changes, whole video content may not be important. Moreover, users may not have the adequate time to access/ store the entire video in real time. We propose a machine learning ensemble method to summarize the events in multiview videos, as a solution to the aforementioned issues. In order to make the accurate decision on keyframes, we trained our ensembles using meta approach where the inter-dependency and illumination changes of views are considered in the training phase. Experimental results on two benchmark datasets show that our model outperforms the state-of-the-art models with the best Recall and F-measure. The computing analysis of the model also indicates that it meets all requirements of real-time applications.

In this paper, we address a few important image analysis problems, which are fundamental to the design of Perceptual User Interface. We use an inexpensive stationary desktop camera to collect the video streams and use them as input to the system. We present an algorithm for segmenting moving foreground object of interest from a complex, but stationary background. This algorithm can cope with illumination changes due to shadows, Automatic Exposure Correction and long term illumination changes in the environment. The segmentation is done real time and works well for both indoor and outdoor scenes. The detected foreground is recognized as human being based on head shoulder profile.

Automatic detection of human faces in the environment is essential and needed to develop a mobile service robot so that it can detect and communicate with surrounding humans. Since the use of shape geometrical features alone is not good enough for reliable detection, color cues are used to augment the reliability of face detector. While color adds new information, it is problematic due to the dependence of image colors on the ambient illumination. Reduction of the illumination effect on image colors is therefore an important preprocessing step if color information is to be utilized for face detection. The methods presented in the previous literature to account for the illumination changes requires that the possible and expected illumination conditions to be specified a priori and the system should be trained in some way to account for the expected image changes. We propose a system of illumination preprocessing that does not require neither prior knowledge nor experimentation with the illuminants. We use a simple attachment with the camera to measure the illumination color and the image colors are compensated for illumination changes in real time using the reflection from the reference attachment nose. The system is implemented and experiments showed the success of the system to detect human faces despite moderate changes of the illumination conditions

Indirect augmented reality (IAR) employs a unique approach to achieve high-quality synthesis of the real world and the virtual world, unlike traditional augmented reality (AR), which superimposes virtual objects in real time. IAR uses pre-captured omnidirectional images and offline superimposition of virtual objects for achieving jitter- and drift-free geometric registration as well as high-quality photometric registration. However, one drawback of IAR is the inconsistency between the real world and the pre-captured image. In this paper, we present a new classification of IAR inconsistencies and analyze the effect of these inconsistencies on the IAR experience. Accordingly, we propose a novel IAR system that reflects real-world illumination changes by selecting an appropriate image from among multiple pre-captured images obtained under various illumination conditions. The results of experiments conducted at an actual historical site show that the consideration of real-world illumination changes improves the realism of the IAR experience.