Adaptive Subsampling for ROI-Based Visual Tracking: Algorithms and FPGA Implementation

The Kalman filter has long been regarded as the optimal solution to many applications in computer vision for example the tracking objects, prediction and correction tasks. Its use in the analysis of visual motion has been documented frequently, we can use in computer vision and open cv in different applications in reality for example robotics, military image and video, medical applications, security in public and privacy society, etc. In this paper, we investigate the implementation of a Matlab code for a Kalman Filter using three algorithm for tracking and detection objects in video sequences (block-matching (Motion Estimation) and Camshift Meanshift (localization, detection and tracking object)). The Kalman filter is presented in three steps: prediction, estimation (correction) and update. The first step is a prediction for the parameters of the tracking and detection objects. The second step is a correction and estimation of the prediction parameters. The important application in Kalman filter is the localization and tracking mono-objects and multi-objects are given in results. This works presents the extension of an integrated modeling and simulation tool for the tracking and detection objects in computer vision described at different models of algorithms in implementation systems.

In the extended Kalman filter (EKF) and the unscented Kalman filter (UKF) of non-linear system, the mathematical model and the noise statistics are essential; this limitation was settled by adaptive algorithm. The adaptive extended Kalman filter (AEKF) and adaptive unscented filter (AUKF) were proposed to solve the filtering problem of the system with unknown mathematical model or noise statistics in information fusion. The simulation result shows that the adaptive algorithm in EKF and UKF can improve the performance of filter and decrease the computing time.

Given the complex traffic factors and strong mobility of a transit vehicles, it is difficult to predict its travel time with the conventional prediction models. The emphasis of this research effort is to improve the adaptability of the Kalman filter model with regard to transit bus travel time prediction. In order to resolve the filter divergence and calculated divergence in the prediction process, an adaptive fading Kalman filter algorithm is put forward in this paper. Compared with the conventional Kalman filter prediction model, a "forgotten factor" is added to the adaptive fading Kalman filter algorithm to restrain the influence of the old data on the filter. Furthermore, it can correct the predictive value and the model parameters with the latest measured value, hence higher adaptability and predictive accuracy. Finally, the GPS data of transit vehicles in the city of Yichun is used in the paper to verify the effectiveness of the new model. The results reveal that the adaptive fading Kalman filter algorithm is more effective than the conventional one. It performs better in terms of convergence and dynamic adaptability, implying a brighter application prospect.

Real time object tracking is an essential part of surveillance and robot applications. The performance of any object tracking system depends on its accuracy and its ability to deal with various sizes of objects, for better results the tracking should happen at a high speed. Object tracking can be defined as the process of segmenting the region of interest from the video sequence and keeping track of the motion in order to extract useful information. The basic fundamentals of video surveillance such as object detection, tracking and recognition using multiple cameras. Whenever an object enters a video frame, it is detected using background subtraction and the detected objects are tracked with a novel Bayesian Kalman filter with simplified Gaussian mixture (BKF-SGM). Object recognition algorithm is applied using Gait method which is employed to support multiple object tracking with its robust performance. Processing of image is done in MATLAB to get detection, tracking and recognized results.

This chapter presents a multi-object tracking system using scale space representation of objects, the method of linear assignment and Kalman filter. In this chapter basically two very prominent problems of multi object tracking have been resolved; the two prominent problems are (i) irrespective of the size of the objects, tracking all the moving objects simultaneously and (ii) tracking of objects under partial and/or complete occlusion. The primary task of tracking multiple objects is performed by the method of linear assignment for which few cost parameters are computed depending upon the extracted features of moving objects in video scene. In the feature extraction phase scale space representation of objects have been used. Tracking of occluded objects is performed by Kalman filter.

Video prediction is the challenging task of generating the future frames of a video given a sequence of previously observed frames. This task involves the construction of an internal representation that accurately models the frame evolutions, including contents and dynamics. Video prediction is considered difficult due to the inherent compounding of errors in recursive pixel level prediction. In this paper, we present a novel video prediction system that focuses on regions of interest (ROIs) rather than on entire frames and learns frame evolutions at the transformation level rather than at the pixel level. We provide two strategies to generate high-quality ROIs that contains potential moving visual cues. The frame evolutions are modeled with a transformation generator that produces transformers and masks simultaneously, which are then combined to generate the future frame in a transformation-guided masking procedure. Compared with recent approaches, our system is able to generate more accurate predictions by modeling the visual evolutions at the transformation level rather than at the pixel level. Focusing on ROIs avoids a heavy computational burden and enables our system to generate high-quality long-term future frames without severely amplified signal loss. Moreover, our system is able to generate diverse plausible future frames, which is important in many real-world scenarios. Furthermore, we enable our system to perform video prediction conditioned on a single frame by revising the transformation generator to produce motion-centric transformers. We test our system on four datasets with different experimental settings and demonstrate its advantages over recent methods, both quantitatively and qualitatively.

Given an object model and a black-box measure of similarity between the model and candidate targets, we consider visual object tracking as a numerical optimization problem. During normal tracking conditions when the object is visible from frame to frame, local optimization is used to track the local mode of the similarity measure in a parameter space of translation, rotation and scale. However, when the object becomes partially or totally occluded, such local tracking is prone to failure, especially when common prediction techniques like the Kalman filter do not provide a good estimate of object parameters in future frames. To recover from these inevitable tracking failures, we consider object detection as a global optimization problem and solve it via Adaptive Simulated Annealing (ASA), a method that avoids becoming trapped at local modes and is much faster than exhaustive search. As a Monte Carlo approach, ASA stochastically samples the parameter space, in contrast to local deterministic search. We apply cluster analysis on the sampled parameter space to redetect the object and renew the local tracker. Our numerical hybrid local and global mode-seeking tracker is validated on challenging airborne videos with heavy occlusion and large camera motions. Our approach outperforms state-of-the-art trackers on the VIVID benchmark datasets.

This paper introduces a new detection method of objects in micro-scale based on dynamic region of interest (ROI) method combined with pyramid template matching (PTM) approach. The dynamic ROI method works based on the position predicting, which is provided by the Kalman filter (KF). Rather than searching the entire image, the detection ROI is located at a limited region. Experimental results show that the detected positions match well with the real motion path. Moreover, the dynamic ROI method reduces the computation time by 30% as compared with the PTM algorithm.

We present an adaptive fast-approximation for sensor configuration which finds near-optimal placements and sensor field of views (FoV). The fast-approximation, either via partition-based or density-based cluster analysis, adapts based on the relation between statistical uncertainty of the path plan and environmental uncertainty. The sensor configurations are performed over regions of interest which most directly influence the path-planning efforts. These regions of interest can include exploratory paths by sampling the probabilistic environment model. The path-planning efforts aim to decide upon a path which minimizes an agent’s exposure to threats in an unknown static environment. The noisy sensor network observations are used to construct a threat field estimate using Gaussian Process Regression each iteration with a stationary kernel and heteroscedastic gaussian likelihood. The optimization of a task-driven information gain determines optimal sensor configurations when maximized. The numerical performance of the direct optimization and the adaptive cluster analysis method is presented. Finally, we show that the cluster centers can be utilized as a dimensionality reduction technique for FoV optimization whereby we only optimize FoV radial coverage.

As of 2010 data centers use 1.5% of global electricity production and this is expected to keep growing [1]. There is a need for a near real-time power consumption modeling/monitoring system that could be used at scale within a Software Defined Data Center (SDDC). The power consumption models and information they provide can then be used to make better decisions for data center orchestration, e.g., whether to migrate virtual machines to reduce power consumption. We propose a scalable system that would 1) create initial power consumption models, as needed, for data center components, and 2) could be continually refined while the components are in use. The models will be used for the near real-time monitoring of power consumption, as well as predicting power consumption before and after potential orchestration decisions. The first step towards this goal of whole data center power modeling and prediction is to be able to predict the power consumption of one server effectively, based on high level utilization statistics from that server. In this paper we present a novel method for modeling whole system power consumption for a server, under varying random levels of CPU utilization, with a scalable random forest based model, that utilizes statistics available at the data center management level.

Distributed protocols that run in dynamic environments such as the Internet are often not able to use an upper bound on the number of potentially participating processes. In these settings adaptive and uniform algorithms are desirable where the step complexity of all operations is a function of the number of concurrently participating processes (adaptive) and the algorithm does not need to know an upper bound on the number of participating processes (uniform). Adaptive algorithms, however, are generally not adaptive with respect to their memory consumption – if no upper bound on the number of participating processes is known in advance – they require unbounded MWMR registers and an unbounded number of such registers (even if only finitely many distinct processes appear), making them impractical for real systems. In this paper we ask whether this must be the case: Can adaptive algorithms where no upper bound on the number of participating processes is known in advance be uniformly implemented with finite memory (if only finitely many distinct processes keep reappearing)? We will show that in the dynamic setting it is impossible to implement long-lived adaptive splitters, collect and renaming with infinitely many bounded MWMR registers, making such adaptive algorithms impractical in dynamic settings. On the positive side we provide algorithms that implement a long-lived uniform adaptive splitter if unbounded registers are available and that implement a non-uniform adaptive splitter with finitely many bounded registers if an upper bound on the number of participating processes is known in advance.KeywordsShared MemoryAdaptive AlgorithmBusy PeriodMutual ExclusionDistinct ProcessThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Automatic detection of region of interest (ROIs) in a complex image or video, such as an angiogram or endoscopic neurosurgery video, is a critical task in many medical image and video processing applications. In this paper, we present a new method that addresses several challenges in automatic detection of ROI of neurosurgical video for ROI coding which is used for neurophysiological intraoperative monitoring (IOM) system. This method is based on an object tracking technique with multivariate density estimation theory, combined with the shape information of the object. By defining the ROIs for neurosurgical video, this method produces a smooth and convex emphasis region within which surgical procedures are performed. A large bandwidth budget is assigned within the ROI to archive high-fidelity Internet transmission. Outside the ROI, a small bandwidth budget is allocated to efficiently utilize the bandwidth resource. We believe this method also can be used to image-guiduance surgery (IGS) systems to track the positions of surgical instruments in the physical space occupied by the patient after some improvement.

Digital image sensors have been utilized in various applications including, but not limited to, consumer electronics, machine vision, and security surveillance. In the early days of digital cameras, charge-coupled-device (CCD) technology was dominant because of its superior imaging quality. In recent years, with the advancement of complementary-metal-oxide-semiconductor (CMOS) technology, CMOS image sensors have prevailed due to their lower power consumption and inexpensive fabrication cost. The design of commercial image sensors focuses on the improvement of imaging quality by increasing spatial resolution with smaller pixel size. When sensors without an on-chip computation capability are applied to the computer vision field, they produce massive amounts of raw image data. To extract valuable image features, such redundant raw data has to be transmitted and then processed by the vision processing system. This process wastes tremendous hardware resources and causes significant power consumption. Therefore, there is a growing demand for the smart image sensor that can perform on-chip image processing to directly export valuable image features such as spatial contrast or temporal motion. In many computer vision applications, motion features are widely used to identify active objects from stationary backgrounds. Once active regions with motion activities are extracted, more advanced processing such as image compression, object segmentation, and pattern recognition can be implemented on the acquired images. Visual motion detection can be achieved by measuring light intensity changes along with time on every pixel. Light intensities can be transformed into electrical signals in the form of currents, charges, or voltages. By detecting temporal variations in these signals with dedicated circuits, a motion-detection function can be implemented on image sensors. Various smart image sensors with on-chip motion feature-extraction capabilities have been reported in recent years. They can be classified into two main categories: “discrete mode” and “continuous mode”. In the first mode, an analog intensity image is acquired by integrating incident light within a certain period. A comparison of two consecutive frames can realize the motion-detection function. However, this algorithm suffers from a severe aliasing effect due to the discrete differentiation between two separate frames. In the “continuous mode”, incident light is directly converted into photo currents or intensity voltages without the integrating operation. By applying continuous differentiation on these signals, motion features can be extracted more efficiently. In order to report motion activities in real time, researchers innovatively developed an asynchronous address-event representation (AER) strategy to export motion address events from smart image sensors to post processors. Unfortunately, image processing on address events is different with the conventional frame-based strategy because of their asynchronous nature, and specific event-based algorithms have to be customized based on applications. The aim of this work is to design a smart CMOS image sensor that can extract and process motion features on the same chip so as to simplify the entire visual system with minimal computation burden. In order to achieve this ultimate goal, several smart motion-detection image sensors have been developed. The first image sensor was designed based on the temporal difference algorithm to on-chip export motion features, and this sensor is a compact visual system that integrates motion detection and feature processing on the same chip. In order to maximize photon sensitivity in every pixel, a feature-extraction image sensor based on an emerging 3D integration technology was developed. The sensor can extract either temporal motion or spatial contour in real time. Also, an event-clustering image sensor specialized for object tracking applications was proposed. This sensor continuously detects motion events and on-chip processes them in parallel to export specific “event-flow” features. Simulation results show that this image sensor significantly simplifies and accelerates visual object tracking systems.

In this paper, the design and FPGA implementation of a low-power adaptive Viterbi decoder with a constraint length of 9 and code rate of 1/2 is presented. In this design, a novel systolic array-based architecture with time multiplexing and arithmetic pipelining for implementing the adaptive Viterbi algorithm is used. A scheme for providing a tolerance to clock-to-data skew to avoid timing violation is proposed. A process of eliminating spurious toggles, for reducing power consumption, is also developed. It is shown that the total power consumption in the implementation of the adaptive algorithm can be reduced by up to 43% compared to that in the implementation of a corresponding non-adaptive Viterbi algorithm, with a negligible increase in the hardware.

Purpose: To increase image registration accuracy, and to remove the time‐ consuming manual region‐of‐interest (ROI) selection process for patient monitoring 3D surface imaging systems (AlignRT,VisionRT,UK), a method was developed to adaptively define an optimal ROI based on discovering topographically stable surfaces over time. Methods: Our method calculates the displacement and velocity of 3D surface points between a series of images captured in real‐time and the reference image. Using a set of selection criteria, the algorithm adaptively constructs a ROI by including only facial surfaces that move rigidly with the target and excludes surfaces that fluctuate locally such as the mandible regions and eyes. Data for five volunteers were acquired over various facial surface and motion types. A total of 27 data sets were analyzed, with 2–7 data sets for each volunteer and each data set contained consecutively captured 40–120 real‐time images. Results: The adaptive ROI selection algorithm correspondingly detected surfaces affected by different types of facial movement independent of target motions. It interpreted the eyes, eyebrows and mandible motions shown in the real‐time 3D images as independent of target motion even when AlignRT misinterpreted them, and excluded them from the final ROI. For 3D images taken with substantial facial motions, either most of the facial areas were excluded or only small areas with random patterns were included in the final ROI. The algorithm excluded the upper side and nose where the view of the cameras was blocked, and thus surface was not well‐ reconstructed in many of real‐time 3D images when couch was rotated greater than 40°. Conclusion: The method successfully located suitable ROI surfaces undergoing minimal displacement and temporal fluctuation. By automating the ROI selection process, the time and complexity of current ROI definition can be reduced, together with user‐dependent registration errors.