Real-Time Image Stabilization for Digital Video Cameras

The strap-down missile-borne image guidance system can be easily affected by the unwanted jitters of the motion of the camera, and the subsequent recognition and tracking functions are also influenced, thus severely affecting the navigation accuracy of the image guidance system. So, a real-time image stabilization technology is needed to help improve the image quality of the image guidance system. To satisfy the real-time and accuracy requirements of image stabilization in the strap-down missile-borne image guidance system, an image stabilization method based on optical flow and image matching with binary feature descriptors is proposed. The global motion of consecutive frames is estimated by the pyramid Lucas-Kanade (LK) optical flow algorithm, and the interval frames image matching based on fast retina keypoint (FREAK) algorithm is used to reduce the cumulative trajectory error. A Kalman filter is designed to smooth the trajectory, which is conducive to fitting to the main motion of the guidance system. Simulations have been carried out, and the results show that the proposed algorithm improves the accuracy and real-time performance simultaneously compared to the state-of-art algorithms.

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

This paper is dedicated to video stabilization for fast moving photographing based on feature points classification, especially for the moving camera in various speeds. The proposed method mainly consists of feature point detection and classification, calculating global motion vector and rotation angle of frame, and frame compensation. It is first to search feature points and then classify these feature points into foreground (i.e., moving object) type and background type based on multiple view geometry and DBSCAN algorithm. Then, the global feature points and their optical-flows are derived and utilized for calculating the global motion vector and global rotation angle of frame. Finally, both global motion vector and global rotation angle are refined through motion smoothing using a Kalman filter for providing better frame compensation to generate stable frames. Experimental results show that the proposed method can moderately stabilize the frames captured by a moving camera.

In this paper a new iterative motion-based segmentation algorithm is presented for object-based video coding. At first, dense motion fields between image pairs are obtained by a multiresolution motion estimation algorithm. Based on these motion fields a set of global motion vectors are determined by an iterative vector quantisation procedure. Using the global motion vectors, the images are segmented into homogenous regions. Covered and uncovered areas of the background and objects are properly recognized by examining two consecutive motion fields. Experimental results for artificial as well for real-life video have shown that the proposed algorithm results in good quality segmentation label fields while it requires only moderate computations.

To achieve a higher compression ratio for very low bit rate (VLBR) video compression, motion estimation has to be more reliable and easier to code. Many segmentation based techniques try to approximate several motion vectors by a single global motion vector so that fewer vectors are required. However, a global motion vector may not be a good representation for the original vectors and the bit saving is not worth the loss. This paper proposes a new scheme for motion vector representation. Many candidates from full search block matching are selected for merging possibility. Blocks which experience similar displacement are combined to form a single object. Upon compensation for object translation or global compensation, accurate motion of each block can be identified by local compensation within an object. Instead of presenting object movement as a vector, a specific index is assigned to reduce the overhead bits. Based on extensive simulation, the proposed method with an appropriate threshold shows an improvement over standard block based hybrid schemes.

In the present paper, an adaptive α - β filtering algorithm is adapted for the motion compensation of EIS(electronic image stabilization). It is characterized of less computation , and easy to be realized in real-time. Inter frame global motion vectors were predicted by feature corners matching based motion estimation. The global camera motion was defined in terms of constant velocity motion models, and adaptive α - β filtering was employed to facilitate smooth operation. Experimental results for many video sequences show that the method can eliminate high frequency dithering , remain the meaningful motion of camera and realize real-time image stabilization satisfactorily. The present algorithm is more robust and higher veracity comparing with other algorithms.

In recent years, a digital video stabilization improving the results of hand-held shooting or shooting from mobile platforms is the most popular approach. In this chapter, the task of digital video stabilization in static scenes is investigated. The unwanted motion caused by camera jitters or vibrations ought to be separated from the objects motion in a scene. Our contribution connects with the development of deblurring method to find and improve the blurred frames, which have strong negative influence on the following processing results. The use of fuzzy Takagi-Sugeno-Kang model for detection the best local and global motion vectors is the novelty of our approach. The quality of test videos stabilization was estimated by Peak Signal to Noise Ratio (PSNR) and Interframe Transformation Fidelity (ITF) metrics. Experimental data confirmed that the ITF average estimations increase up on 3–4 dB or 15–20 % relative to the original video sequences.

To improve the real-time image acquisition and stabilization ability of dynamic robotic dog video acquisition devices, a dynamic robotic dog video image stabilization algorithm based on local low-rank regularization is proposed. Firstly, by studying the image acquisition system of a dynamic robotic dog video acquisition device, a real-time image acquisition and stabilization algorithm suitable for this system was developed based on the prior knowledge of motion morphology components. Then, the rolling curtain effect of the loading video capture device was eliminated through the principle of grid. These results confirmed that the improved algorithm achieved stable performance with approximate Gaussian weights between 170 frames and 230 frames, with the highest ITF value at τ=1 and an average IFT value of 33.42 after 20 iterations. Then, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</i> =30, β=1000, and ω <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> were selected as the default parameters. The average cropping rate of the improved algorithm was only 0.61, the average distortion was 0.42, and the average stability was 0.69. The improved algorithm scored 3.33 in the "crowd" category of videos, ranking first. After verification, the real-time image acquisition and stabilization algorithm designed by the research institute had good real-time performance and stability. It can meet the real-time image acquisition and stabilization requirements of dynamic robotic dog video acquisition devices.

The application in image and signal processing, for programmable devices such as EPLDs and FPGAs combined with high volume components developed for the TV set top box and PC markets, can provide a flexible solution where a number of conflicting requirements exist. The mix of technologies provides the designer with potentially higher performance at lower cost and size over DSP or RISC processor solutions for the rugged environment market applications.

In outdoor surveillance systems, block matching is usually used for image stabilization but prohibited for real-time video rate application due to its tremendous amount of calculation burden, though many strategies induced to save searching time. In this paper, a faster and robust image stabilization technique is presented to eliminate the unwanted video motion during watching time. Unlike other image stabilization algorithms using two projection vectors in each x or y direction, we use only one projection vector to estimate the translation motions in 2 directions between image sequences. Our stabilization algorithm compare the template vector with vectors in the referenced vector matrix constructed and find out the minimum vector-wise MSE. At last, inspired by the block matching method, we introduce its searching strategies into our algorithm to decrease the computation times, thus making real-time video stabilization could be implemented on normally available PC.

The use of global Atmospheric Motion Vectors (AMV) satellite observations in the meteorological data assimilation system based on Local Ensemble Transform Kalman Filter (LETKF) algorithm is considered. The height assignment is the most crucial error source for AMV observations. To reduce its impact, the AMV height reassignment method is implemented; it is based on the consistency coefficient bet ween the observed and the background winds. The other way to improve the analysis quality is a more accurate specification of AMV observation errors. This necessitates the use of the nondiagonal observation-error covariance matrix R in the data assimilation scheme. The first results of these studies are presented. It is demonstrated that the use of AMV observations in the data assimilation system reduces the errors of forecasts computed from the initial data of this system.

RNA computing is a new intelligent optimization algorithm, which combines computer science and molecular biology. Aiming at the weakness of slow convergence rate and poor global search ability in the basic ant colony optimization algorithm due to the unreasonable selection of parameters, this paper utilizes the combination of RNA computing and basic ant colony optimization algorithm to overcome the defects. An improved ant colony optimization algorithm based on RNA computing is proposed. In the iterative process of ant colony optimization algorithm, transformation operation, recombination operation and permutation operation in RNA computing are introduced to optimize the initial parameters including importance factor of pheromone trail α, importance factor of heuristic function β and pheromone evaporation rate ρ to improve the convergence efficiency and global search ability. The performance of the algorithm is evaluated on five instances of the library of traveling salesman problems (TSPLIB) and six typical test functions. The experimental results demonstrate that the proposed RNA-ant colony optimization algorithm is superior than basic ant colony optimization algorithm in optimization ability, reliability, convergence efficiency, stability and robustness.

Electronic digital image stabilization technique plays important roles in video surveillance or object acquisition. Researchers have presented many useful algorithms, which can be classified to three kinds: gray based methods, transformation based methods and feature based methods. When scenario is simple or flat, feature based methods sometimes have imperfect results. Transformation based methods usually accompany large computation cost and high computation complexity. Here we presented an algorithm based on gray projection which divided the whole image into four sub-regions: the upper one, the bottom one, the left one and the right one. For making the translation estimation easier, a central region is also chosen. Then the gray projections of the five sub-regions were counted. From the five pairs of gray projections five group offsets including rotation and translation were obtained via cross correlation between current frame and reference frame gray projections. Then according to the above offsets, the required parameters can be estimated. The expected translation parameters(x axis offset and y axis offset) can be estimated via the offsets from the central region image pair, the rotation angle can be calculated from the left four groups offsets. Finally, Kalman filter was adopted to compute the compensation. Test results show that the algorithm has good estimation performance with less than one pixel translation error and 10 percent rotation error. Based on this kind of gray projection algorithm, a real-time electronic digital image stabilization system has been designed and implemented. System tests demonstrate the system performance reaches the expected aim.

Advanced driver assistance systems (ADAS) are designed to increase driver's situational awareness and road safety by providing essential information, warning and automatic intervention to reduce the possibility/severity of an accident. Of the various types of ADAS modalities available, camera based ADAS are being widely adopted for their usefulness in varied applications, overall reliability and adaptability to new requirements. But camera based ADAS also represents a complex, high-performance, and low-power compute problem, requiring specialized solutions. This paper introduces a high performance front camera ADAS based on a small area, low power System-on-Chip (SoC) solution from Texas Instruments called Texas Instruments Driver Assist 3x (TDA3x). The paper illustrates compute capabilities of the device in implementation of a typical front camera ADAS. The paper also introduces key programming concepts related to heterogeneous programmable compute cores in the SoC and the software framework to use those cores in order to develop the front camera solutions. These aspects will be of interest not only to the ADAS developers but for computer vision and compute intensive embedded system development.

This paper describes the development of a novel algorithm to tackle the problem of real-time video stabilization for unmanned aerial vehicles (UAVs). There are two main components in the algorithm: (1) By designing a suitable model for the global motion of UAV, the proposed algorithm avoids the necessity of estimating the most general motion model, projective transformation, and considers simpler motion models, such as rigid transformation and similarity transformation; (2) to achieve a high processing speed, optical flow-based tracking is employed in lieu of conventional tracking and matching methods used by state-of-the-art algorithms. These two new ideas resulted in a real-time stabilization algorithm, developed over two phases. Stage I considers processing the whole sequence of frames in the video while achieving an average processing speed of 50 fps on several publicly available benchmark videos. Next, Stage II undertakes the task of real-time video stabilization using a multi-threading implementation of the algorithm designed in Stage I.