Tracking a face for knowledge-based coding of videophone sequences

This paper proposes a filtering approach based on global motion estimation (GME) and global motion compensation (GMC) as pre-processing and post-processing for video CODEC. For the pre-processing of video CODEC, group-of-pictures (GOP), i.e., basic unit for GMC and reference frames are first defined for an input video sequence. Next, GME and GMC are sequentially performed for every frame in each GOP. Finally, block-based adaptive temporal filter is applied between global motion compensated frames prior to video encoding. For the post-processing at the end of decoder, every decoded frame is inversely motion-compensated by using the transmitted global motion information. The holes which are generated during inverse motion compensation can be filled using the reference frames. Experimental results the proposed algorithm provides higher BD-PSNRs than before-use on H.264 and HEVC platforms.

In Versatile Video Coding (VVC), local affine motion compensation (LAMC) is adopted to handle complex motions, such as rotation and zooming. However, it is inefficient to use LAMC to handle the global motion due to the following two reasons. First, the use of LAMC may lead to some extra bit cost on the affine motion model parameters. Second, the precision of LAMC is restricted by the MV precision of the control points. Therefore, in this paper, we propose a global homography motion compensation (GHMC) framework to better characterize the global motion. For each coding block, an extra mode is added to perform motion compensation based on an 8-parameter global homography motion model. In addition, an extrapolation scheme is designed to derive the parameters from reference frames to save the bit cost for signaling them. The proposed framework is implemented into the VVC reference software VTM-6.0. Experimental results show that, on average, 0.69% and 0.66% BD-rate reduction is achieved under Low Delay P and Low Delay B configurations, respectively, for sequences with rich complex global motions.

Current Moving Object Detectors in airborne Region of Interest (ROI) coding systems for police surveillance applications used on-board of UAVs are often based on Global Motion Estimation (GME) techniques. Since in these scenarios the camera is moving, simple background removal approaches cannot be applied without a Global Motion Compensation (GMC). Common GMC algorithms assume the ground to be planar, allowing the pixels of the previous frame to be motion compensated into the current frame by applying a projective transformation. The difference image between the compensated frame and the current frame emphasis regions containing possible motion. Such moving object detectors are great in terms of run-time efficiency but are known to lack in terms of accuracy - especially for unstructured regions of moving objects - as well as the robustness against noise. Superpixel segmentation was recently proposed to overcome the issue of the imprecise region cuts given by the difference image. It provides a greatly improved true positive detection rate, but unintentionally also increases the area of false positives. This paper proposes the use of a mesh-based GME and GMC to detect the moving object regions wherein a cluster filter eliminates errors in the optical flow by assuming a smooth vector field as the global motion model. In doing so we improve the coding efficiency of the fully automatic ROI coding system by more than 24% for moving object areas conserving the detection benefits of the integration of superpixel segmentation.

Most standard video compression schemes such as H.264/AVC involve a high complexity encoder with block motion estimation (ME) engine. However, applications such as video reconnaissance and surveillance using unmanned aerial vehicles (UAVs) require a low complexity video encoder. Additionally, in such applications, the motion in the video is primarily global and due to the known movement of the camera platform. Therefore in this work, we propose and investigate a low complexity encoder with global motion based frame prediction and no block ME. We show that for videos with mostly global motion, this encoder performs better than a baseline H.264 encoder with ME block size restricted to 8×8. Furthermore, the quality degradation of this encoder with decreasing bit rate is more gradual than that of the baseline H.264 encoder since it does not need to allocate bits across motion vectors (MVs) and residue data. We also incorporate a spectral entropy based coefficient selection and quantizer design scheme that entails latency and demonstrate that it helps achieve more consistent frame quality across the video sequence.

A global motion compensation (GMC) scheme is described and implemented for low bitrate coding. Experiments show that GMC gives excellent coding results. When coupled with the H.263 low bitrate video coding standard, GMC is capable of achieving up to 27% bitrate saving (or 1.8 dB gain in PSNR) for a high motion sequence such as Stefan. For a moderate motion sequence such as Cost-guard, we obtained an 8% bitrate saving over H.263 at the same quality using GMC.

A motion compensated de-interlacing method with adaptive global motion estimation and compensation is proposed to recover the defects of interlaced video sequence with camera panning, rotating or zooming. GME and GMC are used to recover the change of the whole picture due to camera motions. Two local motion compensated de-interlacing methods are proposed and applied to de-interlace the interlaced video sequences with or without global motion respectively. SAD checking and global/local motion comparator can be used as a block based mode decision system for intra/MC/GMC modes. The proposed algorithm could achieve higher image quality of interlaced video sequences than any other usual de-interlacing algorithm on progressive devices.

Moving object segmentation directly from compressed video stream is a challenging task in video signal processing community. A novel moving object detection algorithm for H.264/AVC stream is presented in this paper. Firstly, a series of operations, including spatial-temporal normalization, bidirectional accumulation, median filtering and thresholding magnitudes, are conducted on the raw motion vector (MV) field so as to gain a robust MV field, which can indicate the moving region reliably. Then global motion compensation in compressed domain is implemented if there exist global motion caused by camera. Finally, the macroblock coded mode with variable size is combined with the temporal information to locate the foreground object. Its effectiveness of the proposed algorithm has been demonstrated in our experimental results. The main advantage of the algorithm lies in its robustness and real-time performance.

A novel motion representation and estimation scheme is presented that leads to improved coding efficiency of block-based video coders like, e.g., H.263. The proposed scheme is based on motion-compensated prediction from more than one reference frame. The reference pictures are warped or globally motion-compensated versions of the previous frame. Affine motion models are used as warping parameters approximating the motion vector field. Motion compensation is performed using standard block matching in the multiple reference frames buffer. The frame reference and the affine motion parameters are transmitted as side information. The approach is incorporated into an H.263-based video codec at minor syntax changes and embedded into a rate-constrained motion estimation and macroblock mode decision frame work. In contrast to conventional global motion compensation, where one motion model is transmitted, we show that multiple global motion models are of benefit in terms of coding efficiency. Significant coding gains in comparison to TMN-10, the test model of H.263+, are achieved that provide bit-rate savings between 20% and 35% for the various sequences tested

Inferring dynamic behavior of the heart from its image sequences is a very important research area in biomedical engineering. It provides an invaluable tool for noninvasive evaluation of myocardial functions. This paper presents estimation algorithms for the analysis of heart motion and deformation over a cardiac cycle, as well as the visualization techniques for the animation of moving heart evolution. The first part of the paper is devoted to the analysis of the heart motion and deformation. The research is based on the general belief that the human heart undergoes both global motion and local deformation, and is conducted on the angiographic data of the heart. The authors identify the global motion as the relative position and orientation change of the heart as a whole and estimate the motion parameters from the 3- D data of the bifurcation points. They also develop a recursive algorithm for estimating global motion and object shape in order to combat the biased distribution of the bifurcation points. Upon compensation for the global motion, a tensor analysis based approach is introduced to parameterize the deformation of localized region. The estimated stretch tensors give the directions and magnitudes of extreme deformation for each localized region. In the second part of the paper, several visualization techniques are presented to vividly examine the spatial and time varying nature of the heart. Animations of global motion compensation and local deformation evolution are generated using the original data and estimation results. Display showing the heart in slow motion is created by interpolating original and estimated data between image frames. Visualization operations such as camera and lighting manipulation, polygon outlining, color coding, and so on are applied to the data to reveal the complex nature of the beating heart.© (1991) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In this paper, we present an improved approach to predictive video decoding based on global and local motion reliability. The framework consists of three processing stages. Global motion (GM) estimation and motion reliability analysis are the key components in the first stage, where we model global motion and refine the MV field. In the second stage, we predict local and global motion for the target frame, and determine corresponding weights based on the Linear Minimum Mean Square Error (LMMSE) criterion. Finally in the third stage, a temporal interpolator is applied to compose two future frames, which are linearly combined to form the final predicted frame. Our results indicate the proposed method achieves better visual quality compare to other state-of-the-art predictive decoding approaches, particularly in sequences involving moving camera and objects.

It is well known that moving object detection under complex background becomes more difficult because of moving cameras. According to the fact that background and objects have different motion, the moving scene can be decomposed into different regions with respect to their motion by means of a radial basis function(RBF) learning scheme. After global background motion compensation, five-dimensional (5-D) feature vectors include pixel intensities, current pixel coordinates and pixel dense optical flow field extracted from image sequences are treated as the inputs of the RBF network. The learning algorithm for the network minimizes a cost function derived from the Bayesian estimation theory. Each output unit of the network is associated to a moving object. Experimental results indicate the algorithm's validity after many complex sequences are tested.

Automatic detection and tracking of subviral particles in image sequences is an indispensable supportive method for modern medicine research programs. This paper describes the development of a highly adaptable camera-to-world system motion invariant tracking algorithm. A translation compensation is obtained by cross correlations. Particles are detected by an implemented existing algorithm. The detected particles are linked by solving a Linear Assignment Problem. For highly stable results the tracks are improved by Kalman filtering. The algorithm is tested on simulated sequences. The results show a great ability for stable tracking.

This paper presents a practical approach to detecting shot cuts and extracting keyframes from video sequences. Shot cut detection has two stages - global motion compensation, followed by an adaptive thresholding algorithm. The motion information is further utilized to extract representative keyframes. Special consideration has been given to achieving real-time performance on a regular PC, which led to a motion estimation algorithm of linear complexity.

A sequential dual-energy subtraction technique for image-guided radiation therapy (IGRT) was developed. Here, we report on a computerized method for creating sequential soft-tissue images and the accuracy of tracking targets on the images obtained, in comparison to conventional fluoroscopic images. Two sets of sequential chest images during respiration of a normal subject were obtained with X-rays of different energy separately with a flat-panel detector (FPD). Sequential soft-tissue images were created from the two sets of sequential images consisting of real-time images and reference template images, respectively. The creation of sequential soft-tissue images consisted of three steps: one-to-one image correspondence of the two sequential images, image registration, and image subtraction in each frame. Motion tracking of lung vessels was then performed by the template-matching technique. For evaluation of the accuracy of motion tracking on the sequential soft-tissue images, the results were compared with those on the original sequential images. Sequential soft-tissue images provided more accurate tracking than the original images (P < 0.01). There was no significant error throughout all frames in the soft-tissue images, whereas the rib shadow introduced a tracking error in the original images. The maximum errors were 4.1 +/- 0.3 mm in the sequential soft-tissue images and 28.1 +/- 20.0 mm in the original images. In conclusion, sequential soft-tissue images were helpful for tracking of a target affected by respiratory motion. Dual-energy subtraction has the potential to improve the accuracy of IGRT without implanted markers.

This paper deals with problems of surface object recognition in urban environmental conditions in infrared sequences using the template matching technique. Target recognition is always necessary in the selection and tracking the defined objects. The objects feature is changed in dependence on time and space condition such as weather condition, urban environment, etc. In this article, we proposed a method for targets recognition using template matching technique in infrared (IR) video sequences. A user simply chooses the given object at some point during detection. On the basis of feature of selected object, the algorithm employed the template matching techniques to find the object. The designed algorithm was tested in program MATLAB and MATLAB - SIMULINK. The paper will deal with a new algorithm using template matching. The idea of the paper will approach to reduce processing time and increase precision for object detection and selection in using template matching technique. There are many similarities criteria for template matching that are sum of absolute differences (SAD), locally scaled sum of absolute difference (LSAD), zero - mean sum of absolute differences (ZSAD), sum of squared differences (SSD), etc. In this paper we have employed the similarity criteria normalized cross correlation. The cross correlation is not invariant to changes in image intensity such as lighting conditions, and the range of correlation coefficient is dependent on the size of the feature, while we can normalize for the effect of changing intensity and template size by using normalized cross correlation. The basic principle of the algorithm is based on the assumption the object is selected at time t with the center of mass T and we also know the greatest relative velocity between the camera and the target ν ct . Thus, the greatest distance which the target can move after the interval Δt will be limited in the circle with the center T at time t and radius R. That means the size of compared image will be smaller than the size of original frame in infrared video sequence. Therefore, the computational time will be faster. In order to reduce the unnecessary computational time of the template matching technique in whole frame, we need to keep the requirement of the selected maximum size of the target is smaller than a half of the size of the frame. It means that the maximum size of the image ƒ(x, y) is equal to the size of the frame in video sequence.