Foreground detection based on motion vector from compressed video

This paper presents an algorithm for automatically extracting significant motion trajectories in sports videos. Our approach consists of four stages: global motion estimation, motion blob detection, trajectory evolution and trajectory refinement. Global motion is estimated from the motion vectors in the compressed video using an iterative algorithm with robust outlier rejection. A statistical hypothesis test is carried out within the Block Rejection Map(BRM), which is the by-product of the global motion estimation, for the detection of motion blobs. Trajectory evolution is the process in which the motion blobs are either appended to an existing trajectory or are considered to be the beginning of a new trajectory based on its distance to an adaptive trajectory description. Finally, the extracted motion trajectories are refined using a Kalman filter. Experimental results on both indoor and outdoor sports videos demonstrate the effectiveness and efficiency of the proposed method.

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

In Versatile Video Coding (VVC), local affine motion compensation (LAMC) is adopted to handle complex motions, such as rotation and zooming. However, it is inefficient to use LAMC to handle the global motion due to the following two reasons. First, the use of LAMC may lead to some extra bit cost on the affine motion model parameters. Second, the precision of LAMC is restricted by the MV precision of the control points. Therefore, in this paper, we propose a global homography motion compensation (GHMC) framework to better characterize the global motion. For each coding block, an extra mode is added to perform motion compensation based on an 8-parameter global homography motion model. In addition, an extrapolation scheme is designed to derive the parameters from reference frames to save the bit cost for signaling them. The proposed framework is implemented into the VVC reference software VTM-6.0. Experimental results show that, on average, 0.69% and 0.66% BD-rate reduction is achieved under Low Delay P and Low Delay B configurations, respectively, for sequences with rich complex global motions.

We present a novel technique for the estimation of global nonrigid motion of an object's boundary without using point correspondences. A complete description of the motion of an object's boundary involves specifying a displacement vector at each point of the boundary. Such a description provides a large amount of information, which needs to be processed further for study of the global characteristics of the deformation. Nonrigid motion can be studied hierarchically in terms of global nonrigid motion and point-by-point local nonrigid motion. The technique presented gives a method for estimating a global affine or polynomial transformation between two object boundaries. The novelty of the technique lies in the fact that it does not use any point correspondences. Our method uses hyperquadric models to model the data and estimate the global deformation. We show that affine or polynomial transformation between two datasets can be recovered from the hyperquadric parameters. The usefulness of the technique is twofold. First, it paves the way for viewing nonrigid motion hierarchically in terms of global and local motion. Second, it can be used as a front end to other motion analysis techniques that assume small motion. For instance, most nonrigid motion analysis algorithms make some assumptions on the type of nonrigid motion (conformal motion, small motion, etc.) that are not always satisfied in practice. When the motion between two datasets is large, our algorithm can be used to estimate the affine transformation (which includes scale and shear) or a polynomial transformation between the two datasets, which can then be used to warp the first dataset closer to the second so as to satisfy the small-motion assumption. We present experimental results with real and synthetic two- and three-dimensional data.

In this paper, we present an improved approach to predictive video decoding based on global and local motion reliability. The framework consists of three processing stages. Global motion (GM) estimation and motion reliability analysis are the key components in the first stage, where we model global motion and refine the MV field. In the second stage, we predict local and global motion for the target frame, and determine corresponding weights based on the Linear Minimum Mean Square Error (LMMSE) criterion. Finally in the third stage, a temporal interpolator is applied to compose two future frames, which are linearly combined to form the final predicted frame. Our results indicate the proposed method achieves better visual quality compare to other state-of-the-art predictive decoding approaches, particularly in sequences involving moving camera and objects.

This paper presents two novel algorithms for estimating the (local and global) motion in a series of range images based on a polynomial expansion. The use of polynomial expansion has been quite successful in estimating optical flow in 2D imagery, but has not been used extensively in 3D or range imagery. In both methods, each range image is approximated by applying a high-order polynomial expansion to local neighborhoods within the range image. In the local motion algorithm, these approximations are then used to derive the translation or displacement estimation within the local neighborhoods from frame to frame within the series of range images (also known as range image flow). An iterative method for computing the local translations is presented. In the global motion algorithm, a global motion model framework is utilized to compute a global motion estimation based on the polynomial expansion of the range images. We evaluate the algorithms on several real-world range image sequences with promising results.

We have investigated the global and local motion tuning properties of curvature coding mechanisms using two shape after-effects believed to be mediated by curvature-sensitive mechanisms: the shape-frequency after-effect, or SFAE, and the shape-amplitude after-effect, or SAAE. The SFAE and SAAE are the phenomena in which adaptation to a sine-wave-shaped contour causes a shift in respectively the apparent shape-frequency and apparent shape-amplitude of a test contour in a direction away from that of the adapting stimulus. In the global motion condition the sinusoidal-shaped contours were made to drift within a fixed stimulus window in the direction of their axis of modulation. In the local motion condition the contour was constructed from a string of Gabors, and their carriers but not envelopes moved. We investigated selectivity to motion direction by using adaptor and test contours that moved either in the same or opposite directions. We found that in the global motion condition both the SFAE and SAAE showed selectivity to motion direction, and that for the same-motion-direction condition, both after-effects increased with shape temporal frequency. We then examined the effect of luminance spatial frequency and luminance temporal frequency on global motion direction selectivity. Luminance temporal frequency accounted for some of the increase in after-effect magnitude with shape temporal frequency, but shape temporal frequency also contributed. The local motion after-effects on the other hand were neither selective to motion direction nor increased with luminance temporal frequency. Taken together, the results are best understood by supposing that curvature is encoded by mechanisms that are selective to motion direction and that the directional selectivity best manifests itself psychophysically when there is sufficient spatio-temporal coverage of the stimulus to stimulate the full array of potentially responsive curvature-coding mechanisms.

Dynamic random dot targets consisting of many localized motion vectors have been used to study the pooling of local motion signals into a global motion percept (Williams and Sekuler, 1984). In such displays, the dots are displaced with a constant step size and the direction of motion for each dot is chosen at random from a specified distribution. When the distribution extends over 360 deg, the display consists only of local random motion of individual dots and no coherent motion is reported. However, when the distribution is less than 360 deg (biased), the stimulus appears to flow in a single direction. We examined the effects of reducing the number of directionally selective (DS) cortical neurons on this integration process. Normal cats and cats with severely reduced proportions of DS neurons were trained on 2 direction discrimination tasks. The discrimination of opposite directions was examined while varying either the range of directions of local motion, or the proportion of dots moving with biased distribution. When all dots in the display were directionally biased, cats with reduced numbers of DS neurons performed the task as well as normal cats and humans (threshold range: 280-320 deg). However, when the proportion of biased dots decreased, these animals had severe deficits. Thus, in the absence of noise, even a very small number of DS neurons can perform spatial pooling of local directional signals, and support normal discrimination of opposite directions. However, a full complement of directional detectors appears necessary when the motion signal is masked by noise. The discrimination of small differences in direction revealed far more severe deficits, even when all the dots in the display were directionally biased (no noise).(ABSTRACT TRUNCATED AT 250 WORDS)

This paper presents a fast image sequence stabilization system that uses one bit transform (IBT) matching based global motion estimation and Kalman filtering based global motion correction. In order to facilitate a fast motion estimation process, image frames are initially converted into a single bit-plane representation using the IBT, and then motion vectors are evaluated matching the IBTs of two consecutive frames using Boolean operations. Because a binary representation is established, the computational complexity of motion estimation is significantly reduced providing a proportionate reduction in power consumption of custom hardware. The global motion vector is computed using local motion vectors of sub-image blocks to further increase the motion estimation speed. For the motion correction part, global motion vectors are accumulated to yield absolute frame displacements and Kalman filtering is employed to achieve stabilization.

In this paper, to detect the moving objects between frames in compressed video and to obtain the best compression video and the noiseless video. We describe a video in which frames by classifying macroblocks (MB), and describe motion estimation (ME), motion vector field (MV) and motion compensation (MC). we propose to classify Macroblocks of each video frame into different classes and use this class information to describe the frame content based on the motion vector. MB class information video applications such as shot change detection, motion discontinuity detection, Outlier rejection for global motion estimation. To reduce the noise and to improve the clarity of the compressed video by using contrast limited adaptive histogram equalization (CLAHE) Algorithm.

The use of the human hand as a natural interface device serves as a motivating force for research in the modeling, analysis and capture of the motion of an articulated hand. Model-based hand motion capture can be formulated as a large nonlinear programming problem, but this approach is plagued by local minima. An alternative way is to use analysis-by-synthesis by searching a huge space, but the results are rough and the computation expensive. In this paper, articulated hand motion is decoupled, a new two-step iterative model-based algorithm is proposed to capture articulated human hand motion, and a proof of convergence of this iterative algorithm is also given. In our proposed work, the decoupled global hand motion and local finger motion are parameterized by the 3D hand pose and the state of the hand respectively. Hand pose determination is formulated as a least-median-of-squares (LMS) problem rather than the nonrobust least-squares (LS) problem, so that 3D hand pose can be reliably calculated even if there are outliers. Local finger motion is formulated as an inverse kinematics problem. A genetic algorithm-based method is proposed to find a sub-optimal solution of the inverse kinematics effectively. Our algorithm and the LS-based algorithm are compared in several experiments. Both algorithms converge when local finger motion between consecutive frames is small. When large finger motion is present, the LS-based method fails, but our algorithm can still estimate the global and local finger motion well.

The processing of motion changes throughout the visual hierarchy, from spatially restricted ‘local motion’ in early visual cortex to more complex large-field ‘global motion’ at later stages. Here we used functional magnetic resonance imaging (fMRI) to examine spatially selective responses in these areas related to the processing of random-dot stimuli defined by differences in motion. We used population receptive field (pRF) analyses to map retinotopic cortex using bar stimuli comprising coherently moving dots. In the first experiment, we used three separate background conditions: no background dots (dot-defined bar-only), dots moving coherently in the opposite direction to the bar (kinetic boundary) and dots moving incoherently in random directions (global motion). Clear retinotopic maps were obtained for the bar-only and kinetic-boundary conditions across visual areas V1–V3 and in higher dorsal areas. For the global-motion condition, retinotopic maps were much weaker in early areas and became clear only in higher areas, consistent with the emergence of global-motion processing throughout the visual hierarchy. However, in a second experiment we demonstrate that this pattern is not specific to motion-defined stimuli, with very similar results for a transparent-motion stimulus and a bar defined by a static low-level property (dot size) that should have driven responses particularly in V1. We further exclude explanations based on stimulus visibility by demonstrating that the observed differences in pRF properties do not follow the ability of observers to localise or attend to these bar elements. Rather, our findings indicate that dorsal extrastriate retinotopic maps may primarily be determined by the visibility of the neural responses to the bar relative to the background response (i.e. neural signal-to-noise ratios) and suggests that claims about stimulus selectivity from pRF experiments must be interpreted with caution.

Block-matching algorithm is the most common technique applied in block-based motion estimation technique. There are several block-matching algorithm based on block-based motion estimation techniques have been developed. Full search (FS), three step search (TSS), new three step search (NTSS), diamond search (DS) and hexagon based search (HS) are the most well known block-matching algorithm. These techniques are applied to video sequences to remove the temporal redundancy for compression purposes and to gauge the motion vector estimation. In addition, the mentioned block-matching algorithms are the baseline techniques that have been used to further develop all the enhanced or improved algorithms. In order to develop the proposed methods, the baseline techniques are studied to develop the proposed algorithms. This chapter proposes modelling of fast unrestricted hexagon search (FUHS16) and unrestricted hexagon-diamond search (UHDS16) algorithms for motion vector estimation, which is based on the theory and application of block-based motion estimation. Both of these algorithms are designed using 16 × 16 block size. In particular, the motion vector estimation, quality performance, computational complexity, and elapsed processing time are emphasised. These parameters have been used to measure the experimental results. It is the aim of this study that this work provides a common framework with which to evaluate and understand block-based matching motion estimation performance. On the theoretical side, four fundamental issues are explored: (1) division of frame, (2) basic block-based matching, (3) motion vector estimation, and (4) block-matching algorithm development. Various existing block-matching motion estimation algorithms have been analysed to develop the fundamental research. Based on the theoretical and fundamental research analysis the FUHS16 and UHDS16 algorithms using 16 × 16 block-based motion estimation formulations were developed. To improve the UHDS16 algorithm, 8 × 8 block-matching technique has been tested. The 8 × 8 block-matching technique is known as UHDS8. The results show positive improvements. From an application perspective, the UHDS8 algorithm efficiently captured the motion vectors in many video sequences. For example, in video compression, the use of motion vectors on individual macro-blocks optimized the motion vector information. The UHDS8 algorithm also offers improvement in terms of image quality performance, computational complexity and elapsed processing time. Thus, this chapter offers contributions in certain areas such as reducing the mechanism of computational complexity in estimating the motion from the video sequences. In particular, the FUHS16, UHDS16 and UHDS8 algorithms were developed to estimate the motion vectors field in the video sequences. Theoretical analysis block-based matching criteria are adapted to FUHS16, UHDS16 and UHDS8 algorithms, which are based on search points technique. Basically, the proposed of FUHS16, UHDS16 and UHDS8 algorithm produces the best motion vector estimation finding based on the block-based matching criteria. Besides that, the UHDS8 algorithm also improves the image quality performances and the search points in terms of the computational complexity. Overall, the study shows that the UHDS8 algorithm produces better results compared to the FUHS16 and UHDS16 algorithm.

Joint ground moving-target tracking identification is a crucial task in a modern combat operation. Due to the entirely different environment, ground moving-target tracking is quite different from airborne target tracking. A major difference lies in target modelling. In airborne target tracking, a target is usually treated as a point, while for ground target tracking, a target is considered a rigid body. Two approaches for 2D rigid-body target modelling are proposed. Equipped with ground moving-target indicator and high-resolution range sensors, the new approaches effectively explore the concepts of local and global motions of a rigid body. The kinematics of a global motion is described by a constant acceleration model, and a local motion is modelled by the pivoting centre and pseudocentre approaches. The proposed models are implemented by the extended Kalman filter with and without a probabilistic data association filter. The simulation results show that the proposed approaches not only correctly track a rigid-body target in a complicated scenario but also simultaneously report its structural information.

To remove the vibration in video content, a video stabilization approach is proposed in this paper. Video stabilization is completed by local rotational motion model in our approach. The local rotational motion is derived from rotation-based block matching method. The global rotational motion is identified by a local rotational motion model. Particularly, global rotational motion is derived by computing the variance of local rotational motion and its direction. Finally, the global rotational motion is smoothed by calculating the motion of neighboring frames. The experimental results demonstrate that the proposed video stabilization approach can produce well smooth videos with smooth rotational motion in various videos. The comparison also shows our superior performance on video stabilization.