A camera motion histogram descriptor for video shot classification

In this paper, we present a novel approach for tennis video analysis, which can automatically classify video shots into 5 classes based on MPEG motion vectors and other features. Two types of features have been used: domain-independent features, such as the local motion activity and the persistent camera pan motion, and domain-dependent, such as the motion activity ratio in the court model. Combining these low-level features with domain knowledge of the tennis game, we can categorize the tennis video shots into five classes, which cover majority of the live tennis video shots, and derive semantic annotation for all shot classes. The results can be used in the higher-level video analysis, including structure analysis, table of content extraction for sports video, video summary and personalization. The proposed approach can easily be extended to analyzing other sports.KeywordsMotion VectorLocal MotionCamera MotionSemantic AnnotationVideo ShotThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We propose a technique for video shot segmentation and classification based on singular value decomposition (SVD). For the input video sequence, we create a feature-frame matrix A, and perform the SVD on it. From this SVD, we are able to not only derive the refined feature space to better segment the video sequence along the time axis, but also define metrics to enable classifications of the detected video shots. Using these SVD properties, we achieve the two goals of accurate video shot segmentation, and visual content-based shot classification at the same time.

Abstract. This paper presents a comprehensive method for the determination of glacier surface motion vector fields at high spatial and temporal resolution. These vector fields can be derived from monocular terrestrial camera image sequences and are a valuable data source for glaciological analysis of the motion behaviour of glaciers. The measurement concepts for the acquisition of image sequences are presented, and an automated monoscopic image sequence processing chain is developed. Motion vector fields can be derived with high precision by applying automatic subpixel-accuracy image matching techniques on grey value patterns in the image sequences. Well-established matching techniques have been adapted to the special characteristics of the glacier data in order to achieve high reliability in automatic image sequence processing, including the handling of moving shadows as well as motion effects induced by small instabilities in the camera set-up. Suitable geo-referencing techniques were developed to transform image measurements into a reference coordinate system.The result of monoscopic image sequence analysis is a dense raster of glacier surface point trajectories for each image sequence. Each translation vector component in these trajectories can be determined with an accuracy of a few centimetres for points at a distance of several kilometres from the camera. Extensive practical validation experiments have shown that motion vector and trajectory fields derived from monocular image sequences can be used for the determination of high-resolution velocity fields of glaciers, including the analysis of tidal effects on glacier movement, the investigation of a glacier's motion behaviour during calving events, the determination of the position and migration of the grounding line and the detection of subglacial channels during glacier lake outburst floods.

This paper presents a novel approach for moving object extraction in the H.264/AVC compressed domain, which based on Ant Colony clustering Algorithm (ACA) and threshold method in macro block layer. Firstly, the Motion Vector (MV) field and the macro block types are extracted from the H.264/AVC compressed video, and then merge MVs with the same characteristic. Secondly, an improved ACA is used to classify the MV field into different motion homogenous regions. At the same time, use macro block types to determine the location of objects. Finally, using the complementarities of macro block template and MVs clustering template to obtain final objects. Experimental results for several video sequences demonstrate that in the case of ensuring accuracy, the proposed approach can extract moving object faster.

Semi-supervised pipe video temporal defect interval localization

In this paper, an interleaved motion vector coding is proposed which supports flexible coding order of motion vector components according to directional property of vertical and horizontal components in motion vector field. It requires no additional information for adaptively selecting the coding order.

We present a novel multi-resolution block matching algorithm (BMA) for fast motion estimation. At the coarsest level, a full search BMA (FSBMA) is performed for searching complex or random motion. Concurrently, spatial correlation of motion vector (MV) field is used for searching continuous motion. Here we present an efficient method for searching full resolution MVs without MV decimation even at the coarsest level. After the coarsest level search, two or three initial MV candidates are chosen for the next level. At the further levels, the MV candidates are refined within much smaller search areas. Simulation results show that in comparison with FSBMA, the proposed BMA achieves a speed-up factor over 710 with minor PSNR degradation of 0.2 dB at most, under a normal MPEG-2 coding environment. Furthermore, our scheme is also suitable for hardware implementation due to regular data-flow.

Frame rate up-conversion is a useful post processing technique to enhance the visual quality of low frame rate video materials. In this paper, we propose a novel frame rate up-conversion algorithm utilizing true motion estimation and adaptive motion vector refinement. First, block-matching algorithm (BMA) based on an efficient motion similarity constrained criterion is employed to track the true motion between adjacent frames and produce a preliminary intermediate motion vector field (MVF).We then classify the intermediate motion vectors into two categories, isolated group and connected group adaptively by motion clustering. A smooth resampling method is later applied to connected group to obtain consistent motion, while isolated motion vectors are re-estimated with bilateral edge-weighted block decomposition (BEWBD). Finally, overlapped block motion compensation (OBMC) is adopted to interpolate the intermediate frame. Extensive experimental results show that the proposed algorithm outperforms conventional methods both subjectively and objectively.

Moving object segmentation directly from compressed video stream is a challenging task in video signal processing community. A novel moving object detection algorithm for H.264/AVC stream is presented in this paper. Firstly, a series of operations, including spatial-temporal normalization, bidirectional accumulation, median filtering and thresholding magnitudes, are conducted on the raw motion vector (MV) field so as to gain a robust MV field, which can indicate the moving region reliably. Then global motion compensation in compressed domain is implemented if there exist global motion caused by camera. Finally, the macroblock coded mode with variable size is combined with the temporal information to locate the foreground object. Its effectiveness of the proposed algorithm has been demonstrated in our experimental results. The main advantage of the algorithm lies in its robustness and real-time performance.

Until today, several software-based approaches to increase the temporal resolution in cardiac computed tomography by estimating motion vector fields (MVFs) have been developed. Thereunder, the majority are motion compensation algorithms, which estimate the MVFs employing a three-dimensional registration routine working on reconstructions of multiple cardiac phases.^{2, 6, 7, 12}We present an algorithm that requires nothing more than the data needed for a short scan reconstruction for motion estimation and motion-compensated reconstruction, which both are based on the reconstruction of volumes from a limited angular range.^{2, 3, 7, 8} Those partial angle reconstructions are centered at different time points during the short scan and have a temporal resolution of about 10ms each. The MVFs are estimated by a constrained cost function optimization routine employing a motion artifact measuring cost function. During optimization, the MVFs are applied directly by warping the partial angle reconstructions, and the motion compensation is established by simply adding the shifted images. In order to enforce smooth vector fields and keep the number of parameters low, the motion is modeled by a low degree polynomial. Furthermore, to find a good estimation of the MVFs even in phases with rapid cardiac motion, the constrained optimization is re-initialized multiple times. The algorithm is validated with the help of a simulation study and applied to patient data, where motion- compensated reconstructions are performed in various cardiac phases. We show that the image quality can be improved, also in more rapid cardiac phases due to re-initialization of the optimization routine.

Fast Deformable Image Registration for Real-Time Target Tracking During Radiation Therapy Using Cine MRI and Deep Learning

Efficient Scene Change Detection and Camera Motion Annotation for Video Classification

Motion characterization plays a critical role in video indexing. An effective way of characterizing camera motion facilitates the video representation, indexing and retrieval tasks. This paper describes a novel nonparametric motion representation to achieve an effective and robust recognition of parts of the video in which camera is static, or panning, or tilting, or zooming, etc. This representation employs the mean shift filtering and the vector histograms to produce a compact description of a motion field. The basic idea is to perform spatio-temporal mode-seeking in the motion feature space and use the histograms-based spatial distributions of dominant motion modes to represent a motion field. Unlike most existing approaches, which focus on the estimation of a parametric motion model from a dense optical flow field (OFF) or a block matching-based motion vector field (MVF), the proposed method combines the motion representation and machine learning techniques (e.g., support vector machines) to perform camera motion analysis from the classification point of view. The main motivation lies in the impossibility of uniformly securing a proper parametric assumption in a wide range of video scenarios. The diverse camera shot sizes and frequent occurrences of bad OFF/MVF necessitates a learning mechanism, which can not only capture the domain-independent parametric constraints, but also acquire the domain-dependent knowledge to tolerate the influence of bad OFF/MVF. In order to improve performance, we can use this learning-based method to train enhanced classifiers aiming at a certain context (i.e., shot size, neighbor OFF/MVFs, and video genre). Other visual cues (e.g., dominant color) can also be incorporated for further motion analysis. Our main aim is to use a generic feature space analysis method to explore a flexible OFF/MVF representation in a nonparametric technique, which could be fed into a learning framework to robustly capture the global motion by incorporating the context information. Results on videos with various types of content (23 191 MVFs culled from MPEG-7 dataset, and 20 000 MVFs culled from broadcast tennis, soccer, and basketball videos) are reported to validate the proposed approach.

In the state-of-art video coding standard HEVC, temporal motion vector (MV) predictor is adopted in order to improve coding efficiency. However, motion vector information in reference frames, which is used by temporal MV predictor, takes significant amount of bits in memory storage. Therefore motion data needs to be compressed before storing into buffer. In this paper we propose an adaptive motion data storage reduction method. First, it divides the current 16x16 block in the reference frame into four partitions. One MV is sampled from each partition and all sampled MVs form a MV candidate set. Then it judges if one or two MVs should be stored into the MV buffer by checking the maximum distance between any two of the MVs in the candidate set. If the maximum distance is greater than a certain threshold, the motion data of the two MVs that have maximum distance are put into memory; otherwise the motion data of the upper left block is stored. The basic goal of the proposed method is to improve the accuracy of temporal MV predictor at the same time reducing motion data memory size. Simulation results show that compared to the original HEVC MV memory compression method in the 4th JCT-VC meeting, the proposed scheme achieves a coding gain of 0.5%~0.6%; and the memory size is reduced by more than 87.5% comparing to without using motion data compression.

This paper presents a three-dimensional method to reconstruct moving objects from cone-beam X-ray projections using an iterative reconstruction algorithm and a given motion vector field. For the image representation, adapted blobs are used, which can be implemented efficiently as basis functions. Iterative reconstruction requires the calculation of line integrals (forward projections) through the image volume, which are compared with the actual measurements to update the image volume. In the existence of a divergent motion vector field, a change in the volumes of the blobs has to be taken into account in the forward and backprojections. An efficient method to calculate the line integral through the adapted blobs is proposed. It solves the problem, how to compensate for the divergence in the motion vector field on a grid of basis functions. The method is evaluated on two phantoms, which are subject to three different known motions. Moreover, a motion-compensated filtered back-projection reconstruction method is used, and the reconstructed images are compared. Using the correct motion vector field with the iterative motion-compensated reconstruction, sharp images are obtained, with a quality that is significantly better than gated reconstructions.