Moving object detection algorithm for H.264/AVC compressed video stream

Current Moving Object Detectors in airborne Region of Interest (ROI) coding systems for police surveillance applications used on-board of UAVs are often based on Global Motion Estimation (GME) techniques. Since in these scenarios the camera is moving, simple background removal approaches cannot be applied without a Global Motion Compensation (GMC). Common GMC algorithms assume the ground to be planar, allowing the pixels of the previous frame to be motion compensated into the current frame by applying a projective transformation. The difference image between the compensated frame and the current frame emphasis regions containing possible motion. Such moving object detectors are great in terms of run-time efficiency but are known to lack in terms of accuracy - especially for unstructured regions of moving objects - as well as the robustness against noise. Superpixel segmentation was recently proposed to overcome the issue of the imprecise region cuts given by the difference image. It provides a greatly improved true positive detection rate, but unintentionally also increases the area of false positives. This paper proposes the use of a mesh-based GME and GMC to detect the moving object regions wherein a cluster filter eliminates errors in the optical flow by assuming a smooth vector field as the global motion model. In doing so we improve the coding efficiency of the fully automatic ROI coding system by more than 24% for moving object areas conserving the detection benefits of the integration of superpixel segmentation.

This paper presents a novel approach for moving object extraction in the H.264/AVC compressed domain, which based on Ant Colony clustering Algorithm (ACA) and threshold method in macro block layer. Firstly, the Motion Vector (MV) field and the macro block types are extracted from the H.264/AVC compressed video, and then merge MVs with the same characteristic. Secondly, an improved ACA is used to classify the MV field into different motion homogenous regions. At the same time, use macro block types to determine the location of objects. Finally, using the complementarities of macro block template and MVs clustering template to obtain final objects. Experimental results for several video sequences demonstrate that in the case of ensuring accuracy, the proposed approach can extract moving object faster.

Abstract. This paper presents a comprehensive method for the determination of glacier surface motion vector fields at high spatial and temporal resolution. These vector fields can be derived from monocular terrestrial camera image sequences and are a valuable data source for glaciological analysis of the motion behaviour of glaciers. The measurement concepts for the acquisition of image sequences are presented, and an automated monoscopic image sequence processing chain is developed. Motion vector fields can be derived with high precision by applying automatic subpixel-accuracy image matching techniques on grey value patterns in the image sequences. Well-established matching techniques have been adapted to the special characteristics of the glacier data in order to achieve high reliability in automatic image sequence processing, including the handling of moving shadows as well as motion effects induced by small instabilities in the camera set-up. Suitable geo-referencing techniques were developed to transform image measurements into a reference coordinate system.The result of monoscopic image sequence analysis is a dense raster of glacier surface point trajectories for each image sequence. Each translation vector component in these trajectories can be determined with an accuracy of a few centimetres for points at a distance of several kilometres from the camera. Extensive practical validation experiments have shown that motion vector and trajectory fields derived from monocular image sequences can be used for the determination of high-resolution velocity fields of glaciers, including the analysis of tidal effects on glacier movement, the investigation of a glacier's motion behaviour during calving events, the determination of the position and migration of the grounding line and the detection of subglacial channels during glacier lake outburst floods.

In this paper, a novel camera motion descriptor is proposed for video shot classification. In the proposed method, raw motion information of consecutive video frames are extracted by computing the motion vector of each macroblock to form motion vector fields (MVFs). Next, a motion consistency analysis is applied on MVFs to eliminate the inconsistent motion vectors. Then, MVFs are divided into nine (3 × 3) local regions and the singular value decomposition (SVD) technique is applied on the motion vectors extracted from each local region in the temporal direction. Consistent motion vectors of a number of MVFs are compactly represented at a time to characterize temporal camera motion. Accordingly, each local region of the whole video shot is represented using a sequence of compactly represented vectors. Finally, the sequence of vectors is converted into a histogram to describe the camera motions of each local region. Combination of all the local histograms is considered as the camera motion descriptor of a video shot. The shot descriptors are used in a classifier to classify video shots. In this work, we use support vector machine (SVM) for performing classification tasks. The experimental results show that the proposed camera motion descriptor has strong discriminative capability to classify different camera motion patterns in professionally captured video shots effectively. We also show that our proposed approach outperforms two state-of-the-art video shot classification methods.

Modern video coding standards such as High Efficiency Video Coding (HEVC) and H.264/MPEG‐4 advanced video coding (AVC) supersede the previous coding standards because of their improved coding efficiency. These standards adopt variable block sizes in frame coding ranging from 4 × 4 to 16 × 16 and 4 × 4 to 64 × 64 for H.264/AVC and HEVC, respectively. The use of variable block sizes for inter prediction provides a significant coding gain compared to coding a macroblock (MB) using regular block size. However, this new feature greatly increases the computational complexity of the encoder when brute‐force rate distortion optimization (RDO) algorithm is used for coding parameter selection. This paper proposes an efficient inter prediction mode selection scheme based on motion homogeneity and residual complexity measures of an MB to speed up the encoding process. The motion homogeneity is assessed through the normalized motion vector (MV) field, and residual complexity is evaluated by the sum of absolute difference (SAD). To acquire the MVs and SADs, motion estimation at 8 × 8 block size is performed using a lightweight recursive motion estimator in which the vector field tends toward true object motion. Based on motion homogeneity and residual complexity of an MB, only a small number of inter prediction modes are selected for the RDO process. The experimental results for H.264/AVC show that the proposed scheme reduces the encoding time by 64% on average without any significant degradation of coding efficiency. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Most standard video compression schemes such as H.264/AVC involve a high complexity encoder with block motion estimation (ME) engine. However, applications such as video reconnaissance and surveillance using unmanned aerial vehicles (UAVs) require a low complexity video encoder. Additionally, in such applications, the motion in the video is primarily global and due to the known movement of the camera platform. Therefore in this work, we propose and investigate a low complexity encoder with global motion based frame prediction and no block ME. We show that for videos with mostly global motion, this encoder performs better than a baseline H.264 encoder with ME block size restricted to 8×8. Furthermore, the quality degradation of this encoder with decreasing bit rate is more gradual than that of the baseline H.264 encoder since it does not need to allocate bits across motion vectors (MVs) and residue data. We also incorporate a spectral entropy based coefficient selection and quantizer design scheme that entails latency and demonstrate that it helps achieve more consistent frame quality across the video sequence.

Tracking a face for knowledge-based coding of videophone sequences

This paper proposes a filtering approach based on global motion estimation (GME) and global motion compensation (GMC) as pre-processing and post-processing for video CODEC. For the pre-processing of video CODEC, group-of-pictures (GOP), i.e., basic unit for GMC and reference frames are first defined for an input video sequence. Next, GME and GMC are sequentially performed for every frame in each GOP. Finally, block-based adaptive temporal filter is applied between global motion compensated frames prior to video encoding. For the post-processing at the end of decoder, every decoded frame is inversely motion-compensated by using the transmitted global motion information. The holes which are generated during inverse motion compensation can be filled using the reference frames. Experimental results the proposed algorithm provides higher BD-PSNRs than before-use on H.264 and HEVC platforms.

In Versatile Video Coding (VVC), local affine motion compensation (LAMC) is adopted to handle complex motions, such as rotation and zooming. However, it is inefficient to use LAMC to handle the global motion due to the following two reasons. First, the use of LAMC may lead to some extra bit cost on the affine motion model parameters. Second, the precision of LAMC is restricted by the MV precision of the control points. Therefore, in this paper, we propose a global homography motion compensation (GHMC) framework to better characterize the global motion. For each coding block, an extra mode is added to perform motion compensation based on an 8-parameter global homography motion model. In addition, an extrapolation scheme is designed to derive the parameters from reference frames to save the bit cost for signaling them. The proposed framework is implemented into the VVC reference software VTM-6.0. Experimental results show that, on average, 0.69% and 0.66% BD-rate reduction is achieved under Low Delay P and Low Delay B configurations, respectively, for sequences with rich complex global motions.

We present a novel multi-resolution block matching algorithm (BMA) for fast motion estimation. At the coarsest level, a full search BMA (FSBMA) is performed for searching complex or random motion. Concurrently, spatial correlation of motion vector (MV) field is used for searching continuous motion. Here we present an efficient method for searching full resolution MVs without MV decimation even at the coarsest level. After the coarsest level search, two or three initial MV candidates are chosen for the next level. At the further levels, the MV candidates are refined within much smaller search areas. Simulation results show that in comparison with FSBMA, the proposed BMA achieves a speed-up factor over 710 with minor PSNR degradation of 0.2 dB at most, under a normal MPEG-2 coding environment. Furthermore, our scheme is also suitable for hardware implementation due to regular data-flow.

A novel motion representation and estimation scheme is presented that leads to improved coding efficiency of block-based video coders like, e.g., H.263. The proposed scheme is based on motion-compensated prediction from more than one reference frame. The reference pictures are warped or globally motion-compensated versions of the previous frame. Affine motion models are used as warping parameters approximating the motion vector field. Motion compensation is performed using standard block matching in the multiple reference frames buffer. The frame reference and the affine motion parameters are transmitted as side information. The approach is incorporated into an H.263-based video codec at minor syntax changes and embedded into a rate-constrained motion estimation and macroblock mode decision frame work. In contrast to conventional global motion compensation, where one motion model is transmitted, we show that multiple global motion models are of benefit in terms of coding efficiency. Significant coding gains in comparison to TMN-10, the test model of H.263+, are achieved that provide bit-rate savings between 20% and 35% for the various sequences tested

A dense estimation of optical flow field within the MPEG-2 compressed domain is proposed, which utilizes only the compressed-domain information, i.e. motion vectors and DCT coefficients. First, motion vectors are pre-processed to estimate the DCT coefficients for P and B frames. Second, initial optical flow is estimated with Black's optical flow estimation framework, in which DC image is substituted by DC+2AC image to provide more intensity information. Third, high confidence test is exploited to generate the dense and accurate motion vector field by removing false and noisy motion vectors. It preserves the advantages of compressed domain processing and improves the existent MPEG velocity field in terms of accuracy and density. Experimental results demonstrate that the proposed approach can provide a satisfactory motion analysis for compressed-domain video object extraction.

Motion vector outlier removal using dissimilarity measure

H.264/AVC 비디오 압축 표준은 압축 효율을 높이기 위해 다양한 크기의 블록을 사용하여 화면 사이의 움직임 예측을 수행한다. H.264/AVC는 가변적인 블록 크기의 움직임 보상을 통해 세밀한 영역의 움직임까지 예측할 수 있어 잔여 영상을 나타내는 정보량을 효과적으로 줄일 수 있다. 복호를 위해서는 각 블록의 움직임 벡터를 전송해야 하는데, 저비트율 환경에서는 움직임 벡터 정보가 전체 비트스트림의 약 40%를 차지한다. 움직임 벡터 정보량을 줄이기 위해 비디오 부호화 전문가 그룹(VCEG)에서는 다양한 움직임 벡터 예측(Motion Vector Competition) 방법을 제안하였다. 다양한 예측 움직임 벡터를 사용하여 실제 전송해야 할 움직임 벡터 차분값(Motion Vector Difference, MVD)의 크기를 줄이기 때문에 압축 효율을 높일 수 있다. 그러나 다양한 예측 움직임 벡터를 사용하기 때문에 선택된 예측 움직임 벡터의 인덱스 정보를 복호기로 전송해야 한다. 이 논문에서는 인덱스 정보를 효율적으로 전송하기 위해 Phased-in 코드를 기반으로 한 새로운 코드워드 표를 제안했다. 실험을 통해 제안한 방법을 이용하여 동일한 화질에서 평균 약 7.24%의 비트율을 절감할 수 있었고, 동일한 비트율에서는 평균 약 0.36dB의 화질을 향상시킬 수 있었다. The H.264/AVC video coding standard performs inter prediction using variable block sizes to improve coding efficiency. Since we predict not only the motion of homogeneous regions but also the motion of non-homogeneous regions accurately using variable block sizes, we can reduce residual information effectively. However, each motion vector should be transmitted to the decoder. In low bit rate environments, motion vector information takes approximately 40% of the total bitstream. Thus, motion vector competition was proposed to reduce the amount of motion vector information. Since the size of the motion vector difference is reduced by motion vector competition, it requires only a small number of bits for motion vector information. However, we need to send the corresponding index of the best motion vector predictor for decoding. In this paper, we propose a new codeword table based on the phased-in code to encode the index of motion vector predictor efficiently. Experimental results show that the proposed algorithm reduces the average bit rate by 7.24% for similar PSNR values, and it improves the average image quality by 0.36dB at similar bit rates.

Frame rate up-conversion is a useful post processing technique to enhance the visual quality of low frame rate video materials. In this paper, we propose a novel frame rate up-conversion algorithm utilizing true motion estimation and adaptive motion vector refinement. First, block-matching algorithm (BMA) based on an efficient motion similarity constrained criterion is employed to track the true motion between adjacent frames and produce a preliminary intermediate motion vector field (MVF).We then classify the intermediate motion vectors into two categories, isolated group and connected group adaptively by motion clustering. A smooth resampling method is later applied to connected group to obtain consistent motion, while isolated motion vectors are re-estimated with bilateral edge-weighted block decomposition (BEWBD). Finally, overlapped block motion compensation (OBMC) is adopted to interpolate the intermediate frame. Extensive experimental results show that the proposed algorithm outperforms conventional methods both subjectively and objectively.