A motion compensation technique using sliced blocks and its application to hybrid video coding

High-efficiency video coding is the most recent video compression standard. HEVC is designed to decrease the bit rate of video transmission without affecting the video quality. The intra prediction of HEVC features about 35 directional modes. The planar mode and DC modes are included into it. The decision about the most appropriate intra mode within the coding unit of high-efficiency video encoder is a vital component in video coding. The intra mode decision has been a crucial, computationally complex processing step and has a share of 85% in the overall video coding complexity. The optimal mode is selected by rough mode decision (RMD) process from all 35 modes and final decision of partitioning is taken through the rate distortion optimization (RDO) process. The brute force RD cost calculation process consumes a large portion of HEVC encoding complexity. This paper presents analysis of the spread of 35 directional modes over the video frame and the correlation between the homogeneous or non-homogeneous characteristics of video content and the spread of directional modes over the video frame. The proposed method is based on the sum of average gradient evaluated for each of the 35 directional modes which help to reduce the number of candidate modes for rough mode decision and RD cost calculation. The performance of the proposed algorithm is evaluated on three distinct classes of video sequences. The offline classification accuracy of the proposed scheme is measured to be 90%. The exhaustive analysis of mode decision carried out in the proposed method will be subsequently useful for training machine learning algorithm for early decision about coding unit depth and fast prediction of the appropriate intra mode. The early depth decision and reduction in the number of candidate coding units to be passed through iterative RD cost computation will drastically reduce the computation complexity and increase the encoding time of high-efficiency video encoder.

Rate-distortion optimized motion compensation was originally proposed to improve the source coding efficiency of low bit-rate video coding. This paper investigates the potential benefits of extensions to explicitly handle transmission over lossy networks. The important impact of appropriate motion compensation on error resilience is identified and exploited to enhance performance. The expected end-to-end distortion, rather than the source coding distortion, is employed in the rate-distortion optimization criterion. The accuracy of the distortion estimate is crucial when optimizing over the motion vectors (as compared to the less sensitive inter/intra mode decision), and is ensured here by adopting the recursive optimal per-pixel estimate (ROPE) method. The main objective is to assess the gains achievable by such source-channel motion compensation, and this is given priority over complexity considerations. Significant performance gains in simulation demonstrate the potential of rate-distortion optimized motion compensation to improve the error resilience of video coding.

Rate-distortion optimization (RDO) is widely employed to maximize coding efficiency in hybrid video coding. Due to an extensive use of spatial-temporal predictions in video coding, a global RDO problem becomes so complex that the processing of each coding unit (e.g. a macro block) is dependent and entangled each other. Usually the RDO is simply performed for each coding unit individually and independently, thus compromising the RDO performance significantly. In this paper, we examine temporal dependent RDO by developing a novel source distortion temporal propagation (SDTP) model, which takes into account the influence of a current coding macro block to future macro blocks in its temporal propagation chain. Accordingly estimations of the influence and the corresponding Lagrange multiplier for the proposed SDTP-based RDO are obtained. Experimental results show consistently a significant coding gain of the proposed scheme over H.264/AVC.

Like H.264, AVS video coding standard also uses macroblock (MB) based motion compensation (MC) and mode decision (MD). Rate distortion optimization (RDO) is the best known mode decision method, but with a high computational complexity that limits its applications. In our paper, firstly an MD algorithm based on RDO is given, which makes more mode candidates enter into RDO mode decision with little hardware resource increment. We further analyze the pipeline structure in detail, and implement a block-level 5-stage hardware pipeline. It can support the real time RDO mode decision processing of 1080P@30fps, and the coding efficiency is about 0.5db higher than the traditional SAD method. Our design is described in high-level Verilog/VHDL hardware description language and implemented under SMIC 0.18-µm CMOS technology with 215K logic gates and 80 KB SRAMs.

Rate-distortion optimization (RDO) plays an essential role in substantially enhancing the coding efficiency. Currently, rate-distortion optimized mode decision is widely used in scalable video coding (SVC). Among all the possible coding modes, it aims to select the one which has the best trade-off between bitrate and compression distortion. Specifically, this tradeoff is tuned through the choice of the Lagrange multiplier. Despite the prevalence of conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to 3-D wavelet-based SVC where the explicit values of the quantization step are not available, with on consideration of the content features of input signal. In this paper, an efficient content adaptive Lagrange multiplier selection algorithm is proposed in the context of RDO for 3-D wavelet-based SVC targeting quality scalability. Our contributions are two-fold. First, we introduce a novel weighting method, which takes account of the mutual information, gradient per pixel, and texture homogeneity to measure the temporal subband characteristics after applying the motion-compensated temporal filtering (MCTF) technique. Second, based on the proposed subband weighting factor model, we derive the optimal Lagrange multiplier. Experimental results demonstrate that the proposed algorithm enables more satisfactory video quality with negligible additional computational complexity.

The motion estimation, which is combined with the motion compensation in the decoding stage, is a fundamental method to achive high compression efficiency in video coding. The method achieves high compression efficiency by reducing temporal redundancy between adjacent pictures. Temporarily adjacent residual pictures also contain some redundancy even though the motion estimation technique removes some of the redundant information. Based on this characteristic, this paper proposes a multilevel residual MC (MRMC) algorithm, which repeats ME and MC on the residual signals in order to incorporate the possibility of further reducing the redundancy in the residual information. This paper mainly deals with the trade-off between the number of bits for residual signals and their corresponding motion vectors (MVs) based on the rate-distortion optimization. In addition, a fast method is presented for the practical use of the proposed MRMC method. The proposed method is implemented on the High Efficiency Video Coding software test model version 16.0. Experimental results show that an average Bjontegaard-delta rate savings of 9.16% can be achieved at a cost of increase of 17.0% in encoding time and 2.3% in decoding time.

The purpose of this study is to investigate the dosimetric accuracy of prostate SBRT when motion is considered. To account for target movement, motion compensation and gating techniques were investigated with PTV margins reduced to 2 mm. To allow for dosimetric measurements a Delta4 phantom, Gafchromic film, and Hexamotion motion platform were utilized. Four motion files were utilized that represent a range of motions. Analysis of measured prostate motions for fifteen patients was performed to ensure detected motions were similar to those previously reported and motion files utilized were suitable. Five patient plans were utilized to allow for the effects of MLC and target motion interplay to be investigated. For both motion compensation and gating techniques, plans were delivered to the stationary phantom and for each of four motion types with/without compensation/gating enabled. Using a 3%, 2 mm and 80% threshold gamma criteria, film measurements had an average pass rate of 80.5% for uncorrected deliveries versus 96.0% for motion compensated deliveries. For gated techniques average pass rates increased from 89.9% for uncorrected to 94.8% with gating enabled. Measurements with the Delta4 arrays were analyzed with a 3%, 2 mm and 10% threshold dose. An average pass rate of 83.8% was measured for uncorrected motions versus 94.8% with motion compensation. For the gated technique an average pass rate of 87.2% was found for uncorrected motions versus 96.9% with gating enabled. These results show that very high gamma pass rates are achievable when motion compensation or gating techniques are applied. When target motion is not accounted for shifts up to 5 mm in planned versus delivered isodose distributions were found. However, when motion compensation, or gated techniques were applied, much smaller differences between planned and delivered isodose distributions were found. With these techniques dose delivery accuracy is greatly improved, allowing for PTV margins to be reduced.

Since MPEG and VCEG jointly standardized the H.265/HEVC (High-Efficiency Video Coding) international standard in 2013, they have just completed H.266/VVC (Versatile Video Coding) [1] as the Final Draft International Standard (FDIS) in July 2020 through the Joint Video Experts Team (JVET) of ISO / IEC JTC1 / SC29 / WG11 MPEG and ITU-T SG16 / Q6 VCEG. VVC supports up to 87 intra coding modes including 65 general directional modes and 20 wide angular modes, which is increased more than twice compared to HEVC. VVC can accommodate not only more detailed intra prediction directions, but also so called wide angular modes which make its special sense in non-square coding blocks. The more detailed intra predictions naturally demand more computation in order to determine the most effective intra-prediction mode. In this sense, we investigate how to pre-prune the prediction candidate lists for a fast ISP intra mode decision and propose searching only a small number of prediction modes based on the shape of a block in the rate-distortion optimization (RDO)–based intra mode decision. The proposed method is verified to decrease the intra prediction processing time with only a little increase in the bit rate and a negligible reduction in PSNR values.

Fast intra mode decision algorithms in video coding are very useful and necessary for practical real-time encoder implementation. It reduces the computational complexity by eliminating unlikely modes from candidate set. However, most of these algorithms may lead to the noticeable quality degradation, especially for the all intra coding scenario. To overcome the quality loss while maintaining the fast encoding speed, we propose a method that estimates the reliability of the fast algorithms based on the edge magnitude histogram values of different modes. It could switch between the original fast algorithm and an enhanced mode decision refinement according to the reliability measurement. Extensive simulation results demonstrate that, with some sacrifice of the encoding speed, the proposed method could greatly improve the video quality by reducing the Bjontegaard Delta Rate (BD-Rate) loss from 9.24% to 4.49% on average, in comparison to the most cited fast intra mode decision algorithm in the H.264/AVC [1], both against the brute force full rate-distortion optimization (RDO) search based solution.

Motion correction in PET has become more important as system resolution has improved. The purpose of this study was to evaluate the accuracy of three motion compensation methods, event-by-event motion compensation with list-mode reconstruction (MOLAR), frame-based motion correction, and post-reconstruction image registration. Motion compensated image reconstructions were carried out with simulated HRRT data, using a range of motion information based on human motion data. ROI analyses in high contrast regions were performed to evaluate the accuracy of all the motion compensation methods, with particular attention to within-frame motion.Our study showed that MOLAR with list-mode based motion correction using accurate motion data can reliably correct for all reasonable head motions. Over all motions, the average ROI count was within 0.1±4.2% and 0.7±0.9% of the reference, no-motion value for two different ROIs. The location of the ROI centroid was found to be within 0.7±0.3mm of that of the reference image for the raphe nucleus. Frame-based motion compensation and post-reconstruction image registration were able to correct for small (<5mm), but the ROI intensity begins to deteriorate for medium motions (5-10mm), especially for small brain structures such as the raphe nucleus. For large (>10mm) motions, the average centroid locations of the raphe nucleus ROI had an offset error of 1.5±1.8mm and 1.8±1.8mm for each of the frame-based methods. For each frame-based method, the decrease in the average ROI intensity was 16.9±4.3% and 20.2±9.9% respectively for the raphe nucleus, and was 5.5±2.2% and 7.4±0.2% for putamen. Based on these data, we conclude that event-by-event based motion correction works accurately for all reasonable motions, whereas frame-based motion correction is accurate only when the within-frame motion is less than 10mm.

CyberKnife, a stereotactic radiosurgery (SRS) technique, delivers highly precise radiation beams to tumors. However, the patient motion, such as respiration and positional changes, can lead to tumor displacement, affecting the treatment efficacy. Fiducial marker tracking plays a crucial role in CyberKnife therapy, enabling real-time tumor position monitoring and ensuring accurate radiation delivery. This paper reviews motion compensation methods based on fiducial marker tracking, explores how fiducial markers facilitate precise tumor localization (tracking precision: 0.2–0.5 mm), and analyzes various motion compensation techniques, including respiratory motion compensation and positional adjustment. By dynamically adjusting the treatment device and radiation dose, fiducial marker tracking and motion compensation significantly enhance the precision of CyberKnife therapy, reduce errors caused by patient motion, and improve treatment outcomes.

We propose a novel video coding scheme to improve the performance of established block-based motion compensation codecs such as MPEG, H261, and H263. The proposed method is a hybrid scheme which introduces model-based global motion compensation as a pre-processing step to the basic block-based motion compensation technique. Performance evaluation tests show that the new method is capable of achieving higher compression rates with the addition of a very small overhead due to global motion estimation. In this paper we describe a codec based on the hybrid motion compensation technique in the context of H263, and present experimental results, comparing the performance of different codecs using rate-distortion curves for several test sequences.

A rate-distortion model for motion prediction efficiency in scalable wavelet video coding is proposed in this paper. The Lagrangian multiplier is widely used to solve the rate-distortion optimization problems in video coding, especially on mode decision and rate-constrained motion estimation. Different from the non-scalable video coding, the scalable wavelet video coding needs to operate under multiple bitrate conditions and it has an open-loop structure. Therefore, the conventional rate-distortion optimization technique is not suitable for the scalable wavelet case. By analyzing the rate-distortion trade-off due to different bits allocated to motion information, we propose a motion prediction gain (MPG) metric to measure motion coding efficiency. Based on the MPG metric, a new cost function for mode decision is thus proposed. Compared with the conventional Lagrangian multiplier optimization method, our experiments show that the new mode decision procedure can generally improve the PSNR performance for, particularly, the combined SNR and temporal scalability.

Although high efficiency video coding (HEVC) has achieved high coding performance, it has been shown that dependent rate-distortion optimization (RDO) can still further improve the coding performance at the encoder side. Inspired by our previous work on temporal-dependent RDO, this paper proposes a rate control method at coding tree unit (CTU) level to obtain both higher rate-distortion (R-D) performance and lower bitrate errors. We first formulate the global optimization problem in hybrid video coding and present a general solution framework based on a weighted Lagrange multiplier, where the weighted coefficient is related to the strength of temporal dependency among coding units, and a propagation factor is introduced for quantitative measurement of the temporal dependency. Second, an estimation method of CTU level R-D model is proposed to characterize R-D relationship more accurately. Finally, under the constraint of frame level bit budget, a formulation combining inter-block dependency and R-D characteristic is developed for optimal bit allocation at the CTU level. In the process of encoding a frame, a relaxed constraint algorithm (RCA) and a strict constraint algorithm (SCA) are employed to reduce the difference between numbers of actual and target bits for the current frame, respectively. Experimental results demonstrate that, on the HEVC test model (HM-16.7), our methods including RCA and SCA achieve considerable R-D performance improvement with lower bitrate errors, outperforming the state-of-the-art schemes. It is worth mentioning that even compared with HEVC without rate control, the RCA can still achieve bitrate savings of 5.4% and 5.1% on average under configurations of low delay P and B frames, respectively.

A four field variable block size motion compensated adaptive de-interlacing method is proposed to improve the accuracy of the motion vectors and lower the occlusions of motion compensated de-interlacing. The proposed de-interlacing method consists of variable block size motion estimation/compensation with four field SAD, interlaced block mode decision, and new block modes. The variable block size motion estimation and compensation improve the accuracy of the motion vectors, especially for spatially-periodic patterns. The new block modes and the interlaced block mode decision make block decisions more precisely, and special patterns that motion compensation cannot be compensated are correctly de-interlaced by these two methods. The subjective view shows an improvement of the accuracy of the motion vectors and the correctness of the mode decision.