Video Coding

Motion correction in PET has become more important as system resolution has improved. The purpose of this study was to evaluate the accuracy of three motion compensation methods, event-by-event motion compensation with list-mode reconstruction (MOLAR), frame-based motion correction, and post-reconstruction image registration. Motion compensated image reconstructions were carried out with simulated HRRT data, using a range of motion information based on human motion data. ROI analyses in high contrast regions were performed to evaluate the accuracy of all the motion compensation methods, with particular attention to within-frame motion.Our study showed that MOLAR with list-mode based motion correction using accurate motion data can reliably correct for all reasonable head motions. Over all motions, the average ROI count was within 0.1±4.2% and 0.7±0.9% of the reference, no-motion value for two different ROIs. The location of the ROI centroid was found to be within 0.7±0.3mm of that of the reference image for the raphe nucleus. Frame-based motion compensation and post-reconstruction image registration were able to correct for small (<5mm), but the ROI intensity begins to deteriorate for medium motions (5-10mm), especially for small brain structures such as the raphe nucleus. For large (>10mm) motions, the average centroid locations of the raphe nucleus ROI had an offset error of 1.5±1.8mm and 1.8±1.8mm for each of the frame-based methods. For each frame-based method, the decrease in the average ROI intensity was 16.9±4.3% and 20.2±9.9% respectively for the raphe nucleus, and was 5.5±2.2% and 7.4±0.2% for putamen. Based on these data, we conclude that event-by-event based motion correction works accurately for all reasonable motions, whereas frame-based motion correction is accurate only when the within-frame motion is less than 10mm.

In this paper, an improved scene-based nonuniformity correction (NC) algorithm for infrared focal plane arrays (IRFPAs) using multiframe registration and iteration method is proposed. This method estimates the global translation and iterates between several adjacent frames. Then mean square error between any two properly registered images is minimized to obtain nonuniformity correction parameters. The detailed method includes three main steps: First, we assume that brightness along the motion trajectory is constant, and a linear detector response and model the nonuniformity of each detector with a gain and a bias. Second, several adjacent frames are used to compute relative motion of any two adjacent frames. Here we use the Fourier shift theorem, their relative translation can be obtained by calculating their normalized cross-power spectrum. We choose K adjacent frames, so the total number of iteration is K*(K-1)/2. Then the mean square error function is defined as the corresponding difference between the two adjacent corrected frames, and it is minimized making use of the least mean square algorithm. The use of correlation of adjacent frames sufficiently, together with iteration strategy between them, can get fast and reliable fixed-pattern noise reduction with low few ghosting artifacts. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods. The performance of the proposed method is thoroughly evaluated with clean infrared image sequences with synthetic nonuniformity and real infrared imagery.

Motion estimation and motion compensation (MEMC) has been applied inside the wavelet-transformed domain for video compression at low bit rates. However, pixel-based motion estimation in the wavelet domain is not as straightforward as in the spatial domain due to the shift variance property caused by decimation. The paper finds a new wavelet-based motion estimation and compensation method, named as the sub-decimation decomposition method, to perform shift invariant motion estimation inside the wavelet transformed domain, so that better matching blocks are compensated to achieve more efficient video compression.

This paper proposes a new motion compensation method using "sliced blocks" in DCT-based hybrid video coding. In H.264 | MPEG-4 Advance Video Coding, a brand-new international video coding standard, motion compensation can be performed by splitting macroblocks into multiple square or rectangular regions. In the proposed method, on the other hand, macroblocks or sub-macroblocks are divided into two regions (sliced blocks) by an arbitrary line segment. The result is that the shapes of the segmented regions are not limited to squares or rectangles, allowing the shapes of the segmented regions to better match the boundaries between moving objects. Thus, the proposed method can improve the performance of the motion compensation. In addition, adaptive prediction of the shape according to the region shape of the surrounding macroblocks can reduce overheads to describe shape information in the bitstream. The proposed method also has the advantage that conventional coding techniques such as mode decision using rate-distortion optimization can be utilized, since coding processes such as frequency transform and quantization are performed on a macroblock basis, similar to the conventional coding methods. The proposed method is implemented in an H.264-based P-picture codec and an improvement in bit rate of 5% is confirmed in comparison with H.264.

Video compression coding technology is the key to implement network video transmission, and the increasing requirements of network content transmission quality accelerate the development of video compression coding technology. H.264 is a highly compressed digital video codec standard, and is a hybrid coding mode of differential pulse code modulation (DPCM) and transform encoding. In this paper, we introduced several techniques of hybrid coding algorithm based on H.264 standard, including differential coding, block matching motion prediction method, normalization criteria, variable-size block motion compensation, and multiple hypothesis motion compensation. We did extensive experiments on variable-size block motion compensation coding method. The experimental results show that, with the premise of the 1/4 pixel motion compensation accuracy and hybrid coding, 8 × 8 or larger block partition of content for normal video object, can further make use of space correlation and improve compression efficiency.

In this paper, a fuzzy inference digital image stabilization method is proposed to adaptively determine better motion compensation method through the selection of two different motion compensation methods. The experimental results show the good performance. It solves the problem of lag and overshoot of motion trajectory.

A motion compensation method that is aimed at correcting motion artifacts of cardiac valves is proposed. The primary focus is the aorticvalve. The method is based around partial angle reconstructions and a cost function including the image entropy. A motion model is applied to approximate the cardiac motion in the temporal and spatial domain. Based on characteristic values for velocities and strain during cardiac motion, penalties for the velocity and spatial derivatives are introduced to maintain anatomically realistic motion vector fields and avoid distortions. The model addresses global elastic deformation, but not the finer and more complicated motion of the valve leaflets. The method is verified based on clinical data. Image quality was improved for most artifact-impaired reconstructions. An image quality study with Likert scoring of the motion artifact severity on a scale from 1 (highest image quality) to 5 (lowest image quality/extreme artifact presence) was performed. The biggest improvements after applying motion compensation were achieved for strongly artifact-impaired initial images scoring 4 and 5, resulting in an average change of the scores by and , respectively. In the case of artifact-free images, a chance to introduce blurring was observed and their average score was raised by 0.42 ± 0.03. Motion artifacts were consistently removed and image qualityimproved.

A fine image motion compensation method for the panoramic TDI CCD camera in remote sensing applications

Motion compensation (MC) is one of the most important technologies capable of removing the temporal redundancy and widely adopted by the main video standards. From the older MPEG-2 to the latest H.264 and the Chinese AVS, many efficient coding tools have been introduced into MC, such as new motion vector prediction, bi-directional matching, quarter precision interpolation, etc. However, these new features enormously increase the computational complexity and the memory bandwidth consumption. In this paper, we introduce a novel architecture design of motion compensation (MC) for multiple video standards including MPEG-2, H.264, and AVS. The proposed design has a macroblock- level pipelined structure which consists of MV predictor, cache-based Fetch, and pixel interpolation unit. The proposed architecture exploits the parallelism in MC algorithm to accelerate the processing speed and uses the dedicated design to optimize the memory access. MV predictor unit can cover all MV prediction algorithms for the three standards and provide a simple error concealment scheme. Cache-based Fetch unit can save 25% memory bandwidth of MC in average and doesn't impact the performance in the worst case. Pixel interpolation unit adopts fully separate 1D filtering structure which is designed to effectively avoid the redundant calculations. The architecture can achieve the real-time multiple-standard decoding for HDTV 1080i (1920*1088 4:2:0 60 field/s) video. The efficient design can work at the frequency of148.5MHz and the total gate count for logic circuit s is about 56K.

Motion compensation is one of the most essential methods for any video compression algorithm. Video frame prediction is a task analogous to motion compensation. In recent years, the task of frame prediction is undertaken by deep neural networks (DNNs). In this thesis we create a DNN to perform learned frame prediction and additionally implement a codec that contains our DNN. We train our network using two methods for two different goals. Firstly we train our network based on mean square error (MSE) only, aiming to obtain highest PSNR values at frame prediction and video compression. Secondly we use adversarial training to produce visually more realistic frame predictions. For frame prediction, we compare our method with the baseline methods of frame difference and 16x16 block motion compensation. For video compression we further include x264 video codec in the comparison. We show that in frame prediction, adversarial training produces frames that look sharper and more realistic, compared MSE based training, but in video compression it consistently performs worse. This proves that even though adversarial training is useful for generating video frames that are more pleasing to the human eye, they should not be employed for video compression. Moreover, our network trained with MSE produces accurate frame predictions, and in quantitative results, for both tasks, it produces comparable results in all videos and outperforms other methods on average. More specifically, learned frame prediction outperforms other methods in terms of rate-distortion performance in case of high motion video, while the rate-distortion performance of our method is competitive with that of x264 in low motion video.

Video transcoding is one of the most essential technologies in consumer electronics server and gateway products for a variable bitrate video streaming system. Especially, for a real-time streaming system, drift-error resilient coded bitstreams that do not require high complexity transcoding processes are desired. For the generation of drift-error resilient coded bitstreams at the encoder side, an investigation on the drift-error propagation characteristic in H.264/AVC by applying different motion compensation (MC) precisions are carried out in this paper. The results show that the drift-error propagation in half-pixel MC precision increases due to the usage of six-tap filter. For quarter-pixel MC precision, the drift-error propagation decreases. Besides that, we found out that the requantization parameter and MC precision only affects the amount of error in the requantized frame but amount of error propagation depends on the MC precision and sequence of the pixel value. It is also found that drift-error propagation ratio is defined at some point regardless of MC precisions.

We present a new method for motion compensation, which combines transformations in the spatial and luminance domains. In most standardised video compression algorithms, motion information is assumed to be translation only, as used by the traditional block matching algorithm (BMA). Such an assumption is not always correct, as in most scenes, namely head and shoulders, object motions are rotations and zooms, among other effects, which cannot be represented as translations. In order to compensate these complex motion features more accurately, we applied geometric transformations (BMGT) for motion compensation. Nevertheless, spatial transformations alone are unable to compensate situations like uncovered background, masking between objects or changes in lighting conditions. In this sense, we introduced an additional transformation in the luminance domain (BMGTI), which has proven to be appropriate to overcome such problems. Experimental results have shown that BMGTI exceeds the performance of BMGT by about 2 dB, and achieves better results than the global brightness compensation (GBC) technique. This method is implemented using a spatial hierarchical block structure, which allows video encoding at low bit rates and reduction of the computational complexity.

With adoption of the hopped-frequency pulse waveform to inverse synthetic aperture radar (ISAR), a novel method for motion compensation in ISAR imaging, named the minimum waveform entropy method of parameter estimation, is proposed. A measure function, called the waveform entropy, is employed to measure the effects of target motion on both Doppler amplitude spectra and slant range profiles. The method can achieve optimal estimation of motion parameters of a target based on the minimum waveform entropy rule. In addition, the method converts a two-dimensional search for the velocity and acceleration of a target into two one-dimensional searches for the velocity and acceleration respectively, which greatly decreases computational cost, and conveniently realizes real-time processing. Simulation results indicate that the method can accomplish accurate estimation of motion parameters of a target with a good anti-noise performance, thus making it possible to image a target in clutter background.

Studying the behaviour of cells using time-lapse microscopic imaging requires automated processing pipelines that enable quantitative analysis of a large number of cells. We propose a pipeline based on state-of-the-art methods for background motion compensation, cell detection, and tracking which are integrated into a novel semi-automated, learning based analysis tool. Motion compensation is performed by employing an efficient nonlinear registration method based on powerful discrete graph optimisation. Robust detection and tracking of cells is based on classifier learning which only requires a small number of manual annotations. Cell motion trajectories are generated using a recent global data association method and linear programming. Our approach is robust to the presence of significant motion and imaging artifacts. Promising results are presented on different sets of in-vivo fluorescent microscopic image sequences.

CyberKnife, a stereotactic radiosurgery (SRS) technique, delivers highly precise radiation beams to tumors. However, the patient motion, such as respiration and positional changes, can lead to tumor displacement, affecting the treatment efficacy. Fiducial marker tracking plays a crucial role in CyberKnife therapy, enabling real-time tumor position monitoring and ensuring accurate radiation delivery. This paper reviews motion compensation methods based on fiducial marker tracking, explores how fiducial markers facilitate precise tumor localization (tracking precision: 0.2–0.5 mm), and analyzes various motion compensation techniques, including respiratory motion compensation and positional adjustment. By dynamically adjusting the treatment device and radiation dose, fiducial marker tracking and motion compensation significantly enhance the precision of CyberKnife therapy, reduce errors caused by patient motion, and improve treatment outcomes.