Deinterlacing algorithm using direction-oriented inverse-free Wiener filtering

The adaptive weighting based on the intensity variations of the neighbor pixels and the distance between the neighbor pixels and a pixel to be interpolated is proposed for the edge-based deinterlacing method. The local edge direction is first determined by the modified edge-based line averaging (MELA) method. Then, according to the local edge direction, the inverse distance and intensity variation based weighting is applied to the fourth-order linear interpolation. The experimental results show that the proposed method provides an improvement over the other existing deinterlacing methods including the MELA method. Also, the proposed method is proved to be suitable for the real-time applications due to its short processing time.

This paper presents an edge-directed image interpolation algorithm. In the proposed algorithm, the edge directions are implicitly estimated with a statistical-based approach. In opposite to explicit edge directions, the local edge directions are indicated by length-16 weighting vectors. Implicitly, the weighting vectors are used to formulate geometric regularity (GR) constraint (smoothness along edges and sharpness across edges) and the GR constraint is imposed on the interpolated image through the Markov random field (MRF) model. Furthermore, under the maximum a posteriori-MRF framework, the desired interpolated image corresponds to the minimal energy state of a 2-D random field given the low-resolution image. Simulated annealing methods are used to search for the minimal energy state from the state space. To lower the computational complexity of MRF, a single-pass implementation is designed, which performs nearly as well as the iterative optimization. Simulation results show that the proposed MRF model-based edge-directed interpolation method produces edges with strong geometric regularity. Compared to traditional methods and other edge-directed interpolation methods, the proposed method improves the subjective quality of the interpolated edges while maintaining a high PSNR level.

Based on the analysis of conventional deinterlacing methods, a new motion adaptive deinterlacing method is proposed. In this paper, we improve three aspects of the conventional motion adaptive method. Firstly, we add a FIR filter to the motion detection part, which not only avoids the wrong detecting of moving objects but also reduces the noises effectively. Secondly, according to the fact, that human eyes are less sensitive to lighter or darker area than gray area, a threshold adaptive method is introduced, which enhanced the subjective quality of deinterlacing. Thirdly, we adopted different interpolation methods for different pixels, which improved the deinterlacing quality of motionless videos. The simulation results show that our proposed deinterlacing method can achieve higher PSNR (peak signal noise ratio) than that of previous studies and can also attain better quality of subjective view.

This paper presents a low-complexity motion compensation (MC) aided motion adaptive (MA) de-interlacing method, aiming at effectively leveraging the fact human eyes are not sensitive to high spatial frequency with rapid motion. The basic idea is to enhance MA de-interlacing by applying MC de-interlacing to process slow-moving objects. Compared with conventional MC de-interlacing, the motion estimation only needs to search in a much smaller area and can adaptively switch between the same parity and opposite parity fields according to the motion vector. Simulation demonstrates that the proposed method only incurs less than 5% of computational complexity of general MC de-interlacing, and meanwhile can improve the PSNR by up to 10 dB compared with conventional MA methods. Its silicon cost and energy consumption are further evaluated by an ASIC design at 0.13 mum technology node.

In this paper, we propose a deinterlacing method with motion adaptive vertical temporal filtering, which utilizes the correlations between adjacent frames. In this model, we first interpolate the missing lines of the current frame and adjacent frames by using an intrafield deinterlacing method. Then we compute the pixel differences between the current frame and the adjacent frames. Since the differences between the adjacent frames would show similar patterns, we can use these patterns to improve deinterlacing performance. In other words, instead of performing deinterlacing in the frame domain, we perform the operation in the frame difference domain. Since the proposed method produces good performance in stationary regions, we selectively apply the vertical temporal filter. Then we apply the proposed method iteratively in order to enhance video quality. The proposed method shows low complexity and still produces superior performance. Experimental results show that the proposed method provides noticeable improvements over existing methods in terms of both subjective and objective evaluations.

This paper proposes a novel interpolation method using edge orientation vector calculated by Hessian matrix. Existing polynomial-based interpolation methods cause blurring effects on edge. In addition, existing edge-based interpolation methods are suffered from edge aliasing and color distortion. To compensate for these problems, we propose an improved edge-based interpolation method considering edge direction on neighbor pixels. To interpolate image, unknown pixels are interpolated by internal division. To determine internal division ratio, the proposed method calculates the edge orientations for diagonal pixels and norm of pixel value differences for horizontal and vertical pixels. The proposed method improves the quality of interpolated images by increasing the average peak signal-to-ratio by 3.33 dB compared to benchmark method.

This study presents a novel intra‐field deinterlacing algorithm using gradient‐guided interpolation (GGI) and weighted average of directional estimation (WADE). In this method, the authors classify each missing pixel into two categories according to different local region gradient features. After the classification of the local region, one of the two different interpolation methods is chosen for the corresponding region. The two interpolation methods are GGI method and the interpolation method using WADE. The average weighted method is used for smooth or texture regions because of its high interpolation performance by adopting the directional detail information among the neighbouring pixels. On the other hand, the GGI method is used for complex regions with strong or weak edges since it has high edge‐preserving ability because of considering not only the similarity of pixel intensities, but also the pixel gradient features. The WADE and GGI methods are the main contributions of this study. Compared with the traditional deinterlacing methods, the proposed method improves both the objective and subjective qualities of the interpolated images. In addition, the proposed method possesses a very low computational burden compared with most intra‐field deinterlacing methods.

In this paper, we propose an efficient de-interlacing method for H.264-coded video sequences with different resolutions. In our proposed method, using the syntax elements (SEs) in the H.264 bitstreams, two new strategies are delivered to improve the de-interlacing quality. The first strategy is based on the intra mode to improve the quality of the regions with skewed edges. The second strategy is based on the inter mode to refine the quality of de-interlaced videos as well as alleviate the error propagation side effect. Experimental results on popular test video sequences with the resolutions of common international format (CIF), quarter CIF (QCIF), standard-definition (SD), and full high-definition (HD) demonstrate the superiority of the proposed de-interlacing method in terms of both the objective video quality measure and subjective visual effect when compared with the state-of-the-art method by Dong and Ngan.

Autocorrelation-based interlaced to progressive format conversion

In general, there are two kinds of linear classification methods: one is MSE, and the other is FLD. Because linear methods are not sufficient to analyze the data with nonlinear patterns, the nonlinear methods KMSE and KFD are hence developed from MSE and FLD, respectively. Both transform the instances from the original attribute space to the high-dimensional feature space and then linear methods are applied. The objective of FLD and KFD is to find the directions on which the projection of training instances can provide the maximal separability of classes. FLD and KFD are known to be inefficient for datasets with a large amount of attributes and instances, respectively. To improve the computing efficiency, we use MSE for linear classification problems. However, MSE, like SVM, can use only the one-against-one or the one-against-the-rest approach to solve the multi-class problems. Both are inefficient compared to FLD and KFD where only one model is built to discriminate multiple classes simultaneously. Thus, we develop the multi-class MSE with Sherman-Woodbury formula to improve the computation efficiency. It can deal with multiple classes simultaneously by a class-labeling scheme. The different class-labeling schemes are determined by the Gram-Schmidt process. The nonlinear application, multi-class KMSE, is also developed from the multi-class MSE. Then, a simulated example is used to show how the proposed method works and to visualize the meaning of the class-labeling scheme. Finally, two real-world datasets are used for comparing the proposed method with other conventional methods.

A four field variable block size motion compensated adaptive de-interlacing method is proposed to improve the accuracy of the motion vectors and lower the occlusions of motion compensated de-interlacing. The proposed de-interlacing method consists of variable block size motion estimation/compensation with four field SAD, interlaced block mode decision, and new block modes. The variable block size motion estimation and compensation improve the accuracy of the motion vectors, especially for spatially-periodic patterns. The new block modes and the interlaced block mode decision make block decisions more precisely, and special patterns that motion compensation cannot be compensated are correctly de-interlaced by these two methods. The subjective view shows an improvement of the accuracy of the motion vectors and the correctness of the mode decision.

In the conventional video processing architecture, it is the responsibility of video decoder to decode the raw picture out while it is the responsibility of video post-processor to do operations such as de-interlacing, de-blocking, alpha-blending, as well as the 3:2 pull down detection in DVD playback solution. Our solution mainly focuses on MPEG-2 video decoding, but it can be used more widely in other video format. Conventionally, the video decoder merely provides useful guidance to the decoded picture consumer, the video post-processor. Normally, decisions are made by video post-processor itself to select de-interlacing methods, such as BOB, WAVE and motion adaptive de-interlacing. This paper introduces an interactive method between video decoder and video post-processor. Before providing decoded frame to video post-processor, the video decoder has make decision upon de-interlacing method by analyzing three or more frames in the frame buffers. This method is proved to be memory saving.

본 논문에서는 시공간 밝기 변화량에 따라 공간적 보간과 시간적 보간 결과를 가중합하여 주사선을 보간하는 효율적인 디인터레이싱 알고리즘을 제안한다. 공간적 보간에서는 에지 방향성 보정 기능을 포함한 새로운 에지기반 공간적 보간 기법을 적용한다. 일반적으로 기존의 에지를 고려한 공간적 디인터레이싱 알고리즘은 에지의 방향을 잘못 추정할 경우 보간된 영상에서 심각한 화질저하가 발생한다. 이를 보완하고 정확한 에지의 방향을 찾아내기 위해 밝기 차만을 이용한 기존 방법들과는 달리 제안된 방법은 에지 방향성을 검출하고 그 정보에 따라 가중최대빈도수 필터를 이용하여 보간될 화소의 에지의 방향성을 보정한다. 또한, 시간적 변화량 검출 오류를 줄이기 위해 미디언 필터를 적용하여 움직임 검출 성능을 향상시키고, 최종 화소 보간과정에서 시공간적인 밝기 변화량에 따라 공간적 보간과 시간적 보간 결과값의 가중합을 사용하여 움직임이 적은 영역에서의 화질을 향상시킨다. 실험결과 제안된 방법은 기존의 디인터레이싱 방법들에 비하여 주관적 화질뿐만 아니라 객관적인 성능도 우수함을 알 수 있다. In this paper, we propose an efficient deinterlacing algorithm which interpolates the missing scan lines by weighted summing of the intra and the inter interpolation pixels according to the spatio-temporal variation. In the spatial interpolation, we adopt a new edge based spatial interpolation method which includes edge directional refinement. The conventional edge dependent interpolation algorithms are very sensitive to noise due to the failure of estimating edge direction. In order to exactly detect edge direction, our method first finds the edge directions around the pixel to be interpolated and then refines edge direction of the pixel using weighted maximun frequent filter. Futhermore, we improve the accuracy of motion detection by reducing the possibility of motion detection error using 3 tab median filter. In the final interpolation step, we adopt weighted sum of intra and inter interpolation pixels according to spatio-temporal variation ratio, thereby improving the quality in slow moving area. Simulation results show the efficacy of the proposed method with significant improvement over the previous methods in terms of the objective PSNR quality as well as the subjective image quality.

Wiener filter is widely used for image denoising and restoration. It is alternatively known as the minimum mean square error filter or the least square error filter, since the objective function used in Wiener filter is an age-old benchmark called the Mean Square Error (MSE). Wiener filter tries to approximate the degraded image so that its objective function is optimized. Although MSE is considered to be a robust measurement metric to assess the closeness between two images, recent studies show that MSE can sometimes be misleading whereas the Structural Similarity (SSIM) can be an acceptable alternative. In spite of having this misleading natured objective function, Wiener filter is being heavily used as a fundamental component in many image denoising and restoration algorithms such as in current state-of-the-art of image denoising- BM3D. In this study, we explored the problem with the objective function of Wiener filter. We then improved the Wiener filter by optimizing it for SSIM. Our proposed method is tested using the standard performance evaluation methods. Experimental results show that the proposed SSIM optimized Wiener filter can achieve significantly better denoising (and restoration) as compared to its original MSE optimized counterpart. Finally, we discussed the potentials of using our improved Wiener filter inside BM3D in order to eventually improve BM3D’s denoising performance.

For practical holographic video system, it is important to generated holographic fringe as fast as possible. We have proposed an approximation method that can calculate the Fresnel hologram fast. To compute the hologram, an object is assumed as a collection of self-illuminated points and the fringes from each object point are superposed. To determine the fringe, a distance between object point and sampling point on the hologram is used to obtain phase of the light. Since sampled hologram usually has small pixel intervals, the difference of the distance values between adjacent pixels is also small and its n-th order difference becomes a constant. Therefore, the distance value at certain pixel can be obtained from the neighbor pixel with simple additions. We have investigated approximation errors and computational speed of the method. The numerical results show that the proposed method is quite effective. The distance error can be reduced less that one wavelength with practical parameters and the computational speed becomes 16 times faster than conventional method. With the proposed method, a hologram, which has horizontal parallax only, 1.3 mega- pixels and 1,000 object points, can be calculated less than on second with a personal computer.