Video preprocessing techniques for real-time video compression-noise level estimation techniques

Rate Control is an important component in a video encoder for date storage or real-time visual communication. In this paper, we will discuss the rate control in MPEG encoder for real-time video communication over Variable Bit Rate (VBR) channel. In interactive video communication, the video transmission is subject to both channel rate constraints and end-to-end delay constraints. Our goal in this paper is to modify the rate control in MPEG-2 encoder and satisfy the rate constraints, and study how to improve the video quality in the scenario of VBR transmission. Here, we employ Leaky- Bucket to describe the traffic parameters and monitor the encoder's output. Depending on the Rate-distortion models developed by us, we present a rate control algorithm to achieve almost uniform distortion both within a frame and between frames in a scene. With adaptive rate-distortion models and additional function of scene detecting, our method can robustly deal with scenes of different statistical characteristics. Comparing to MPEG2 TM5, in real time video communication, we could keep the constant buffer delay while maintain the decoded image quality stable. Furthermore, the bit allocation in our algorithm is more reasonable and controllable. Therefore, our method realized the advantages that advanced by VBR video communication, such as small end-to-end delay, consistent image quality and high channel efficiency.

In real-time video communications, the rate control strategies must be utilized to satisfy the end-to-end delay and prevent the encoding buffer from over/underflow. In other words, to acquire the best possible video quality with a minimal quality variation in the playback video, an accurate rate-distortion (R-D) model of the video source is critical in optimizing the bit allocation for video coding. In the paper, an exponential functions based R-D model is proposed for intra-coded frames in video coding. Numerous experiments have consistently shown that the proposed model outperforms other popular R-D models in terms of both the estimation accuracy and computation complexity, making it suitable for rate control in real-time video coding.

Till recently dedicated hardware has been required to perform real-time MPEG-2 video encoding. However, with increase in clock frequency and the introduction of video-specific instruction sets, general-purpose processors and DSPs can now approximate the function and performance of single-function hardware. In this paper, a software-only MPEG-2(MP@ML) video encoder implementation aimed at real-time operation on embedded platforms and general-purpose processors is described. The main contribution of this work is a set of algorithmic optimizations that significantly reduces the computational complexity of the encoding process while slightly degrading the subjective video quality compared to encoders that are more exhaustive.

Interpolation is the main bottleneck in AVS real-time high definition video encoder for its high memory bandwidth and large calculation complexity caused by the new coding features of variable block size and 4-tap filter. In this paper, an efficient VLSI architecture of interpolation supporting AVS Baseline@L4 is presented. Vertical redundant data reuse, horizontal redundant data reuse and sub-pixel data reuse schemes are presented to reduce memory bandwidth and processing cycle. The separated 1-D interpolation filters are used to improve throughput and hardware utilization. The proposed design is implemented on the Vertex4 XC4VLX200 field programmable gate array with operating frequency of 150 MHz and can support 1080p (1920 times 1080) 30 fps AVS real-time encoder. It is a useful intellectual property design for real-time high definition video application.

This paper describes very cost-effective algorithms to detect scene changes and field repetition in video sequences for a real-time MPEG encoder. It also provides a scheme to support a dynamic GOP structure reflecting the detection outcome on the fly. With these features, the encoder can encode video more efficiently in either quality or bitrate aspect. The proposed detection methods only utilize the existing information, field motion vectors and picture coding type, from MPEG coding and are very suitable for (but not limited to) software-based encoders.

In order to enable real-time software-based video encoding, in this work we optimized the prediction stage of an H.264 video encoder, in the complexity sense. Thus, besides looking for the coding options which lead to the best coded representation in terms of rate and distortion (RD), we constrain to a complexity (C) budget. We present a complexity optimized framework (RDC-optimized) which allows for real-time video compression and that does not make use of frame-skipping to comply to the desired encoding speed. We developed our framework around an open source software implementation of the H.264/AVC, the the ×264 encoder. Results show that tight complexity control is attainable in practice, with very little loss in RD performance.

Parallel fast inter mode decision for H.264/AVC encoding

A real-time frame-layer rate control algorithm is proposed for low bit rate video coding over IEEE 802.11 wireless local area network (WLAN). The proposed rate control method performs bit allocation at the frame level to minimize the average distortion over an entire sequence as well as variations in distortion between frames. A new frame-layer rate-distortion model is derived, and a non-iterative optimization method is used for low computational complexity. The proposed algorithm produces low encoding time delay, and is suitable for real-time low-complexity video encoder. Experimental results indicate that the proposed control method provides better visual and PSNR performance than the existing TMN8 rate control method.

This paper describes a motion estimation (ME) engine that achieves both power efficiency and coding quality for real-time ultra-high definition video encoders. Sum of absolute difference (SAD) calculations are performed with bit-shortened SAD engines to reduce power, and bit extraction positions are adaptively shifted in accordance with the flatness of picture luminance values to maintain calculation precision. Power simulations with a High Efficiency Video Coding encoder LSI design [2] show that the ME engine reduces power consumption by 18–39% with 4-bit shortened SAD calculation, while coding efficiency loss is suppressed by up to 57% with adaptive bit-shifting. The ME engine is especially effective for high dynamic range pictures with downscaled luminance values, which are suitable for today's 4K and 8K video contents.

A real-time software MPEG1 video encoder on a multimedia RISC processor, V830R/AV, is presented. The V830R/AV processor provides 64-bit SIMD media-enhanced instructions. The motion estimation is accelerated by a combination of the partial absolute difference instruction and a method to sum up partial block differences. The IDCT in conformity with the IEEE standard is simply implemented by the multiply-accumulate instruction with symmetric rounding. The encoder program structure is carefully reconstructed to reduce instruction cache misses, which heavily degrade the processor performance. Preload instructions are used to load data to the data cache in parallel with successive instruction execution. Consequently, the cache miss penalties are drastically reduced. The current version of the software MPEG1 video encoder on the 200 MHz V830R/AV processor encodes SIF (352/spl times/240 pels) video at 30 frames/s with I, P and B picture types in 193 M clocks/s.

Embedded video surveillance has become a large market as the number of installed cameras around us can show. At the same time, video compression technique has been rapidly developed in recent years. As a new generation of video coding compression standard, extensive attention has been given to H.264/AVC. On the other hand, because of good cost-effective and low-power consumption, DSP is widely used in the fields of embedded system and multimedia processing. There are needs for embedded video surveillance that can make use of the video compression technique in more efficiency security systems. In this paper, a scheme of design and implementation about video surveillance system based on H.264/AVC is designed and described. It is built on ADSP-BF561, with a wide range of peripheral interface, to achieve several functions, such as video compression, data stream control, network transmission, PTZ and so on. The experimental results indicate that the stability and other expected features of system, including Real-time surveillance video encoding, can be ensured.

In this paper, we propose a fast and effective mode decision (MD) algorithm based on rate distortion optimization (RDO) for AVS high definition video encoder. The fast algorithm is composed of two parts. Firstly, mode pre-selection based on sum of absolute difference (SAD) is employed to reduce modes for candidate so as to alleviate the dramatic throughout burden. Secondly, we adopt 4-way parallel scanning technique to reduce the cycles of each mode decision based on RDO. Theoretical analysis and experimental results show that the proposed fast algorithm can meet the needs of 720P and 1080P real-time high definition AVS video encoding. Besides, the mode pre-selection algorithm provides a similar performance to the all modes enabled algorithm. And the 4-way parallel technique using negligible extra resources increases the speed of coding bits estimation by 3.4 times than traditional techniques.KeywordsAVSmode decisionRDOzigzag scanVLC

In this paper the filtering of noise in image sequences using spatio-temporal motion compensated techniques is considered. Noise in video signals degrades both the image quality and the performance of subsequent image processing algorithms. Although the filtering of noise in single images has been studied extensively, there have been few results in the literature on the filtering of noise in image sequences. A number of filtering techniques are proposed and compared in this work. They are grouped into recursive spatio-temporal and motion compensated filtering techniques. A 3-D point estimator which is an extension of a 2-D estimator due to Kak [5] belongs in the first group, while a motion compensated recursive 3-D estimator and 2-D estimators followed by motion compensated temporal filters belong in the second group. The motion in the sequences is estimated using the pel-recursive Wiener-based algorithm [8] and the block-matching algorithm. The methods proposed are compared experimentally on the basis of the signal-to-noise ratio improvement and the visual quality of the restored image sequences.

The exploitation of spatial and temporal correlation within video signals is a cornerstone of video coding, which is extensively employed to achieve high coding efficiency. The presence of noise in video signal deteriorates the correlations, leading to a degradation in coding efficiency. Motion Compensated Temporal Filter (MCTF) is a pre-processing tool designed to remove noise from the source signal, thereby enhancing temporal correlations among adjacent frames. In this paper, a Content-Adaptive MCTF method (CAMCTF) is proposed to enhance the filtering adaptability of MCTF for diverse video contents. The proposed CAMCTF method consists of Quadtree Block Partitioning Scheme (QBPS), Offset Block Weighted Compensation (OBWC) and Structural Similarity (SSIM) based Filtering Weight Adjustment (SSFWA). Specifically, QBPS generates size-adaptive Motion Compensation Block (MCB) to better cater to the video content characteristics. Subsequently, OBWC is employed to address the uneven prediction quality of MCBs. Furthermore, the filtering weights of each MCB are fine-tuned by SSFWA based on the SSIM index. The simulation results show that the proposed CAMCTF method can achieve 1.23% and 0.94% BD-rate gains for RA and LD configurations, respectively, on top of Versatile Video Coding (VVC) reference software VTM-23.4.

In order to tackle the recording video denoising problem with non-stationary image contents, a critical task is to estimate statistical properties of the composite noise in video signals. After investigation of a variety of test video sequences, based on the wavelet decomposition of the noise, the estimation can be conducted by empirical modeling of the marginal distributions of wavelet coefficients as a subband-dependent parameterized generalized Gaussian distribution. Further development of a new spatio-temporal based composite noise suppression technique is also provided to restore surveillance video, based on the estimation of the parameters of the model. The experiments showed that the proposed techniques obtained promising results.