R-D Optimized Frame-Skipping Transcoder for Low Bit Rate Video Transmission

A new 3-D segmentation-based coding technique is proposed, which can transmit a reasonable quality of video through very low bit rate channel. Also, its bit rate regulation is robust to intra frame coding such as scene changes and frame refreshments. Through the experiments, a visually acceptable video quality can be obtained at less than 64 kbps. >

We introduce a new model of video transmission at low bit rates in CDMA cellular systems. This model provides an efficient alternative to computer simulation in the analysis and design of these relatively complex systems. Our model can be used to analyze video sequences even in the presence of rapid motion and fast changes of the motion pattern. Comparisons with results of detailed simulations of the entire system using various video test sequences demonstrate the excellent accuracy of our model. This paper also provides several application examples that illustrate the usefulness of our contribution.

A new feedforward control method to control encod-ing parameters is proposed for low bit rate video encoding. At low transmission bit rates, all input video frames cannot be transmitted and an encoding frame interval varies with the information amount in each encoded frame. Subjective video quality is evaluated from both picture degradation and encoding frame intervals. Because picture degradation and encoding frame intervals are traded off, encoding parameter control is very important. The proposed method is based on a hybrid transform predictive coding scheme with adaptive coefficient selection. The method estimates the amount of information for some encoding parameters and selects parameters so that given control characteristics can be achieved. Computer simulations for both the proposed method and a conventional feedback method show the effectiveness of the new method. This method will be applied to an NETEC series low bit rate video codec.

A subjective quality evaluation was performed to quantify viewer responses to various visual defects that appear in low bit rate video, both at full and reduced frame rates. The stimuli were eight video sequences compressed by three encoders operating at five bit rate/frame rate combinations. The sequences covered a broad range of typical streaming video content. Viewers continuously registered their responses to the stimuli as per the SSCQE method of ITU-R Recommendation BT.500-8. Data from 19 viewers was collected. An analysis of particularly disturbing artifacts is performed. The effects of changes in frame rate at a fixed bitrate on perceived quality are studied, demonstrating a preference for lower frame rates. A comparison of PSNR to the perceived quality ratings is made, revealing several notable trends.

A real-time frame-layer rate control algorithm is proposed for low bit rate video coding over IEEE 802.11 wireless local area network (WLAN). The proposed rate control method performs bit allocation at the frame level to minimize the average distortion over an entire sequence as well as variations in distortion between frames. A new frame-layer rate-distortion model is derived, and a non-iterative optimization method is used for low computational complexity. The proposed algorithm produces low encoding time delay, and is suitable for real-time low-complexity video encoder. Experimental results indicate that the proposed control method provides better visual and PSNR performance than the existing TMN8 rate control method.

This paper proposes a method for performing the camera identification based on very low bit rate videos with the overall noise patterns having time varying statistics. First, the overall noise pattern of each frame of each video is converted to a vector. Then, the odd order statistic moments of these vectors are computed. By performing the principal component analysis, only the most major component of each statistic moment vector is attained. These components of all the frames form a feature vector for each video. To minimize the intraclass separation and maximize the interclass separation, the linear discriminant analysis is performed. As many eigenvalues of the interclass separation matrix are close to zero, the column vectors which span the null spaces of the corresponding matrices are found and the feature vector of each video is projected to these columns and forms a new vector. It is found that these new vectors are pairwisely linear separable. Hence, a bank of perceptrons can be applied to perform the camera identification. Computer numerical simulations show that our proposed method significantly outperforms the conventional correlation based method and the conventional support vector machine based method.

Emotion recognition from facial expressions is tremendously useful, especially when coupled with smart devices and wireless multimedia applications. However, the inadequate network bandwidth often limits the spatial resolution of the transmitted video, which will heavily degrade the recognition reliability. We develop a novel framework to achieve robust emotion recognition from low bit rate video. While video frames are downsampled at the encoder side, the decoder is embedded with a deep network model for joint super-resolution (SR) and recognition. Notably, we propose a novel max-mix training strategy, leading to a single "One-for-All" model that is remarkably robust to a vast range of downsampling factors. That makes our framework well adapted for the varied bandwidths in real transmission scenarios, without hampering scalability or efficiency. The proposed framework is evaluated on the AVEC 2016 benchmark, and demonstrates significantly improved stand-alone recognition performance, as well as rate-distortion (R-D) performance, than either directly recognizing from LR frames, or separating SR and recognition.

The increasing popularity of video consumption from mobile devices requires an effective video coding strategy. To overcome diverse communication networks, video services often need to maintain sustainable quality when the available bandwidth is limited. One of the strategy for a visually-optimised video adaptation is by implementing a region-of-interest (ROI) based scalability, whereby important regions can be encoded at a higher quality while maintaining sufficient quality for the rest of the frame. The result is an improved perceived quality at the same bit rate as normal encoding, which is particularly obvious at the range of lower bit rate. However, because of the difficulties of predicting region-of-interest (ROI) accurately, there is a limited research and development of ROI-based video coding for general videos. In this paper, the phase spectrum quaternion of Fourier Transform (PQFT) method is adopted to determine the ROI. To improve the results of ROI detection, the saliency map from the PQFT is augmented with maps created from high level knowledge of factors that are known to attract human attention. Hence, maps that locate faces and emphasise the centre of the screen are used in combination with the saliency map to determine the ROI. The contribution of this paper lies on the automatic ROI detection technique for coding a low bit rate videos which include the ROI prioritisation technique to give different level of encoding qualities for multiple ROIs, and the evaluation of the proposed automatic ROI detection that is shown to have a close performance to human ROI, based on the eye fixation data.

Multi- rate control technology has important applications in rate control and video transmission. However, most existing methods do not give measure to improve the accuracy of the prediction model parameters. In this paper, we propose sample length adaptive prediction scheme for parameter Estimation in HEVC multi-dimensional rate control. We fully consider frame difference, and set the I frame insertion interval as the length of the sample, In the calculation of the process, according sliding window, bandwidth utilization and FR of last update time to change sample length. At the same time, we focus on how to choose the initial parameters of the video initially acutely moment. We believe such a study brings us a new knowledge into the topic of multi-dimensional rate control and video transmission. The experimental results show that our model-based solutions produce better video quality compared with correlation methods.

Objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to provide an analysis of 4 objective video quality metrics tested on low bit rate videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metric, modified Watson's digital video quality metric, video structural similarity metric, and the perceptual video quality metric. The test videos consist of H.264 compressed videos with CIF and QCIF frame sizes, at various bit rates (24 kbps - 384 kbps) and frame rates (7.5 fps - 30 fps). Here, results and analysis derived from the comparison of the metrics are provided.

We introduce a low-complexity, high-performance DCT-based video coding algorithm that is designed for very low bit rates. The algorithm employs a fast statistical and rate-distortion-optimized motion estimation method with integer-pel accuracy. The searching is constrained to a small diamond-shaped area whose center is the statistical mean, which is obtained using a finite-state machine modeling method based on dynamic state vector quantization. The same modeling method is also employed for coding the motion vectors. A simple rate-distortion criterion is used for alternating between three modes of operation for each 8/spl times/8 block: motion-only coding, motion-compensated predictive coding, and intra coding. Experimental results indicate that the proposed algorithm can outperform current H.263-based video coders significantly at very low bit rates. But, more important is that our algorithm's rate, distortion, and complexity can be easily controlled, a feature desired in many very low bit rate video communication applications due to power and mobility constraints.

To achieve a higher compression ratio for very low bit rate (VLBR) video compression, motion estimation has to be more reliable and easier to code. Many segmentation based techniques try to approximate several motion vectors by a single global motion vector so that fewer vectors are required. However, a global motion vector may not be a good representation for the original vectors and the bit saving is not worth the loss. This paper proposes a new scheme for motion vector representation. Many candidates from full search block matching are selected for merging possibility. Blocks which experience similar displacement are combined to form a single object. Upon compensation for object translation or global compensation, accurate motion of each block can be identified by local compensation within an object. Instead of presenting object movement as a vector, a specific index is assigned to reduce the overhead bits. Based on extensive simulation, the proposed method with an appropriate threshold shows an improvement over standard block based hybrid schemes.

In this paper, an error concealment scheme is proposed to conceal an entirely lost frame in a compressed video bitstream due to errors introduced during transmission. The proposed scheme targets low bit rate video transmission applications using H.264/AVC. The motion field of the lost frame is first reconstructed by copying the co-located motion vectors and reference indices from the last decoded reference frame. After the motion field estimation of the missing frame, motion compensation is performed to reconstruct the frame. This technique reuses existing modules of the video decoder and it does not incur extra complexity compared to decoding a normal frame. It has also been adopted as a non-normative decoder option to the JM reference software at the JVT meeting in Poznan, Poland in July 2005 [1] and has been incorporated into the SA4 video ad hoc group's toolkit at the 3GPP meeting at Paris [2] in September 2005. Simulation results will show its improved performance over other simple error concealment schemes such as "frame copy," both subjectively and objectively, without significant complexity overhead.

A new architecture of wavelet based frame skipping transcoding process for multipoint video conferencing is proposed in this paper. The work involves new structures for a video coder and a transcoder which are wavelet-based. The video transcoder extracts information of motion activities from the video bitstream produced by a wavelet-based video coder. Using the proposed frame skipping process in the wavelet domain, temporal scalability can be achieved. The video quality of inactive sub-sequences can be easily adjusted using the video combiner. This is achieved by discarding fine details of the bitstream or performing frame skipping in the wavelet domain. In other words, more bits can be reallocated to active sub-sequences to achieve a good visual quality with smooth motion. In addition, the video coder is region-based so that different wavelet kernels can be used for the foreground and background. This setting can, on one hand, reduce the computational complexity significantly. On the other hand, by considering unequal importance of various regions, a high video quality for foreground objects is always guaranteed whilst an acceptable background quality can also be maintained even under low bitrate environments. Since the video transcoder only needs to rearrange the video quality level according to their motion activities or performing frame skipping in wavelet domain, a significant saving in computational complexity can be achieved as compared to the conventional video combiner using transcoding approach. The new video coder and transcoder are then used to realize a multipoint video conferencing system. Experimental results are included at the end of the paper, which show a good improvement in performance due to the proposed architecture.

Current rate control schemes in video coding standards do not have efficient frame-level bit allocation because of the inherent constraints in real-time encoding. In this paper, we assume an offline video encoding environment and proposed a rate control scheme based on optimal bit allocation for low bit rate streaming applications. Specifically, we apply a /spl rho/-domain rate-distortion (R-D) model, originally applied at macroblock (MB) level, to frame-level. Based on this frame-level R-D model and a two-pass encoding method, we are able to allocate bits among video frames in an optimal way so that video sequences can be coded at low bit rate with an improved quality. Experimental results demonstrate the proposed scheme is able to achieve not only noticeable reduction in average distortion but also a more consistent and smoother visual quality.