A Framework for Advanced Video Traces: Evaluating Visual Quality for Video Transmission Over Lossy Networks

This paper presents a new scheme for robust video transmission over packet erasure wireless channels based on Wyner-Ziv coding of motion regions. The multipath fading and shading of the wireless channels usually lead to loss or erroneous video packets which on occasions result in some spontaneous drop in video quality. Existing approaches with forward error correction (FEC) and error concealment have not been able to provide the desired robustness in video transmission. We develop a new scheme with a motion-based Wyner-Ziv coding (MWZC) by leveraging distributed source coding (DSC) ideas for error robustness. This new scheme is based on the fact that motion regions of a given video frame are particularly important in both objective and perceptual video quality and hence should be given preferential Wyner-Ziv coding based embedded protection. To achieve high coding efficiency, we determine the underlining motion regions based on a rate-distortion model. Within the framework of H.264/AVC specification, motion region determination can be efficiently implemented using flexible macroblock ordering (FMO) and data partitioning (DP). The bit stream generated by the proposed scheme consists two parts: the systematic portion generated from conventional H.264/AVC bit stream and the supplementary bit stream for error robust video transmission generated by the Wyner-Ziv coding of motion regions. Experimental results demonstrate that the proposed scheme significantly outperforms both decoder-based error concealment (DBEC) and conventional FEC with DBEC approaches.

Video traffic is widely expected to account for a large portion of the traffic in future wireline and wireless networks, as multimedia applications are becoming increasingly popular. Consequently, the performance evaluation of networking architectures, protocols, and mechanisms for video traffic becomes increasingly important. Video traces, which give the sizes, deadlines, and qualities of the individual video frames in a video sequence, have been emerging as convenient video characterizations for networking studies. In this tutorial we give an introduction to the use of video traces in networking studies. First we give a brief overview of digital video and its encoding and playout. Then we present a library of traces of single- and two-layer encoded video. We discuss the statistical properties of the traces and the resulting implications for the transport of video over networks. Finally we discuss the factors that need to be considered when using video traces in network performance evaluations. In particular, we introduce performance metrics that quantify the quality of the delivered video. We outline a procedure for generating video load for network simulations from the traces, and discuss how to meaningfully analyze the outcomes of these simulations.

With the exponential growth of video traffic over wireless networked and embedded devices, mechanisms are needed to predict and control perceptual video quality to meet the quality of experience (QoE) requirements in an energy-efficient way. This paper proposes an energy-efficient QoE support framework for wireless video communications. It consists of two components: 1) a perceptual video quality model that allows the prediction of video quality in real-time and with low complexity, and 2) an application layer energy-efficient and content-aware forward error correction (FEC) scheme for preventing quality degradation caused by network packet losses. The perceptual video quality model characterizes factors related to video content as well as distortion caused by compression and transmission. Prediction of perceptual quality is achieved through a decision tree using a set of observable features from the compressed bitstream and the network. The proposed model can achieve prediction accuracy of 88.9% and 90.5% on two distinct testing sets. Based on the proposed quality model, a novel FEC scheme is introduced to protect video packets from losses during transmission. Given a user-defined perceptual quality requirement, the FEC scheme adjusts the level of protection for different components in a video stream to minimize network overhead. Simulation results show that the proposed FEC scheme can enhance the perceptual quality of videos. Compared to conventional FEC methods for video communications, the proposed FEC scheme can reduce network overhead by 41% on average.

The aim is to provide corresponding quality of service (QoS) guarantee for real-time video data transmission. To ensure the high quality and smooth playback of video sequence at the receiving end, the design of multimedia transmission is made. In view of the shortcomings of selective frame loss, this paper adopts the active frame loss algorithm, which discards the nonkey frames according to the probability. With the increase of the frame loss rate at the transmitter, the proportion of decoded frames increases rapidly and reaches the maximum when the frame loss rate is 0.1. It is proved that active frame loss can control the bit rate more accurately to make full use of bandwidth resources and avoid the waste of bandwidth resources.

Transmission of compressed video through a data network may involve loss of data owing to channel errors and/or congestion. To conceal the effect of such errors in perceived video quality, concealment schemes happen to be an important choice in terms of their pertinence to encoder-decoder models. This paper focuses on the scenario where error bursts do not extend to more than one frame of digital video. For this, there exist block-based and pixel-based concealment techniques and their applicability is often ruled by a trade-off between perceived video quality and computational complexity. The paper proposes a variable complexity concealment technique and compares the results with existing schemes qualitatively and quantitatively for a range of MPEG test sequences and macroblock loss rates.

Adaptive bitstream switching of scalable video

Developing real time multimedia applications for best effort networks such as the Internet requires prohibitions against jitter delay and frame loss. This problem is further complicated in wireless networks as the rate of frame corruption or loss is higher in wireless networks while they generally have lower data rates compared to wired networks. On the other hand, variations of the bandwidth and the receiving device characteristics require data rate adaptation capability of the coding method. Multiple Description Coding (MDC) methods are used to solve the jitter delay and frame loss problems by making the transmitted data more error resilient, however, this results in reduced data rate because of the added overhead. MDC methods do not address the bandwidth variation and receiver characteristics differences. In this paper a new method based on integrating MDC and the scalable video coding extension of H.264 standard is proposed. Our method can handle both jitter delay and frame loss, and data rate adaptation problems. Our method utilizes motion compensating scheme and, therefore, is compatible with the current video coding standards such as MPEG-4 and H.264. Based on the simulated network conditions, our method shows promising results and we have achieved up to 36dB for average Y-PSNR.

The multipath fading and shading of the wireless networks usually lead to the loss or error of video packets which results in significant video quality degradation. Existing approaches with forward error correction (FEC) or error concealment are unable to provide the desired robustness in video transmission. In this work, we develop a novel motion-based Wyner-Ziv coding (MWZC) scheme by leveraging distributed source coding (DSC) ideas for error robustness. The MWZC scheme is based on the fact that motion regions of a given video frame are particularly important to both objective and perceptual video quality and hence should be given preferential Wyner-Ziv coding based embedded protection. To achieve high coding efficiency, we determine the underlining motion regions based on a rate-distortion model. Within the framework of H.264/AVC specification, motion region determination can be efficiently implemented using Flexible Macroblock Ordering (FMO) and Data Partitioning (DP). The bit stream consists of two parts: the systematic portion generated from conventional H.264/AVC bit stream and the supplementary bit stream generated by the proposed feedback free rate allocation algorithm for Wyner-Ziv coding of motion regions. Experimental results demonstrate that the proposed scheme significantly outperforms both decoder-based error concealment (DBEC) and conventional FEC with DBEC approaches.

Video streaming over mobile handsets is among most effective media distribution technologies to reach out end users. In this research we investigate the multiuser shared channel for unidirectional video transmission with adaptive time-varying channel capacity. By providing users with immediate access to video databases across various service providers, the quality of experience and network traffic optimization are two important parameters which are investigated in this paper. According to the video packet type, the new rate control algorithm either exploits a fast adaptation in order to protect user's perceived video quality or helps the operator to increase network throughput. Generally, network throughput and perceptual quality of received video are important parameters of concern in this research. The combined strategy is shown to yield both throughput and video quality that are arbitrarily close to the optimal operating point achieved in video transmission contexts over modern multimedia networks.

In this paper, we propose a multi-objective optimization framework to address the joint mobile association and resource allocation problem in a video transmitted wireless heterogeneous network. We consider user quality of experience (QoE) as one of the design objectives together with two other performance metrics to characterize the design tradeoffs among perceived video quality, power consumption, and network resource consumption. In order to find the Pareto optimal solutions for a multi-objective optimization problem, we first apply weighted Tchebycheff approach to aggregate multiple objectives and minimize the Tchebycheff distance between the optimal solution and utopia solution. Dual decomposition technique is then introduced to decompose the above mentioned problem into a series of similar subproblems. By using linear relaxation and variable transformation, standard convex optimization method is applied to solve these subproblems efficiently and the optimal mobile association and resource allocation are obtained. System level simulation studies numerically demonstrate the tradeoffs among three design objectives and provide an insightful understanding on the performance compromise under multi-objective conditions.

Objective video quality metrics can be viewed as myopic expert systems that focus on particular aspects of visual information in video, such as image edges or motion parameters. We conjecture that the combination of many such high-level metrics leads to statistically-significant improvement in the prediction of reference-based perceptual video quality in comparison to each individual metric. To examine this hypothesis in a systematic and rigorous manner, we use: (i) the LIVE and the EPFL/PoliMi databases that provide the difference mean opinion scores (DMOS) for several video sequences under encoding and packet-loss errors; (ii) ten well-known metrics that range from mean-squared error based criteria to sophisticated visual quality estimators; (iii) five variants of regression-based supervised learning. For 400 experimental trials with random (non-overlapping) estimation and prediction subsets taken from both databases, we show that the best of our regression methods: (i) leads to statistically-significant improvement against the best individual metrics for DMOS prediction for more than 97% of the experimental trials; (ii) is statistically-equivalent to the performance of humans rating the video quality for 36.75% of the experiments with the EPFL/PoliMi database. On the contrary, no single metric achieves such statistical equivalence to human raters in any of the experimental trials.

Demand on multimedia content by consumers' handheld devices over wireless channels is on the increase. In view of the accelerated trend towards consumption of high quality video, power utilization by mobile devices is expected to excessively increase. Hence, it becomes equally important to advance more efficient power minimization techniques, in light of the short battery life in portable devices. However, power minimization algorithms that adopt consumers' perceptual quality of video have not received adequate research. This paper proposes a joint optimization of energy and quality requirements in a multiuser orthogonal frequencydivision multiplexing environment. A multi-objective optimization problem is formulated with the aim to identify bitrate allocations among users such that total power is minimized, and average quality is maximized. For this, a content-aware and energy-efficient resource allocation scheme (CaERAS) is proposed based on genetic and greedy algorithms. Simulation results show that CaERAS, as a lowcomplexity scheme, outperforms comparable methods in terms of efficiency and selectivity of suboptimal solutions. It is shown to acquire a suboptimal solution in as low as 0.0025 of the search space in previous methods. Also, a significant average saving of 85.66% in required energy is observed in broadcast transmission as opposed to unicast transmission.

Although on-device video super-resolution enables high-quality live 360-degree streaming on mobile devices, existing methods often waste energy by overlooking perceived visual quality. In this paper, we present EOS, an energy-efficient on-device super-resolution system for mobile omnidirectional video (ODV) live streaming. EOS reduces energy waste by dynamically adjusting super-resolution complexity based on the predicted visual quality of super-resolved frames. This approach raises two challenges: (1) designing an adaptive inference policy that maximizes energy savings while minimizing degradation in Quality-of-Experience (QoE), and (2) developing a method to predict visual quality under the constraints of mobile ODV live streaming. To tackle these challenges, EOS introduces EOS SR and a No-Reference Up-scaling Quality Prediction scheme. EOS SR employs a device-agnostic, scalable deep neural network optimized for mobile devices, with an energy-aware scheduler that jointly selects the optimal super-resolution model and GPU frequency. The No-Reference Upscaling Quality Prediction scheme estimates visual quality across arbitrary viewpoints in real time without requiring high-resolution reference videos. Experiments on commodity smartphones show that EOS reduces average power consumption by 34.6%–49.9% compared to baseline methods, while preserving high visual quality and frame rates.

Ad hoc networks are characterized with frequent link failures and network topology change. These features result in a great challenge to transmit video over ad hoc networks. We propose a novel scheme for robust video transmission over ad hoc networks. Video stream is coded by using multiple description coding strategy and then a clustering-based multiple paths algorithm is applied for routing over ad hoc networks. The paper describes the general issues of multiple description coding and the principle of clustering-based multiple paths algorithm. Simulation results clearly demonstrate the effectiveness of this scheme for supporting error resilient video transmission over ad hoc networks.

In visual art and design, color composition not only affects the aesthetic value of the work but also plays a deeper role in the psychological feelings and emotional reactions of the audience. The purpose of this study is to explore how color composition affects the prediction of the psychological quality of images so as to provide scientific guidance for artistic creation and design. By using the theories and methods of psychology, aesthetics and computer science, this study conducted a series of experiments and analyses to reveal the mechanism of different color configurations in visual perception and their impact on image quality evaluation. The relevant literature on color theory and visual psychology is understood, and the theoretical framework of the research is established. Then, using computer vision technology, a large number of images with different colors were evaluated for visual quality to obtain objective data. Viewers’ subjective evaluations of these images were collected to examine how color composition affects an individual’s visual experience and emotional response. Color saturation, contrast and tone matching have significant effects on the prediction of the visual psychological quality of images. Color configurations with high saturation and moderate contrast can enhance the appeal and viewing value of the image, while harmonious color combinations can help trigger a positive emotional response from the viewer. In addition, the study also found that individual differences, such as gender, age and cultural background, also affect the effect of color composition on the prediction of visual psychological quality.