Learning from experience: A dynamic closed-loop QoE optimization for video adaptation and delivery

HTTP Adaptive Streaming (HAS) is the de-facto standard for video delivery over the Internet. Splitting the video clip into small segments and providing multiple quality levels per segment allows the client to dynamically adapt the quality to current network conditions. The performance of HAS, and as a consequence the user Quality of Experience (QoE), is influenced by a multitude of parameters. This includes adjustable settings like quality switching thresholds, the initial buffer level, or the maximum buffer, as well as video characteristics like segment duration or the variation of segment sizes along the video. Recently, a couple of analytical models for video streaming have been proposed, allowing to compare these input parameters and derive their impact on QoE-relevant metrics for HAS-based video delivery. The outcome of these models are typically asymptotic probabilities, distribution functions, or centralized and standardized moments. For instance, these models do not yield any temporal information in terms of stalling events or requested video quality. This contradicts to QoE prediction models like P.1203, which compute the QoE based on the chronological sequence of a specific video playback. So far, it is unclear how and to which extent the generalized results of analytical models can be utilized to derive sequence-based QoE values or the QoE distribution for a set of sequences for similar input parameters with stochastic variations. To address this problem, we compare testbed measurements with the output of a GI/GI/1 model with pq-policy and buffer-based quality switching capability to conclude to which extent the results still allow to approximate the video QoE.

With smart and connected communities emerging as an important facet of the Internet of Things (IoT) paradigm, next generation Building Automation Systems (BAS) become evident as one of their prime offerings. The performance of BAS is highly dependent on how well a building does on the fronts of energy efficiency and indoor Quality of Experience (QoE) provided to the users. Higher QoE or comfort expectations impose heavy demands on overall energy consumption of buildings. The inherent conflict in the objectives, namely, maximizing of QoE for users while minimizing the energy costs incurred in the process, is recognised as a complex problem and hence the intrinsic details in its mathematical formulation need attention. This paper presents a survey of energy cost and QoE optimization strategies proposed in recent years with emphasis on the objectives and modelling of the multi-objective optimization problem. A brief summary of the tools and techniques used to solve the optimization problem is also presented. A few research directions are also provided based on the findings of the survey.

The increasing popularity of video (i.e., audio-visual) applications or services over both wired and wireless links has prompted recent growing interests in the investigations of quality of experience (QoE) in online video transmission. Conventional video quality metrics, such as peak-signal-to-noise-ratio and quality of service, only focus on the reception quality from the systematic perspective. As a result, they cannot represent the true visual experience of an individual user. Instead, the QoE introduces a user experience-driven strategy which puts special emphasis on the contextual and human factors in addition to the transmission system. This advantage has raised the popularity and widespread usage of QoE in video transmission. In this paper, we present an overview of selected issues pertaining to QoE and its recent applications in video transmission, with consideration of the compelling features of QoE (i.e., context and human factors). The selected issues include QoE modeling with influence factors in the end-to-end chain of video transmission, QoE assessment (including subjective test and objective QoE monitoring) and QoE management of video transmission over different types of networks. Through the literature review, we observe that the context and human factors in QoE-aware video transmission have attracted significant attentions since the past two to three years. A vast number of high quality works were published in this area, and will be highlighted in this survey. In addition to a thorough summary of recent progresses, we also present an outlook of future developments on QoE assessment and management in video transmission, especially focusing on the context and human factors that have not been addressed yet and the technical challenges that have not been completely solved so far. We believe that our overview and findings can provide a timely perspective on the related issues and the future research directions in QoE-oriented services over video communications.

Evaluating (audio)visual quality and Quality of Experience (QoE) from the user’s perspective, has become a key element in optimizing users’ experiences and their quality. Traditionally, the focus lies on how multi-level quality features are perceived by a human user. The interest has however gradually expanded towards human cognitive, affective and behavioral processes that may impact on, be an element of, or be influenced by QoE, and which have been underinvestigated so far. In addition, there is a major discrepancy between the new, broadly supported and more holistic conceptualization of QoE proposed by Le Callet et al. (2012) and traditional, standardized QoE assessment. This paper explores ways to tackle this discrepancy by means of a multi-instrumental approach. More concretely, it presents results from a lab study on video quality (N=27), aimed at going beyond the dominant QoE assessment paradigm and at exploring affective aspects in relation to QoE and in relation to perceived overall quality. Four types of data were collected: ‘traditional’ QoE self-report measures were complemented with ‘alternative’, emotional state- and user engagement-related self-report measures to evaluate QoE. In addition, we collected EEG (physiological) data, gazetracking data and facial expressions (behavioral) data. The video samples used in test were longer in duration than is common in standard tests allowing us to study e.g. more realistic experience and deeper user engagement. Our findings support the claim that the traditional QoE measures need to be reconsidered and extended with additional, affective staterelated measures.

We consider the problem of optimizing video delivery for a network supporting video clients streaming stored video. Specifically, we consider the joint optimization of network resource allocation and video quality adaptation. Our objective is to fairly maximize video clients' Quality of Experience (QoE) realizing tradeoffs among the mean quality, temporal variability in quality, and fairness, incorporating user preferences on rebuffering and cost of video delivery. We present a simple asymptotically optimal online algorithm, NOVA, to solve the problem. NOVA is asynchronous, and using minimal communication, distributes the tasks of resource allocation to network controller, and quality adaptation to respective video clients. Video quality adaptation in NOVA is also optimal for standalone video clients, and is well suited for use in the DASH framework. Further, NOVA can be extended for use with more general QoE models, networks shared with other traffic loads and networks using fixed/legacy resource allocation.

The use of video streaming services are increasing in the cellular networks, inferring a need to monitor video quality to meet users’ Quality of Experience (QoE). The so-called no-reference (NR) models for estimating video quality metrics mainly rely on packet-header and bitstream information. However, there are situations where the availability of such information is limited due to tighten security and encryption, which necessitates exploration of alternative parameters for conducting video QoE assessment. In this study we collect real-live in-smartphone measurements describing the radio link of the LTE connection while streaming reference videos in uplink. The radio measurements include metrics such as RSSI, RSRP, RSRQ, and CINR. We then use these radio metrics to train a Random Forrest machine learning model against calculated video quality metrics from the reference videos. The aim is to estimate the Mean Opinion Score (MOS), PSNR, Frame delay, Frame skips, and Blurriness. Our result show 94% classification accuracy, and 85% model accuracy (R2 value) when predicting the MOS using regression. Correspondingly, we achieve 89%, 84%, 85%, and 82% classification accuracy when predicting PSNR, Frame delay, Frame Skips, and Blurriness respectively. Further, we achieve 81%, 77%, 79%, and 75% model accuracy (R2 value) regarding the same parameters using regression.

The video delivery over wireless networks has risen in popularity in the recent years. However, in order to provide a high quality of experience (QoE) to the end users, it is necessary to deal with several challenges ranging from the fluctuating bandwidth and scarce resources to the high error rates. The use of these error-prone networks unveils the need for an adaptive mechanism to ensure the quality of the delivered video streams. Adaptive forward error correction (FEC) techniques with QoE assurance are desired to protect the stream, preserving the video quality. The adaptive FEC-based mechanism proposed in this article uses several video characteristics and packet loss rate prediction to shield real-time video transmission over static wireless mesh networks, improving both user experience and the usage of resources. This is possible through a combination of a random neural network, to categorise motion intensity of the videos, and an ant colony optimisation scheme, for dynamic redundancy allocation. The benefits and drawbacks are demonstrated through simulations and assessed with QoE metrics, showing that the proposed mechanism outperforms both adaptive and non-adaptive schemes.

As the demand for mobile services grows exponentially, the focus on wireless network optimization has been changed from Quality of Service (QoS) for the network to Quality of Experience (QoE) for users. The network optimization research about QoS in the past cannot surely meet the requirements of the users' QoE. Therefore, in this paper, a handover solution is proposed to improve the QoE while considering the QoE balance for a LTE network that provides different services. Compared with other optimization of QoE nowadays, the proposed solution balances and improves the QoE of users. The proposed solution controls handover parameter by running a handover optimization algorithm based on the dynamic particle swarm optimization (DPSO) in a central controller, which finally optimizes the overall QoE and reduces the proportion of users with extremely poor QoE. The simulation results show that the DPSO algorithm ensures the quality of the solution and increases the speed of convergence by nearly twice that of the standard particle swarm optimization algorithm (SPSO). After adopting the DPSO handover solution, the overall QoE of users is increased by 6.22 % and 4.59 % while the variance of users' QoE is decreased by 14.2 % and 22.6 %, compared with the full handover solution and the traditional handover solution respectively. The number of users with QoE less than 1.9 is reduced to 0 with the proposed solution.

The demand for multimedia content over the Internet protocol network is growing exponentially with Internet users’ growth. Despite high reliability and well-defined infrastructure for Internet protocol communication, Quality of Experience (QoE) is the primary focus of multimedia users while getting multimedia contents with flawless or smooth video streaming in less time with high availability. Failure to provide satisfactory QoE results in the churning of the viewers. QoE depends on various factors, such as those related to the network infrastructure that significantly affects perceived quality. Furthermore, the video delivery’s impact also plays an essential role in the overall QoE that can be made efficient by delivering content through specialized content delivery architectures called Content Delivery Networks (CDNs). This article proposes a design that enables effective and efficient streaming, distribution, and caching multimedia content. Moreover, experiments are carried out for the factors impacting QoE, and their behavior is evaluated. The statistical data is taken from real architecture and analysis. Likewise, we have compared the response time and throughput with the varying segment size in adaptive bitrate video streaming. Moreover, resource usage is also analyzed by incorporating the effect of CPU consumption and energy consumption over segment size, which will be counted as effective efforts for sustainable development of multimedia systems. The proposed architecture is validated and indulged as a core component for video streaming based on the use case of a Mobile IPTV solution for 4G/LTE Users.

The growing demand on video streaming services is the main motive power for video quality assessment. Due to the rapid development of video streaming services, customers are more enjoying higher quality videos, and their perceptions on video service will directly influence the service provider's performance, which makes it significant to study end-users' subjective perception, named as Quality of Experience (QoE), on video streaming for both service providers and end-users. In this paper, a QoE evaluation model is proposed to predict the end-users' perception on video streaming service considering different video content types. This QoE model, named as Video-Mean Opinion Score (VMOS) model, directly focuses on end-users' feeling. Thus, the Key Performance Indicators (KPIs), which can be straightly felt by end-users, are mapped to QoE score without considering the network parameters. The excellent performance of VMOS model for QoE evaluation on video quality has been verified via plenty subjective Mean Opinion Score (MOS) test, which includes 180 video samples with 1280 valid votes. The Pearson Correlation Coefficient between VMOS score and MOS is as high as 0.925, which indicates that this model can evaluate users' perception on video quality with almost the same accuracy as subjective test.

Recently, with the spread of smart devices and the development of networks, the demand for video streaming has increased, and HTTP adaptive streaming has been gaining attention. HTTP adaptive streaming can guarantee QoE (Quality of Experience) because it selects the video quality according to the network state. However, in wireless networks, delay and packet loss rates are high and the available bandwidth fluctuates sharply. Therefore, QoE is degraded when the quality is selected on the basis of the measured bandwidth. In this paper, we propose an adaptive quality control scheme to improve QoE of video streaming in wireless networks. The proposed scheme calculates two factors, the buffer underflow probability and the instability, by considering the buffer state and the changes of quality level. Using these factors, the proposed scheme defines a quality control region that consists of four sub-regions. The video quality is determined by applying different control strategy to each sub-region. The results of experiments have shown that the proposed scheme improves QoE compared to the existing quality control schemes by minimizing the buffer underflow and the unnecessary quality changes and maximizing the average video quality.

This letter proposes a quality-of-experience (QoE)-aware scheduler that maximizes the average number of satisfied users. QoE is different from quality-of-service (QoS) in that QoE is determined by users' subjective satisfaction while QoS is objectively measured. Hence, users' satisfaction needs to be incorporated into the scheduler to better understand QoE maximization. In this regard we propose a simple but effective user participation scheme exploiting one-bit feedback that indicates user's satisfaction upon the end of service. Then, user-centric QoE function is derived, which serves our objective function. Assuming that the QoE function can be successfully modeled by a sigmoid function we solve the non-convex sigmoid optimization. We show that, interestingly, adding non-trivial fairness constraints for starvation prevention can convexify the problem, and we obtain the unique optimal solution. By comparing the case without user participation we show that user participation can substantially improve the total QoE, e.g., nearly 100% improvement of the low 10 percentile users and 40% improvement of the low 50 percentile users.

In this work, the authors present a novel bandwidth management solution for optimizing overall quality of experience (QoE) of multiple video streaming sessions. Instead of allocating bandwidth equally among competing flows, they propose to tailor the bandwidth allocation to both content complexity of requested video and playout buffer status of individual clients. The authors formulate the multi-client bandwidth allocation problem within the convex optimization framework, which is flexible enough to accommodate a wide variety of video quality metrics. Further, the authors present a practical architecture based on software defined networking (SDN) with two components: video quality monitoring and video quality optimization. Testbed-based experiments confirm that with quality-optimized allocation the network can support up to 75% more users at the same level of quality-of-experience (QoE) than conventional equal-rate allocations.

Cellular network operators are now expected to maintain a good Quality of Experience (QoE) for many services beyond circuit-switched voice and messaging. However, new smart-phone services, such as Over The Top (OTT) video delivery, are not under an operator's control. Furthermore, complex interactions between network protocol layers make it challenging for operators to understand how network-level parameters (e.g., inactivity timers, handover thresholds, middle boxes) will influence a specific app's QoE. This paper takes a first step to address these challenges by presenting a novel approach to estimate app QoE using passive network measurements. Our approach uses machine learning to obtain a function that relates passive measurements to an app's QoE. In contrast to previous approaches, our approach does not require any control over app services or domain knowledge about how an app's network traffic relates to QoE. We implemented our approach in Prometheus, a prototype system in a large U.S. cellular operator. We show with anonymous data that Prometheus can measure the QoE of real video-on-demand and VoIP apps with over 80% accuracy, which is close to or exceeds the accuracy of approaches suggested by domain experts.

Assessing audiovisual Quality of Experience (QoE) is a key element to ensure quality acceptance of today's multimedia products. The use of descriptive evaluation methods allows evaluating QoE preferences and the underlying QoE features jointly. From our previous evaluations on QoE for mobile 3D video we found that mainly one dimension, video quality, dominates the descriptive models. Large variations of the visual video quality in the tests may be the reason for these findings. A new study was conducted to investigate whether test sets of low QoE are described differently than those of high audiovisual QoE. Reanalysis of previous data sets seems to confirm this hypothesis. Our new study consists of a pre-test and a main test, using the Descriptive Sorted Napping method. Data sets of good-only and bad-only video quality were evaluated separately. The results show that the perception of bad QoE is mainly determined one-dimensionally by visual artifacts, whereas the perception of good quality shows multiple dimensions. Here, mainly semantic-related features of the content and affective descriptors are used by the naive test participants. The results show that, with increasing QoE of audiovisual systems, content semantics and users' a_ective involvement will become important for assessing QoE differences.