Predicting Perceptual Quality in Internet Television Based on Unsupervised Learning

Introduction and penetration of high-speed and innovative mobile data services including social media and video services have been increasing significantly across the globe. This growth is driven by deployment and availability of capable and affordable mobile networks and smartphones. These trends have also been observed in Ethiopia as the country’s sole incumbent operator ethio telecom has expanded coverage and capacity of its mobile data network, especially for universal mobile telecommunications system (UMTS) network. The successful provision of these innovative services requires mobile operators to have continuous monitoring of the quality of their networks and customer satisfaction. To that end, this monitoring allows the operator to make timely network optimization, marketing and other relevant decisions that enhance service quality. Traditionally measurement and analysis of network quality of service (QoS) is performed using data from network management system (NMS) and drive/walk test (DT/WT). In recent times, the usage QoS data from crowdsourcing (CS) is also emerging as useful monitoring tool. However, all these methods do not fully capture users’ quality of experience (QoE), which is a function of not only network QoS but also other factors including subjective users’ technical literacy and opinion. As a result, operators have recently provided attention not only for QoS but also QoE. Yet, most quality related analyses are performed using data from only subjective survey, NMS, DT/WT, or CS. It is not straightforward to find quality investigation based on data from all the methods. More importantly, such quality investigation has not been made in the context of Ethiopia’s mobile market. In this paper, we perform QoS and QoE evaluation for UMTS mobile data service based on quality data from all NMS, WT, CS and subjective survey for a business area in Addis Ababa. The evaluation is made using downlink user throughput and mean opinion score (MOS) metrics. The obtained 50th percentile throughput results from NMS, WT and CS are good enough for web browsing and offline video streaming but not for high definition (HD) video calls and live HD video streaming. Furthermore, CS provides optimistic results compared to NMS and WT and its 90th percentile throughput is very good even for live HD video streaming. Moreover, the obtained MOS results from subjective survey almost fit to the QoS results, thus indicating that the network QoS is a significant contributor to users’ perceived QoE.

The quality perceived by the user, a concept known as Quality of user Experience (QoE), is gaining momentum. Together with the efforts of providing appropriate architectures able to deliver the expected level of Quality of Service (QoS), users' perception of the level of quality could play a key role in making a VoIP multimedia application to succeed or to fail. QoE can be measured from different approaches; subjective ones are based on polls while objective ones are mainly based on QoS network performance parameters. Although a subjective evaluation could be costly and time consuming, it gives the most accurate results. Moreover, wireless environments incorporate new challenges in terms of QoS and QoE because of inherent wireless communication characteristics. In this way, our contribution is twofold. First, we carry out a subjective QoE assessment for VoIP applications in a real wireless environment. Skype, Gizmo5, ooVoo, and Damaka are the selected VoIP applications under study. Next, we contrast QoE results with QoS evaluation from the captured video calls made during the poll. Results show that there is no precise match on both assessments, because of QoE parameters that cannot be inferred from QoS analysis, because of the different effect of some QoS parameters which prevail, or even because of the popularity of the VoIP application.

Betas: Deriving quantiles from MOS-QoS relations of IQX models for QoE management

무선 접속을 통한 인터넷 서비스가 보편화된 최근 사람이 서비스를 이용하는 과정에서 실제로 느끼는 품질인 사용자 체감 품질(QoE; Quality of Experience)의 중요성이 더욱 강조되고 있는데 사용자 체감 품질은 서비스 품질 (QoS; Quality of Service)와 같이 객관적인 수치화가 불가능하다는 특징이 있다. 유선과는 달리 무선 접속을 통해서 제공되는 IP 서비스는 매우 많은 요인에 의해서 사용자 체감 품질이 영향을 받을 수 있다. 따라서 본 논문에서는 대표적인 무선 접속 서비스인 HSPA (High Speed Packet Access)를 통해서 음성 서비스가 제공될 때 측정 가능한 품질지표를 선정하고 이들에 대한 실측값을 통계적으로 분석하여 서비스 품질과 사용자 체감 품질 지표와의 상관관계를 밝히고자 한다. 분석 결과 RSSI (Received Signal Strength Indicator)와 전송 지연의 상관관계가 매우 높고 그에 이어 전송 지연과 MOS (Mean Opinion Score)와 매우 높은 상관관계를 가짐을 알 수 있었다. As internet services are being provided through a wireless access, the importance of quality of experience (QoE) is stressed that is defined as the quality that indicates user's actual feeling when a service is provided. Unlike quality of service (QoS) that can be expressed as a numerical value, it is difficult to represent QoE in an objective way. If an internet service is serviced over a wireless channel, its QoE can be affected by a number of factors such as fading, mobility and so on. This paper, therefore, attempts to specify the relationship between QoE and QoS by conducting practical measurements for the voice service through 3G high speed packet access (HSPA) access network. Analysing the measured results, it has been shown that received signal strength indicator (RSSI) has a great influence on mean opinion score (MOS) through transmission delay.

Massively Multiplayer Online Cloud Game (MMOCG) is a combination of cloud gaming and Massively Multiplayer Online Game (MMOG). Running MMOGs on the cloud can reduce the cost for the players, attract potential players, and meet the nowadays preference of mobile devices. However, the MMOCGs introduce new challenges that need to be addressed. MMOGs usually consume more computational resources because of their sophisticated graphics. Hosting MMOCGs can be more expensive for game operators than hosting other genres of games on the cloud. In order to earn a decent profit, the cost of hosting MMOCGs must be reduced to an acceptable cost. Moreover, MMOCG introduce extra cloud servers to MMOG, which increased the burden of network transmission. So, the MMOCGs must have a good quality of service (QoS) and quality of experience (QoE) to retain the players. In this thesis, we study the server allocation problem to improve MMOCGs in many aspects, including money cost, QoS, and QoE. We proposed three different server allocation strategies to address the challenges of MMOCGs in different situations. We first consider the MMOCG system with several cloud gaming platforms that lie in different locations in the world, each can provide a type of server that can host MMOG on them. The cloud gaming system only obtains the information necessary for gaming. With only the log-in information, a server allocation problem is defined to reduce the money cost of hosting MMOCGs and provide an acceptable network latency for the players. The problem is formulated into an optimization problem that aims to minimize the weighted combination of server rental cost, data transfer cost, and network latency. A genetic algorithm is developed to solve the minimization problem. Extensive experiments are conducted to evaluate the proposed method and compare it with the state-of-the-art as well. The experimental results show that the method gives a lower money cost and a shorter network latency most of the time. We then consider the MMOCG system that can collect the past logging in and out data of the players. We proposed a server allocation problem that aims to reduce the cost of cloud server rental and improve the QoE of the system. A genetic algorithm with gaming time prediction is proposed to solve the problem. The gaming time of the players logging into the system is predicted by the model trained by a neural network. We conduct experiments to evaluate our method and the proposed gaming time prediction model. The results show that our prediction model can achieve almost highly accurate prediction and the server allocation strategy we proposed outperforms the prior art in both server rental and QoE. At last, we consider the influence of QoE on the movement of players during gaming time and propose a server allocation problem aiming to improve the QoE of players during gaming time. We propose a genetic algorithm with trace prediction to address this problem. A trace prediction that predicts the future positions of players for a few time steps when they log into the MMOCG system. Experiments are conducted to evaluate our method by measuring the network latency, mean opinion score, and the time consuming computation. The results show that our method outperforms the genetic algorithm without trace prediction in over 50% of the cases.

In an era of rapid urbanization, the quality of network services in urban environments is becoming increasingly important. This article explores efforts to improve network service quality in several sampang cities through an evaluation of Quality of Service (QoS) and user perceptions of Quality of Experience (QoE). The research is based on a survey conducted at various locations in sampang city to collect data on the reliability, speed, and availability of network services. The survey involved a diverse sample of users, covering a wide range of ages, professions, and levels of technology usage. In addition, we also analyzed users' perceptions of service quality based on their experiences of using the network in an urban environment. The results highlighted several key challenges in improving QoS, such as user density, signal interference, and network infrastructure limitations. However, we also found that strategies such as the use of advanced network technologies, such as 5G and beamforming, as well as traffic prioritization based on application type, can help to significantly improve QoS. In addition, we found a strong positive correlation between improved QoS and improved QoE, suggesting that improved quality of service has a direct impact on improving user experience. These findings provide valuable insights for network operators and policy makers in their efforts to improve network services in dense urban environments. The research also recommends further investment in network technologies and the development of policies that support more effective management of network traffic to ensure optimal quality of service.

Recently, the evaluation on quality of the air interface in a wireless network has shifted from emphasis on QoS (quality of service) only to incorporation of QoE (quality of experience). However, it seems that QoE metrics have not been related to the surroundings of the served users so far, which may have a great influence on users' expected QoE. For instance, with a high level of QoS, a user's expectation of QoE could still be terrible because of the serious interference from the surroundings such as noise, ambient light or shaking of terminals due to mobility. From another perspective, it means that in such a case a high level of QoS may be helpless to enhance users' QoE and the requirement could be relaxed accordingly. In this paper, an approach called surrounding-adaptive QoE adjustment (SAQA) is proposed to optimize the capacity and the quality of air interface with voice service as an example. The basic idea of the approach is that user equipment (UE) detects the surroundings interference which may impact users' QoE and then adjusts QoS according to the detected results: a lower level of QoS is chosen in a bad environment to enhance the capacity, while a higher level of QoS is selected to guarantee the users' QoE in a better circumstance. Simulation results show that this method can remarkably enhance the voice capacity of the communication systems as well as guarantee users' QoE.

Multimedia applications have expanded and resulted in huge demands for high resolution videos in order to enhance the end user Quality of Experience (QoE). Therefore, QoE becomes a critical factor that is often addressed in design considerations of Consumer Electronics (CE) devices. Furthermore, error prone transmission conditions can influence the QoE, potentially degrading the end-user multimedia experience. Although video coding standards such as Versatile Video Coding (VVC) take potential mitigation measures to improve the bandwidth reductions, it fails to address other significant network Quality of Service (QoS) parameters that impact transmission based QoE such as transmission delay and Packet Loss Rate (PLR) in the encoding processes. In response, this paper first derives a relationship between QoE that addresses the network QoS parameters and VVC dependent parameters. Subsequently, the derived QoE model is used with the VVC encoding process to adaptively optimize Lagrange parameters as well as the quantization parameter. Extensive experiments report significant improvements to VVC Test Model (VTM) 7.0, achieving superior QoE gains over the state-of-the-art research and up to 3.41% over the anchor implementations.

Recently user quality of experience (QoE) is employed in evaluating end user satisfaction in communications systems. Generally, current approaches for QoE assessment are obtrusive, laboratory based and offline. Estimation of user satisfaction in static manner based on mean opinion score is not directly related to instantaneous individual end user contentment. In this paper, based on correlations between user's physiological signals and her/his feelings about the service quality, a non-intrusive and user centric QoE assessment system for voice communications is developed. The findings of this study indicate that the emotional patterns in response to the changes in channel quality can be adapted to estimate the level of satisfaction in a QoE assessment system in a live manner. Based on experimental results, two categories of users are identified: sensitive and insensitive towards quality degradations. The results indicate that for the sensitive users, our non-intrusive subjective quality assessment method outperforms ITU-T P.563 standard with respect to root mean square error; while, the results are much better among the insensitive users.

The growing demand on video streaming services is the main motive power for video quality assessment. Due to the rapid development of video streaming services, customers are more enjoying higher quality videos, and their perceptions on video service will directly influence the service provider's performance, which makes it significant to study end-users' subjective perception, named as Quality of Experience (QoE), on video streaming for both service providers and end-users. In this paper, a QoE evaluation model is proposed to predict the end-users' perception on video streaming service considering different video content types. This QoE model, named as Video-Mean Opinion Score (VMOS) model, directly focuses on end-users' feeling. Thus, the Key Performance Indicators (KPIs), which can be straightly felt by end-users, are mapped to QoE score without considering the network parameters. The excellent performance of VMOS model for QoE evaluation on video quality has been verified via plenty subjective Mean Opinion Score (MOS) test, which includes 180 video samples with 1280 valid votes. The Pearson Correlation Coefficient between VMOS score and MOS is as high as 0.925, which indicates that this model can evaluate users' perception on video quality with almost the same accuracy as subjective test.

Video streaming over MANET (Mobile Ad-hoc Network) is challenging due to specific features that characterize this network. The importance of MANET-specific features has been already emphasized in terms of QoS (Quality of Service). Nevertheless, these parameters have been considered to a limited extent in terms of QoE (Quality of Experience), and thereby, they still remain not well understood. Therefore, this research has been motivated by the challenge to provide a deeper understanding of MANET-specific parameters and their impact on QoE for video streaming. Experimental study has included two phases, i.e., video streaming emulation over MANET and end-user survey to collect MOS (Mean Opinion Score) rates. Results obtained from the statistical analysis imply that there exists strong and statistically significant impact of individual MANET-specific parameters and their interaction on QoE for video streaming.

With the soaring demands for high speed data communication, as well as transmission of various types of services with different requirements over cellular networks, having a decent radio resource management is considered vital in Long Term Evolution (LTE) system. In particular, satisfying the quality of service (QoS) requirements of different applications is one of the key challenges of radio resource management that needs to be dealt by the LTE system. In this paper, we propose a cross-layer design scheme that jointly optimizes three different layers of wireless protocol stack, namely application, Medium Access Control, and physical layer. The cross-layer optimization framework provides efficient allocation of wireless resources across different types of applications (i.e., real-time and non real-time) run by different users to maximize network resource utilization and user-perceived QoS, or also known as Quality of Experience (QoE). Here, Mean Opinion Score is used as a unified QoE metric that indicates the user-perceived quality for real-time or multimedia services notably video applications. Along with multimedia services, the proposed framework also takes care of non-real-time traffic by ensuring certain level of fairness. Our simulation, applied to scenarios where users simultaneously run different types of applications, confirms that the proposed QoE-oriented cross-layer framework leads to significant improvement in terms of maximizing user-perceived quality as well as maintaining fairness among users.

Mobile video traffic is dominant in cellular and enterprise wireless networks. With the advent of myriads of applications from video telephony and streaming to virtual reality, network administrators face the challenge to provide high quality of experience (QoE) in the face of diverse wireless conditions and application contents. Yet, state-of-the-art networks lack analytics for QoE, as this requires support from the application or user feedback. While there are existing techniques to map quality of service (QoS) to QoE by training machine learning (ML) models without requiring user feedback, these techniques are limited to only few applications (e.g., Skype), due to insufficient QoE ground-truth annotation for ML. To address these limitations, we focus on video telephony applications and model key artefacts of spatial and temporal video QoE. Our key contribution is designing content- and device-independent metrics and training across diverse WiFi conditions. We show that our metrics achieve a median 90% accuracy by comparing with mean-opinion-score (MOS) from more than 200 users and 800 video samples. Our content-independent metrics significantly reduce the MOS prediction error of previous works and are validated over three popular video telephony applications – Skype, FaceTime and Google Hangouts.

As the demand for mobile services grows exponentially, the focus on wireless network optimization has been changed from Quality of Service (QoS) for the network to Quality of Experience (QoE) for users. The network optimization research about QoS in the past cannot surely meet the requirements of the users' QoE. Therefore, in this paper, a handover solution is proposed to improve the QoE while considering the QoE balance for a LTE network that provides different services. Compared with other optimization of QoE nowadays, the proposed solution balances and improves the QoE of users. The proposed solution controls handover parameter by running a handover optimization algorithm based on the dynamic particle swarm optimization (DPSO) in a central controller, which finally optimizes the overall QoE and reduces the proportion of users with extremely poor QoE. The simulation results show that the DPSO algorithm ensures the quality of the solution and increases the speed of convergence by nearly twice that of the standard particle swarm optimization algorithm (SPSO). After adopting the DPSO handover solution, the overall QoE of users is increased by 6.22 % and 4.59 % while the variance of users' QoE is decreased by 14.2 % and 22.6 %, compared with the full handover solution and the traditional handover solution respectively. The number of users with QoE less than 1.9 is reduced to 0 with the proposed solution.

Crowdsourcing has emerged in recent years as a potential strategy to enlist the general public to solve a wide variety of tasks. With the advent of ubiquitous Internet access, it is now feasible to ask an Internet crowd to conduct QoE (Quality of Experience) experiments on their personal computers in their own residences rather than in a laboratory. The considerable size of the Internet crowd allows researchers to crowdsource their experiments to a more diverse set of participant pool at a relatively low economic cost. However, as participants carry out experiments without supervision, the uncertainty of the quality of their experiment results is a challenging problem. In this paper, we propose a crowdsourceable framework to quantify the QoE of multimedia content. To overcome the aforementioned quality problem, we employ a paired comparison method in our framework. The advantages of our framework are: 1) trustworthiness due to the support for cheat detection; 2) a simpler rating procedure than that of the commonly-used but more difficult mean opinion score (MOS), which places less burden on participants; 3) economic feasibility since reliable QoE measures can be acquired with less effort compared with MOS; and 4) generalizability across a variety of multimedia content. We demonstrate the effectiveness and efficiency of the proposed framework by a comparison with MOS. Moreover, the results of four case studies support our assertion that the framework can provide reliable QoE evaluation at a lower cost.