QuRate

In recent years, with the rapid development of virtual reality (VR), 360-degree panoramic video has grown up to become a popular Internet service. Streaming 360-degree video on today's Internet is challenging due to its ultra-high bandwidth and ultra-low motion-to-photon latency requirements. Although some VR solutions use tiling technology and Scalable Video Coding(SVC) encoding for 360-degree video transmission to reduce bandwidth consumption, it is still being challenging to transmit 360-degree video streams from remote content servers due to network latency. Caching popular video content at the edge of the network can decrease network latency and bandwidth consumption by reducing the number of future requests that have to be sent all the way to remote content servers. Because of the limited cache size, it is essential to decide which content to cache and how to replace the cache when the cache size is insufficient. In this paper, we propose an edge cache replacement strategy named Size-Popularity-Layer-FoV strategy(SPLF) for tile-based 360-degree video streaming using SVC. To improve caching performance, every video chunks' cache value is estimated according to its popularity, size, SVC layer and whether it is in FoV. Simulation results manifest that the proposed strategy can not only improve the hit rate and video quality compared to various baseline algorithms, but also decrease the playback stalls and the duration of rebuffering.

In this study, the perceptual video quality (PVQ) for the vogue 360-degree video with compression artefacts viewed on the visual reality (VR) head-mounted display (HMD) is obtained for the first time by an elaborately designed subjective experiment. The characteristic of the PVQ of omnidirectional (360-degree) video on the VR HMD and PC monitor is then investigated. The PVQ on the HMD is found to be linearly related to the video coding quality (VCQ) on the PC monitor. A quality evaluation model is then proposed based on the mapping formula and a new assessment parameter, where the impact of the video resolution and display device is involved. In this way, the traditional video quality assessment metrics for two-dimensional video can be extended to assess the PVQ of the 360-degree video viewed on the HMD. At last, the video structural similarity metric is taken as an example to predict the input of the proposed model, i.e. the VCQ of the 360-degree video. Experimental results demonstrate that the proposed model can effectively and conveniently solve the challenge of PVQ assessment for the 360-degree video with compression distortions on the HMD.

Currently, we witness dramatically increasing interest in immersive media technologies like Virtual Reality (VR), particularly in omnidirectional video (OV) streaming. Omnidirectional (also called 360-degree) videos are panoramic spherical videos in which the user can look around during playback and which therefore can be understood as hybrids between traditional movie streaming and interactive VR worlds. Unfortunately, streaming this kind of content is extremely bandwidth intensive (compared to traditional 2D video) and therefore, Quality of Experience (QoE) tends to deteriorate significantly in absence of continuous optimal bandwidth conditions. In this paper, we present a first approach towards subjective QoE assessment for omnidirectional video (OV) streaming. We present the results of a lab study on the QoE impact of stalling in the context of OV streaming using head-mounted displays (HMDs). Our findings show that subjective testing for immersive media like OV is not trivial, with even simple cases like stalling leading to unexpected results. After a discussion of characteristic pitfalls and lessons learned, we provide a a set of recommendations for upcoming OV assessment studies.

Virtual reality and 360-degree video streaming are growing rapidly; however, streaming 360-degree video is very challenging due to high bandwidth requirements. To address this problem, the video quality is adjusted according to the user viewport prediction. High quality video is only streamed for the user viewport, reducing the overall bandwidth consumption. Existing solutions use shallow buffers limited by the accuracy of viewport prediction. Therefore, playback is prone to video freezes which are very destructive for the Quality of Experience (QoE). We propose using layered encoding for 360-degree video to improve QoE by reducing the probability of video freezes and the latency of response to the user head movements. Moreover, this scheme reduces the storage requirements significantly and improves in-network cache performance.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

In this paper, we describe a new telepresence system which enables a user to look around a virtualized real world easily in network environments. The proposed system includes omni-directional video viewers on web browsers and allows the user to look around the omni-directional video contents on the web browsers. The omni-directional video viewer is implemented as an Active-X program so that the user can install the viewer automatically only by opening the web site which contains the omni-directional video contents. The system allows many users at different sites to look around the scene just like an interactive TV using a multi-cast protocol without increasing the network traffic. This paper describes the implemented system and the experiments using live and stored video streams. In the experiment with stored video streams, the system uses an omni-directional multi-camera system for video capturing. We can look around high resolution and high quality video contents. In the experiment with live video streams, a car-mounted omni-directional camera acquires omni-directional video streams surrounding the car, running in an outdoor environment. The acquired video streams are transferred to the remote site through the wireless and wired network using multi-cast protocol. We can see the live video contents freely in arbitrary direction. In the both experiments, we have implemented a view-dependent presentation with a head-mounted display (HMD) and a gyro sensor for realizing more rich presence.

Three hundred and sixty degree video is becoming more and more popular on the Internet. By using a Head-Mounted Display, 360-degree video can render a Virtual Reality (VR) environment. However, it is still a big challenge to understand Quality of Experience (QoE) of 360-degree video since user experience during watching 360-degree video is a very complex phenomenon. In this paper, we aim to investigate four QoE aspects of 360-degree video, namely, perceptual quality, presence, cybersickness, and acceptability. In addition, four key QoE-affecting factors of encoding parameters, content motion, rendering device, and rendering mode are considered in our study. To the best of our knowledge, this is the first work that covers a large number of factors and QoE aspects of 360-degree video. In this study, a subjective experiment is conducted using 60 video versions generated from three original 360-degree videos. Based on statistical analysis of the obtained results, various findings on the impacts of the factors on the QoE aspects are provided. In particular, regarding the impacts of encoding parameters, it is found that the difference of QoE is negligible between video versions encoded at 4 K and 2.5 K resolutions. Also, it is suggested that 360-degree video should not be encoded at HD resolution or lower when watching in VR mode using Head Mounted Display. In addition, the bitrate for good QoE varies widely across different video contents. With respect to the content motion factor, its impact is statistically significant on the perceptual quality, presence, and cybersickness. In a comparison of two rendering device sets used in this study, there is no statistically significant difference found for the acceptability and cybersickness. However, the differences of the perceptual quality and presence are indicated to be statistically significant. Regarding the rendering mode, a comparison between VR and non-VR modes is also conducted. Although the non-VR mode always achieves higher perceptual quality scores and higher acceptability rates, more than a half of the viewers prefer the VR mode to the non-VR mode when watching versions encoded at the resolutions of fHD or higher. By contrast, the non-VR mode is preferred at the HD resolution.

360-degree video streaming is an important component of virtual reality. A video streaming server stores video streams with various quality levels. These streams are fragmented into several video segments. In 360-degree video streaming, these segments are tiled into video tiles. In general, the client changes the viewport quality level based on predictions of the user’s viewport and network throughput. In named data networking (NDN), the throughput measured at the client is not always stable due to network-inherent caching. Therefore, the viewport video quality changes (referred to as quality switching) frequently in 360-degree video streaming over NDN. In this paper, we propose buffer-based adaptation using scalable video coding for 360-degree video streaming over NDN. In the proposed method, a buffer-based adaptation algorithm and the prefetching of video tiles are used to avoid video quality switching. A simple performance evaluation using computer simulations shows that the proposed method reduces the quality switching compared with that for a rate-based adaptation algorithm.

The rapid growth of user expectations and network technologies has proliferated the service needs of 360-degree video streaming. In the light of the unprecedented bitrates required to deliver entire 360-degree videos, tile-based streaming, which associates viewport and non-viewport tiles with different qualities, has emerged as a promising way to facilitate 360-degree video streaming in practice. Existing work on viewport prediction primarily targets prediction accuracy, which potentially gives rise to excessive computational overhead and latency. In this paper, we propose a sinusoidal viewport prediction (SVP) system for 360-degree video streaming to overcome the aforementioned issues. In particular, the SVP system leverages 1) sinusoidal values of rotation angles to predict orientation, 2) the relationship between prediction errors, prediction time window and head movement velocities to improve the prediction accuracy, and 3) the normalized viewing probabilities of tiles to further improve adaptive bitrate (ABR) streaming performance. To evaluate the performance of the SVP system, we conduct extensive simulations based on real-world datasets. Simulation results demonstrate that the SVP system outperforms state-of-the-art schemes under various buffer thresholds and bandwidth settings in terms of viewport prediction accuracy and video quality, revealing its applicability to both live and video-on-demand streaming in practical scenarios.

There is a growing interest nowadays of using immersive technologies to promote Cultural Heritage (CH), engage and educate visitors, tourists and citizens. Such examples refer mainly to the use of Virtual Reality (VR) technology or focus on the enhancement of the real world by superimposing digital artefacts, so called Augmented Reality (AR) applications. A new medium that has been introduced lately as an innovative form of experiencing immersion is the 360-degree video, imposing further research challenges. This paper presents a study on the creation of an immersive application that is based on 360-degree video and targets the users’ education in the importance of CH. The user is engaged in a journey where they learn about preserved artefacts in the city of Rethymno, Greece and their historical value in a playful way. The integrated architecture and development steps for creating such applications that are based on 360- egree video are demonstrated. The overall design process is driven by the goal to increase the level of immersion offered by incorporating game elements in a realistic storytelling experience. The final application can be experienced with the use of a VR low-cost device such as the Google cardboard and a compatible mobile phone. The study concludes with a set of recommendations and future directions for offering more immersive experiences in the CH sector.

Wireless 360-degree video delivery provides virtual reality (VR) immersive experience where each user can freely switch his/her viewing orientation. The existing schemes of the wireless 360-degree video streaming use digital-based compression and transmission. However, they have many disadvantages in terms of video traffic and quality: cliff effect due to unstable wireless channels, large video traffic due to the extremely high resolution of the 360-degree video, and the perceptual redundancy within the transmitted video. To solve the above problems, this paper proposes a novel wireless 360-degree video transmission scheme called 360Cast. 360Cast adopts the analog-based transmission, including power allocation and analog modulation, to achieve graceful video quality improvement even in time-varying wireless channels. Here, the power allocation considers the distortion of human perception and sphere-to-plane projection to maximize the human perceptual quality at the 360-degree video playback. In addition, 360Cast predicts and transmits the user's viewport based on the recent HMD user's orientation by using dynamic linear regression (DLR) and only transmits the viewport for traffic reduction. Evaluation results show that the proposed 360Cast provides better human perceptual quality via HMD compared with the existing analog transmission schemes, i.e., SoftCast and FoveaCast, by using the integration of viewport prediction, power allocation, and analog modulation.

The increasing popularity of virtual reality and 360-degree video streaming has created a demand for streaming high-quality video content over cellular networks. However, streaming 360-degree videos over cellular networks poses a challenge due to the need for ultra-low latency and high bandwidth. Although viewport adaptive streaming presents a promising solution to reduce the bandwidth requirement of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$360^{\circ }$</tex-math></inline-formula> video, it remains challenging to predict the Field of View (FoV) due to the adverse impact of prediction errors on the user's Quality of Experience (QoE). Moreover, bitrate adaptation gets complicated due to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$360^{\circ }$</tex-math></inline-formula> videos and the dynamic nature of the cellular network. Therefore, in this paper, we propose an intelligent framework, termed Seer, to fully realize <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$360^{\circ }$</tex-math></inline-formula> video streaming over cellular networks by leveraging Multi-access Edge Computing (MEC). It incorporates an FoV prediction scheme and a bitrate adaptation policy into the MEC platform. In particular, the FoV prediction scheme uses perceived video features coupled with time-series based viewport prediction to achieve higher prediction accuracy. For an effective bitrate selection in cellular network conditions, a Reinforcement Learning (RL) based bitrate adaptation policy is further employed. The results demonstrate that Seer's FoV prediction scheme delivers higher prediction accuracy, while the streaming framework significantly reduces the bandwidth requirements and improves the user's QoE.

360-degree video is an affordable and easy-to-use technology for social science research. It holds significant potential for capturing spatio-temporal aspects of the social world from a fully omni-directional spatial perspective; however, gaps remain as to how it can be used to support field-based data collection and analysis. In this short piece we offer two contributions to the literature on 360-degree video for qualitative social science research on place. First, we draw on evidence from our multi-city study of ‘urban platform temporalities’ to develop a step-by-step procedure for producing and analyzing 360-degree digital video datasets, demonstrating the potential of the technology for what we term whole scene capture . We provide practical advice on software, hardware, camera usage, video processing, and ethical considerations; and introduce the 360-video qualitative coding technique of spherical simultaneous perspective . Adding new evidence of its use to already established literatures on 360-degree immersive video ethnographies and virtual human-environment exposure research, our method for systematic 360-degree capture of spatio-temporal data is applicable to a range of social science studies with a field-based data collection component. Finally, drawing together technological understandings of immersion from the field of VR with its ethnographic meaning, we then articulate the notion of immersive holism as a quality of 360-degree video that enables deep, meaningful, and comprehensive knowledge of place.

With Virtual Reality (VR) devices and content getting increasingly popular, understanding user behaviors in virtual environment is important for not only VR product design but also user experience improvement. In VR applications, the head movement is one of the most important user behaviors, which can reflect a user's visual attention, preference, and even unique motion pattern. However, to the best of our knowledge, no dataset containing this information is publicly available. In this paper, we present a head tracking dataset composed of 48 users (24 males and 24 females) watching 18 sphere videos from 5 categories. We carefully record how users watch the videos, how their heads move in each session, what directions they focus, and what content they can remember after each session. Based on this dataset, we show that people share certain common patterns in VR spherical video streaming, which are different from conventional video streaming. We believe the dataset can serve good resource for exploring user behavior patterns in VR applications.

Recently, virtual reality (VR) has become popular in many fields. Users can view a 360-degree video in all directions and have an immersive experience with a VR device. However, the transmission of a 360-degree video, which is compared to a traditional video, is a significant burden on wireless networks. In other words, users may suffer poor quality of experience (QoE) under constrained power. In order to improve transmission efficiency and ensure the QoE of users, in this paper, we propose an adaptive transmission strategy (ATS) for resource allocation in 360-degree videos to multiple users using a non-orthogonal multiple access (NOMA) scheme. In ATS, the delivery mode (multicast or unicast) of tiles is based on the users' field of views (FoVs) and their channel conditions, and the quality of tiles is dynamically determined by their priorities. The simulation results show that ATS significantly improves the QoE of users under constrained power.