An end-to-end Video Quality Metric considering impacts from both encoding and transmission

The focus of this book is on the design of video quality metrics with multi-way data analysis in general and not on the development of a particular metric itself. It is therefore not sensible to design a video quality metric highly optimised for a certain application. Yet in order to assess if multi-way data analysis provides an advantage in the design of video quality, two simple examples for no-reference video quality metrics are introduced in this chapter. They represent the main categories in the engineering approach to video quality metrics: one bitstream-based and one pixel-based video quality metric. The first example describes how a video quality metric can be designed with H.264/AVC bitstream-based no-reference features, suitable for the visual quality estimation of H.264/AVC encoded video sequences. The second example describes a more universal approach, utilizing only pixel-based no-reference features and is thus also suitable for other coding technologies, as the features are selected with respect to generally occurring coding artefacts.

Objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to provide an analysis of 4 objective video quality metrics tested on low bit rate videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metric, modified Watson's digital video quality metric, video structural similarity metric, and the perceptual video quality metric. The test videos consist of H.264 compressed videos with CIF and QCIF frame sizes, at various bit rates (24 kbps - 384 kbps) and frame rates (7.5 fps - 30 fps). Here, results and analysis derived from the comparison of the metrics are provided.

In adaptive video streaming, optimizing the selection of representations for the encoding bitrate ladder has a significant impact on the quality and economics of media delivery. An efficient way to select representations for the bitrate ladder of a given clip is to consider the Satisfied User Ratio (SUR) of the perceived quality of consecutive representations. This ensures that only representations with one Just Noticeable Difference (JND) are encoded and streamed by avoiding encoding similar-quality representations. VMAF (Video Multi-method Assessment Fusion) presently stands as the most commonly utilized quality metric for constructing bitrate ladders. Hence, the precise determination of JND-optimal encoding step-sizes for the VMAF proxy holds paramount importance; nevertheless, this task is intricate and can present considerable challenges.In this paper, we evaluate the effectiveness of different Video Quality Metrics (VQMs) in predicting SUR for the VMAF proxy to better capture content-specific characteristics. Our experimental results provide evidence that incorporating VQMs can improve the precision of the SUR prediction for the VMAF proxy. Compared to a state-of-the-art approach that utilizes video complexity metrics, our proposed approach, which incorporates two quality metrics—specifically, VMAF and SSIM calculated at an optimized quantization parameter (QP)—achieves a substantially reduced Mean Absolute Error (MAE) of 1.67. In contrast, the state-of-the-art approach yields an MAE of 2.01. Hence, we recommend using the above quality metrics to improve the accuracy of the SUR prediction for the VMAF proxy.

Currently, three-dimensional (3D) video is gaining increasing popularity by providing immersive user experience. Compared against conventional 2D video, 3D video excels at bringing a “live” scene closer to the users, and/or trying to place the users in the environment of the displayed content. However, streaming 3D video sequences over the IP networks is challenging due to the impact of dynamic network conditions on user quality. Accurate objective 3D video quality assessment is critical for advanced real-time video streaming adaptation solutions. Most state-of-the-art objective 3D video quality metrics are reference-based and require access to the original 3D video sequences, which is not possible for the real-time applications. This paper proposes the extended No-reference objective 3D Video Quality Metric (eNVQM) for real time 3D video quality assessment. eNVQM establishes a correlation between network packet loss and stereoscopic 3D video quality and was tuned according to extensive subjective testing results. Performance of eNVQM is studied in comparison with two state-of-the-art objective video quality metrics: structural similarity index (SSIM) and video quality metric (VQM).

It is beneficial to have adaptive resolution selection during encoding for the viewer’s optimal experience in a video delivery scenario. For instance, per-title content-adaptive techniques have been exploited for OTT/VOD delivery. An established solution is to build the selection of better resolution on typical video quality metrics. In this context, this paper first introduces measuring the performance of video quality metrics to accurately predict which encoding resolution is subjectively better suited to a particular scene of interest in a video. Then, with a Random Forest (RF) classifier, a novel, subjectively accurate classifier called RF-based fusion metric is proposed to decide which encoding resolution is best suited to compensate for individual metrics’ disparate performance over different quality ranges. The proposed RF classifier encompasses classical video quality metric scores as features and is trained to be closer to subjective experimental results. Performance comparison of this proposed RF-based fusion metric is discussed in comparison to other metrics and subjective experiments.

The use of video streaming services are increasing in the cellular networks, inferring a need to monitor video quality to meet users’ Quality of Experience (QoE). The so-called no-reference (NR) models for estimating video quality metrics mainly rely on packet-header and bitstream information. However, there are situations where the availability of such information is limited due to tighten security and encryption, which necessitates exploration of alternative parameters for conducting video QoE assessment. In this study we collect real-live in-smartphone measurements describing the radio link of the LTE connection while streaming reference videos in uplink. The radio measurements include metrics such as RSSI, RSRP, RSRQ, and CINR. We then use these radio metrics to train a Random Forrest machine learning model against calculated video quality metrics from the reference videos. The aim is to estimate the Mean Opinion Score (MOS), PSNR, Frame delay, Frame skips, and Blurriness. Our result show 94% classification accuracy, and 85% model accuracy (R2 value) when predicting the MOS using regression. Correspondingly, we achieve 89%, 84%, 85%, and 82% classification accuracy when predicting PSNR, Frame delay, Frame Skips, and Blurriness respectively. Further, we achieve 81%, 77%, 79%, and 75% model accuracy (R2 value) regarding the same parameters using regression.

The dominant image and video quality metrics rarely consider the physical specification of video content (luminance, frame rate, resolution) or incorporate models of human vision. In this talk, I will show how a robust video quality metric can be built from fundamental models of low-level human vision and how such a metric can generalize to any display specification (size, resolution, dynamic range) and viewing conditions (ambient light, viewing distance). I will explain how we built a model of contrast sensitivity for achromatic and chromatic spatiotemporal patterns (castleCSF) and combined it with a contrast masking model to create our new video quality metric - ColorVideoVDP. ColorVideoVDP brings many advantages, such as explainability, robustness to unseen distortions, differentiable formulation, and the ability to adapt to the physical characteristics of the content (frame rate, HDR/SDR colour spaces).

While existing video and image quality datasets have extensively studied natural videos and traditional distortions, the perception of synthetic content and modern rendering artifacts remains underexplored. We present a novel video quality dataset focused on distortions introduced by advanced rendering techniques, including neural supersampling, novel‐view synthesis, path tracing, neural denoising, frame interpolation, and variable rate shading. Our evaluations show that existing full‐reference quality metrics perform sub‐optimally on these distortions, with a maximum Pearson correlation of 0.78. Additionally, we find that the feature space of pre‐trained 3D CNNs aligns strongly with human perception of visual quality. We propose CGVQM, a full‐reference video quality metric that significantly outperforms existing metrics while generating both per‐pixel error maps and global quality scores. Our dataset and metric implementation is available at https://github.com/IntelLabs/CGVQM.

We present a low complexity approach for the estimation of the temporal and spatial activity parameters of videos which are captured by a front-facing camera of a vehicle based on context information of the vehicle. The estimated parameters are integrated into an objective video quality metric, which can be used to determine the perceptual quality of a compressed video stream. Our proposed video quality metric has very low computational complexity, which makes it suitable for live video streaming applications. It shows a high Pearson correlation of 0.98 with an average root-mean-square error of 6%, as verified by statistical analysis with data from subjective tests.

Video streaming is a promising multimedia application and is gaining popularity over wireless/mobile communications. The quality of the video depends heavily on the type of content. The aim of the paper is threefold. First, video sequences are classified into groups representing different content types using cluster analysis based on the spatial (edges) and temporal (movement) feature extraction. Second, we conducted experiments to investigate the impact of packet loss on video contents and hence find the threshold in terms of upper, medium and lower quality boundary at which userspsila perception of service quality is acceptable. Finally, to identify the minimum send bitrate to meet quality of service (QoS) requirements (e.g. to reach communication quality with mean opinion score (MOS) greater than 3.5) for the different content types over wireless networks. We tested 12 different video clips reflecting different content types. We chose peak-signal-to-noise-ratio (PSNR) and decodable frame rate (Q) as end-to-end video quality metrics and MPEG4 as the video codec. The work should help optimizing bandwidth allocation for specific content in content delivery networks.

We present a review on human visual system (HVS) based digital video quality metrics. Particularly, three objective video quality metrics are discussed and analyzed in detail because they represent the state-of-the-art of HVS based quality metric research and have been proposed to and verified by VQEG (Video Quality Expert Group) as the candidates of a possible ITU standard. The purpose of the paper is to provide an up-to-date knowledge of the HVS modeling, how the characteristics of the HVS have been incorporated into quality metrics, and the implementation issues of the metrics as well as the directions of future research.

Online monitoring of multimedia networks is required to ensure seamless and ubiquitous delivery of services to the end users. Quality of multimedia content, such as video streams, often gets degraded due to network losses such as packet loss. Frame freezing artifacts are introduced in a video stream when packet loss or packet delay takes place. Estimating the perceptual impact of these artifacts on quality of experience of end users helps service providers to maintain quality of service. In this paper, we have presented a novel no-reference video quality metric, which measures the impact of frame freezing due to packet loss and delay in video streaming networks. The proposed metric is based on several features that directly impact the quality of experience of end users. These features, including motion characteristics of videos, are calculated using the temporal information between video frames and then combined mathematically to form a video quality metric. Different weights are assigned to different features for better performance of the proposed metric. With detailed experiments, we have shown that our method outperforms other contemporary methods in terms of high accuracy and low computation time in frame freeze detection, low root mean square values, high coefficient of determination, and high correlation between subjective and objective measurements. We have used five video databases for our model's evaluation and validation. Furthermore, we have shown that our method is statistically superior to the other models in comparison.

A holistic modeling for QoE estimation in live video streaming applications over LTE Advanced technologies with Full and Non Reference approaches

Objective video quality metrics are an essential part of modern video processing pipelines, guiding video encoding decisions and encoding recipes, helping adaptive bitrate streaming algorithms make smart decisions and providing system-level monitoring capabilities. We will offer a breakdown of an end-to-end such pipeline, highlighting which types of video quality metrics are deployed in each system component and then focus on the single aspect that makes social videos so much different - and one can argue more difficult - to process: their wildly varying and typically inferior source quality. We will then discuss how no-reference video quality metrics have been typically used to measure user-generated video content quality with limited success and make a case for how the video industry can unite and solve this problem at its root.

Clinical relevance YouTube is a widely used medical information source, offering educational content on health conditions and treatments. However, its unregulated nature raises concerns about accuracy, reliability, and accessibility. Assessing online medical content quality is important to ensure that patients receive credible, evidence-based guidance. Background Keratoconus is a progressive corneal disorder that can lead to visual impairment. Corneal cross-linking (CXL) is an effective treatment that strengthens the corneal structure and halts disease progression. YouTube is frequently used by patients seeking information about diseases and treatments. This study aimed to evaluate the quality and reliability of YouTube videos on CXL. Methods A cross-sectional analysis was conducted on YouTube videos related to CXL. Five commonly searched keywords were identified using Google Trends, and a YouTube search was performed. Among the first 100 videos ranked by relevance, 69 met the inclusion criteria. Video characteristics, including source, content type, narrator gender, and accessibility features, were recorded. Quality and reliability were assessed using the DISCERN, Global Quality Score (GQS), and Journal of the American Medical Association (JAMA) scoring systems. Statistical analyses examined differences in video quality based on information source and gender. Results The overall quality of YouTube videos on CXL was average, with a mean DISCERN score of 42.5 ± 11.8, GQS of 3.28 ± 0.94, and JAMA score of 2.75 ± 0.68. Videos uploaded by clinicians (39.1%) and health channels (31.9%) had significantly higher quality scores than patient-uploaded videos (29%) (p < 0.05). No significant correlation was found between video popularity and quality metrics. Narrator gender had no significant effect on video quality or popularity metrics (p > 0.05). Conclusion While clinician- and health channel-produced videos demonstrated higher quality, this did not translate into greater engagement, highlighting a disconnect between popularity and informational value.