Human visual system based objective digital video quality metrics

Objective video quality metrics reported in the literature have so far been focused on TV signals with large frame sizes, full TV frame rates, and high compressed bit rates. The aim of this paper is to provide an analysis of 4 objective video quality metrics tested on low bit rate videos with small frame sizes, various frame rates, and low bit rates. The video quality metrics used were the NTIA video quality metric, modified Watson's digital video quality metric, video structural similarity metric, and the perceptual video quality metric. The test videos consist of H.264 compressed videos with CIF and QCIF frame sizes, at various bit rates (24 kbps - 384 kbps) and frame rates (7.5 fps - 30 fps). Here, results and analysis derived from the comparison of the metrics are provided.

Since 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group.

Currently, three-dimensional (3D) video is gaining increasing popularity by providing immersive user experience. Compared against conventional 2D video, 3D video excels at bringing a “live” scene closer to the users, and/or trying to place the users in the environment of the displayed content. However, streaming 3D video sequences over the IP networks is challenging due to the impact of dynamic network conditions on user quality. Accurate objective 3D video quality assessment is critical for advanced real-time video streaming adaptation solutions. Most state-of-the-art objective 3D video quality metrics are reference-based and require access to the original 3D video sequences, which is not possible for the real-time applications. This paper proposes the extended No-reference objective 3D Video Quality Metric (eNVQM) for real time 3D video quality assessment. eNVQM establishes a correlation between network packet loss and stereoscopic 3D video quality and was tuned according to extensive subjective testing results. Performance of eNVQM is studied in comparison with two state-of-the-art objective video quality metrics: structural similarity index (SSIM) and video quality metric (VQM).

Welcome to the third column on the ACM SIGMM Records from the Video Quality Experts Group (VQEG). The last VQEG plenary meeting took place online from 14 to 18 December. Given the current circumstances, it was organized all online for the second time, with multiple sessions distributed over five to six hours each day allowing remote participation of people from different time zones. About 130 participants from 24 different countries registered to the meeting and could attend the several presentations and discussions that took place in all working groups. This column provides an overview of this meeting, while all the information, minutes, files (including the presented slides), and video recordings from the meeting are available online in the VQEG meeting website. As highlights of interest for the SIGMM community, apart from several interesting presentations of state-of-the-art works, relevant contributions to ITU recommendations related to multimedia quality assessment were reported from various groups (e.g., on adaptive bitrate streaming services, on subjective quality assessment of 360-degree videos, on statistical analysis of quality assessments, on gaming applications, etc.), the new group on quality assessment for health applications was launched, and an interesting session on 5G use cases took place, as well as a workshop dedicated to user testing during Covid-19. In addition, new efforts have been launched related to the research on quality metrics for live media streaming applications, and to provide guidelines on implementing objective video quality metrics (ahead of PSNR) to the video compression community. We encourage those readers interested in any of the activities going on in the working groups to check their websites and subscribe to the corresponding reflectors, to follow them and get involved.

We present a low complexity approach for the estimation of the temporal and spatial activity parameters of videos which are captured by a front-facing camera of a vehicle based on context information of the vehicle. The estimated parameters are integrated into an objective video quality metric, which can be used to determine the perceptual quality of a compressed video stream. Our proposed video quality metric has very low computational complexity, which makes it suitable for live video streaming applications. It shows a high Pearson correlation of 0.98 with an average root-mean-square error of 6%, as verified by statistical analysis with data from subjective tests.

Objective video quality metrics are an essential part of modern video processing pipelines, guiding video encoding decisions and encoding recipes, helping adaptive bitrate streaming algorithms make smart decisions and providing system-level monitoring capabilities. We will offer a breakdown of an end-to-end such pipeline, highlighting which types of video quality metrics are deployed in each system component and then focus on the single aspect that makes social videos so much different - and one can argue more difficult - to process: their wildly varying and typically inferior source quality. We will then discuss how no-reference video quality metrics have been typically used to measure user-generated video content quality with limited success and make a case for how the video industry can unite and solve this problem at its root.

Chapter 4 - Subjective and objective quality assessment for omnidirectional video

In image and video compression and transmission, it is important to rely on an objective image/video quality metric which accurately represents the subjective quality of processed images and video sequences. In some scenarios, it is also important to evaluate the quality of the received video sequence with minimal reference to the transmitted one. For instance, for quality improvement of video transmission through closed-loop optimisation, the video quality measure can be evaluated at the receiver and provided as feedback information to the system controller. The original image/video sequence--prior to compression and transmission--is not usually available at the receiver side, and it is important to rely at the receiver side on an objective video quality metric that does not need reference or needs minimal reference to the original video sequence. The observation that the human eye is very sensitive to edge and contour information of an image underpins the proposal of our reduced reference (RR) quality metric, which compares edge information between the distorted and the original image. Results highlight that the metric correlates well with subjective observations, also in comparison with commonly used full-reference metrics and with a state-of-the-art RR metric.

Although a lot of progress has been made in the development of 2D objective video quality metrics, the area of 3D video quality metrics is still in its infancy. Many of the proposed metrics are simply adaptations of 2D quality metrics that consider the depth channel as an extra color channel. In this paper, we propose a 3D no-reference objective quality metric that estimates 3D quality taking into account spatial distortions, excessive disparity, depth representation and temporal information of the video. The metric is resolution and frame-rate independent. To estimate the amount of spatial distortion in the video, the proposed metric uses a blockiness metric. The contribution of motion and excessive disparity to 3D quality is calculated using a non-linear relative disparity measure and a frame-rate proportional motion measure. The metric's performance is verified against the COSPAD1 database. The MOS predicted using the proposed metric obtained good correlation values with the subjective scores. The performance was on average better than the performance of two simple 2D full reference metrics: SIMM and PSNR.

A method for deriving an objective video quality metric that emulates the human video quality metric is described. The objective metric comprises a family of measurements that quantify spatial, temporal, luminance, and chrominance distortions, followed by a prediction algorithm. The intent is that the derived metric will work well over a wide range of digital and analog video transmission and storage technologies. After describing the need for these measures, a detailed description of the derivation process and some preliminary results are given. >

International audience

The focus of this book is on the design of video quality metrics with multi-way data analysis in general and not on the development of a particular metric itself. It is therefore not sensible to design a video quality metric highly optimised for a certain application. Yet in order to assess if multi-way data analysis provides an advantage in the design of video quality, two simple examples for no-reference video quality metrics are introduced in this chapter. They represent the main categories in the engineering approach to video quality metrics: one bitstream-based and one pixel-based video quality metric. The first example describes how a video quality metric can be designed with H.264/AVC bitstream-based no-reference features, suitable for the visual quality estimation of H.264/AVC encoded video sequences. The second example describes a more universal approach, utilizing only pixel-based no-reference features and is thus also suitable for other coding technologies, as the features are selected with respect to generally occurring coding artefacts.

For industry, the need to access accurate and reliable objective video metrics has become more pressing with the advent of new video applications and services such as mobile broadcasting, Internet video, and Internet Protocol television (IPTV). Industry-class objective quality- measurement models have a wide range of uses, including equipment testing (e.g., codec evaluation), transmission- planning and network-dimensioning tasks, head-end quality assurance, in- service network monitoring, and client-based quality measurement. The Video Quality Experts Group (VQEG) is the primary forum for validation testing of objective perceptual quality models. The work of VQEG has resulted in International Telecommunication Union (ITU) standardization of objective quality models designed for standard- definition television and for multimedia applications. This article reviews VQEG's work, paying particular attention to the group's approach to validation testing.

Objective video quality metrics can be viewed as myopic expert systems that focus on particular aspects of visual information in video, such as image edges or motion parameters. We conjecture that the combination of many such high-level metrics leads to statistically-significant improvement in the prediction of reference-based perceptual video quality in comparison to each individual metric. To examine this hypothesis in a systematic and rigorous manner, we use: (i) the LIVE and the EPFL/PoliMi databases that provide the difference mean opinion scores (DMOS) for several video sequences under encoding and packet-loss errors; (ii) ten well-known metrics that range from mean-squared error based criteria to sophisticated visual quality estimators; (iii) five variants of regression-based supervised learning. For 400 experimental trials with random (non-overlapping) estimation and prediction subsets taken from both databases, we show that the best of our regression methods: (i) leads to statistically-significant improvement against the best individual metrics for DMOS prediction for more than 97% of the experimental trials; (ii) is statistically-equivalent to the performance of humans rating the video quality for 36.75% of the experiments with the EPFL/PoliMi database. On the contrary, no single metric achieves such statistical equivalence to human raters in any of the experimental trials.

Uploading videos to platforms for sharing has been a core feature of the internet for nearly two decades. Advances in video encoding algorithms, their standardization, and increased computational power have significantly improved the visual quality of online videos. This paper evaluates the video quality across several popular platforms, including YouTube, Facebook, and Vimeo. Using three test sequences with varying content characteristics, the study employs full-reference objective video quality metrics—VMAF, PSNR-HVS-M, and MS-SSIM—to assess video quality at resolutions supported by all platforms. The results provide insights into the comparative performance of each platform in delivering high-quality video