High Dynamic Range versus Standard Dynamic Range compression efficiency

HDR video quality assessment: Perceptual evaluation of compressed HDR video

The migration from High Definition (HD) TV to Ultra High Definition (UHD) is already underway. In addition to an increase of picture spatial resolution, UHD will bring more color and higher contrast by introducing Wide Color Gamut (WCG) and High Dynamic Range (HDR) video. As both Standard Dynamic Range (SDR) and HDR devices will coexist in the ecosystem, the transition from Standard Dynamic Range (SDR) to HDR will require distribution solutions supporting some level of backward compatibility. This paper presents a new HDR content distribution scheme, named SL-HDR1, using a single layer codec design and providing SDR compatibility. The solution is based on a pre-encoding HDR-to-SDR conversion, generating a backward compatible SDR video, with side dynamic metadata. The resulting SDR video is then compressed, distributed and decoded using standard-compliant decoders (e.g. HEVC Main 10 compliant). The decoded SDR video can be directly rendered on SDR displays without adaptation. Dynamic metadata of limited size are generated by the pre-processing and used to reconstruct the HDR signal from the decoded SDR video, using a post-processing that is the functional inverse of the pre-processing. Both HDR quality and artistic intent are preserved. Pre- and post-processing are applied independently per picture, do not involve any inter-pixel dependency, and are codec agnostic. Compression performance, and SDR quality are shown to be solidly improved compared to the non-backward and backward-compatible approaches, respectively using the Perceptual Quantization (PQ) and Hybrid Log Gamma (HLG) Opto-Electronic Transfer Functions (OETF).

Tone-mapping-operators (TMOs) provide a useful means for converting high dynamic range (HDR) images to low dynamic range (LDR) images so that they can be viewed on standard displays, but this may influence the visual quality of experience (QoE) of the end-user. There is a need to understand the impact of TMOs to inform the choice of TMO algorithms for different displays, especially for small-screen-devices (SSDs) such as those used in mobile phones. This is important, as mobile devices are becoming the primary means of consuming multimedia contents. However, few studies have been undertaken to assess the impact of TMOs and viewing devices (especially SSDs) on the visual QoE of the user when using. In this chapter, we evaluate subjectively and objectively, the commonly used TMOs in different displays and resolutions for colour and grey-scale HDR images. Our results show that viewing devices have an influence on the TMOs performance, suggesting the need for a careful choice of TMO to enhance the viewing-QoE of the end-user. As expected, the higher resolution, the better HDR-image quality. Surprisingly, there was no significant difference between the Mean of Opinion Score (MOS) scores for colour and grey-scale images in SSDs. The device and TMOs affect QoE for colour and grey HDR-image equally. We found Shannon entropy (SE) to be a good objective measure of quality for colour and grey HDR images, suggesting that entropy may find use in automated HDR quality control assessment schemes, while; HDR-VDP-2 is a good objective measure for colour HDR image only.

The new generation of organic light emitting diode display is designed to enable the high dynamic range (HDR), going beyond the standard dynamic range (SDR) supported by the traditional display devices. However, a large quantity of videos are still of SDR format. Further, most pre-existing videos are compressed at varying degrees for minimizing the storage and traffic flow demands. To enable movie-going experience on new generation devices, converting the compressed SDR videos to the HDR format (i.e., compressed-SDR to HDR conversion) is in great demands. The key challenge with this new problem is how to solve the intrinsic many-to-many mapping issue. However, without constraining the solution space or simply imitating the inverse camera imaging pipeline in stages, existing SDR-to-HDR methods can not formulate the HDR video generation process explicitly. Besides, they ignore the fact that videos are often compressed. To address these challenges, in this work we propose a novel imaging knowledge-inspired parallel networks (termed as KPNet) for compressed-SDR to HDR (CSDR-to-HDR) video reconstruction. KPNet has two key designs: Knowledge-Inspired Block (KIB) and Information Fusion Module (IFM). Concretely, mathematically formulated using some priors with compressed videos, our conversion from a CSDR-to-HDR video reconstruction is conceptually divided into four synergistic parts: reducing compression artifacts, recovering missing details, adjusting imaging parameters, and reducing image noise. We approximate this process by a compact KIB. To capture richer details, we learn HDR representations with a set of KIBs connected in parallel and fused with the IFM. Extensive evaluations show that our KPNet achieves superior performance over the state-of-the-art methods.

High dynamic range (HDR) imaging technology has been widely implemented in digital microscopes for taking still images of high contrast specimens. However, capturing HDR microscopic video is much more challenging. In this paper, we present a HDR microscopic video system based on GPU accelerated computing. We show that by combining CPU and GPU computing, it is possible to build a stable HDR video system using a single off the shelf camera. We show that capturing multiple frames of different exposure intervals, aligning consecutive neighboring frames, constructing HDR radiance map and tone mapping the radiance map for display, can be realized by using GPU computing to accelerate the processing speed. We present experimental results to show the effectiveness of our system and how HDR video can reveal much more details than conventional videos.

Evaluating the quality of high dynamic range (HDR) images has emerged as a challenging and contemporary topic with the proliferation of HDR content. In this work, a new HDR compression (HDRC) database is proposed, aiming to provide a benchmark for the development of full-reference HDR image quality assessment (IQA) algorithms when facing the latest HDR compression distortions. In particular, the proposed HDRC database is the first HDR-IQA database to incorporate Versatile Video Coding (VVC) compression distortions, closely associated with practical application scenarios. Furthermore, the proposed HDRC database is currently the largest HDR-IQA database, including 80 reference images and 400 distorted images. Extensive experiments are conducted by studying the performance compared to existing HDR-IQA databases when evaluating three HDR-specific IQA models and nine IQA models prevalent for low dynamic range (LDR) content, revealing the challenges the proposed HDRC database brings. The results indicate that the existing IQA models demonstrate noticeable decreases in accuracy when assessing new compression distortions, underscoring the need for the development of novel HDR-IQA models. Consequently, the suggested HDRC database can serve as a potential database for HDR-IQA research, fostering a comprehensive exploration of the associated fields.

There are an increasing number of databases describing subjective quality responses for HDR (high dynamic range) imagery with various distortions. The dominant distortions across the databases are those that arise from video compression, which are primarily perceived as achromatic, but there are some chromatic distortions due to 422 and other chromatic sub-sampling. Tone mapping from the source HDR levels to various levels of reduced capability SDR (standard dynamic range) are also included in these databases. While most of these distortions are achromatic, tone-mapping can cause changes in saturation and hue angle when saturated colors are in the upper hull of the of the color space. In addition, there is one database that specifically looked at color distortions in an HDR-WCG (wide color gamut) space. From these databases we can test the improvements to well-known quality metrics if they are applied in the newly developed color perceptual spaces (i.e., representations) specifically designed for HDR and WCG. We present results from testing these subjective quality databases to computed quality using the new color spaces of Jzazbz and ICTCP, as well as the commonly used SDR color space of CIELAB.

High Dynamic Range (HDR) imaging captures a wide range of luminance existing in real-world scenes. Due to large luminance levels and higher brightness of HDR displays, artefacts can be more noticeable to the Human Visual System (HVS). In a first attempt to experimentally quantify those noticeable levels for HDR images, we pioneered in conducting an exhaustive and comprehensive Just Noticeable Difference (JND) subjective experiment of which the outcome is presented in this paper. Six distortions including JPEG, JPEG2000, noise, blur, contrast change, and quantization artefacts have been considered in the test. The distortions were applied to 10 HDR images in 100 distortion levels resulting a database of 6000 HDR test images. The subjects were asked to find the image JND location on each set of 100 images they had the freedom to explore. The effect of content features on the noticeable threshold selection is investigated per distortion type. Our results in some cases show a significant correlation between content features and JNDs. We are hoping that our results can contribute to further exploitation of a precise HVS model for HDR quality assessment and optimization of the coding and bit allocation in HDR compression.

Rate-distortion optimized layered coding of high dynamic range videos

High dynamic range (HDR) video compression has until now been approached by using the high profile of existing state-of-the-art H.264/AVC (Advanced Video Coding) codec or by separately encoding low dynamic range (LDR) video and the residue resulted from the estimation of HDR video from LDR video. Although the latter approach has a distinctive advantage of providing backward compatibility to 8-bit LDR displays, the superiority of one approach to the other in terms of the rate distortion trade-off has not been verified yet. In this paper, we first give a detailed overview of the methods in these two approaches. Then, we experimentally compare two approaches with respect to different objective and perceptual metrics, such as HDR mean square error (HDR MSE), perceptually uniform peak signal to noise ratio (PU PSNR) and HDR visible difference predictor (HDR VDP). We first conclude that the optimized methods for backward compatibility to 8-bit LDR displays are superior to the method designed for high profile encoder both for 8-bit and 12-bit mappings in terms of all metrics. Second, using higher bit-depths with a high profile encoder is giving better rate-distortion performances than employing an 8-bit mapping with an 8-bit encoder for the same method, in particular when the dynamic range of the video sequence is high. Third, rather than encoding of the residue signal in backward compatible methods, changing the quantization step size of the LDR layer encoder would be sufficient to achieve a required quality. In other words, the quality of tone mapping is more important than residue encoding for the performance of HDR image and video coding.

Currently, high dynamic range (HDR) videos with high resolution (HR) have become popular due to the display and the rendered technological advancements. However, making ultra-high definition (UHD) with HDR videos is expensive. The legacy low-resolution (LR) standard dynamic range (SDR) format is still largely used in practice. It is necessary to search for a solution to transform LR SDR videos into UHD HDR format. In this paper, we consider joint super resolution and learning inverse tone mapping an issue of high-frequency reconstruction and local contrast enhancement, and we propose an architecture based on a generative adversarial network to apply joint SR-ITM learning. Specifically, we include the residual ResNeXt block (RRXB) as a basic module to better capture high-frequency textures and adopt YUV interpolation to achieve local contrast enhancement. By adopting a generative adversarial network as a pivotal training mechanism, our designs show advantages in both integration and performance. Our code is now available on GitHub: SR-ITM-GAN.

In this paper, we propose a bit-depth scalable lossless coding method for high dynamic range (HDR) images based on a reversible logarithmic mapping. HDR images are generally expressed as floating-point data, such as in the OpenEXR or RGBE formats. Our bit-depth scalable coding approach outputs base layer data and enhancement layer data. It can reconstruct the low dynamic range (LDR) image from the base layer data and reconstructs the HDR image by adding the enhancement layer data. Most previous two-layer methods have focused on the lossy coding of HDR images. Unfortunately, the extension of previous lossy methods to lossless coding does not significantly compress the enhancement layer data. This is because the bit depth becomes very large, especially for HDR images in floating-point data format. To tackle this problem, we apply a reversible logarithmic mapping to the input HDR data. Moreover, we introduce a format conversion to avoid any degradation in the quality of the reconstructed LDR image. The proposed method is effective for both OpenEXR and RGBE formats. Through a series of experiments, we confirm that the proposed method decreases the volume of compressed data while maintaining the visual quality of the reconstructed LDR images.

High dynamic range (HDR) images are highly desirable and widely applied in our daily life. However, most conventional digital cameras are only able to capture low dynamic range (LDR) images. To produce HDR images, a popular and effective method is to merge multiple LDR images with different exposure settings. To achieve high quality HDR images, determining the exposure settings when capturing the multiple LDR images is of great importance. Though many researchers have paid attention to the exposure determination issue for HDR imaging, most of them have focused on DSLR cameras and surveillance video cameras. Meanwhile, mobile devices nowadays have increasing use but few studies in literature focus on them. Since cameras of mobile devices have some features different from DSLR cameras and surveillance video cameras, many existing exposure determination methods may not be directly applied on them. To fill this research gap, this paper proposes a dynamic exposure determination method for HDR photography on mobile devices, in particular on iOS mobile devices. In the proposed method, three LDR images are captured on iOS devices, which are then merged to generate an HDR result image that can cover more details of the target scene. Considering the time and memory limits of mobile devices, the proposed method analyzes the histogram of the target scene thumbnail to determine the three exposure settings with suitable exposure gaps. Based on our proposed method, we have developed an iOS HDR Camera App. The experimental results demonstrate that the HDR result images of this App are capable to contain more details in both bright area and dark area than those of Apple's default HDR Camera App on iPhone 5s for real-world target scenes.

The genuine viewing quality of High Dynamic Range (HDR) data brought an evolution to content production and display manufacture markets. However, the efficiency of compression and transmission of HDR content still needs to be improved. In this paper we modify the H.264/AVC standard to better characterize HDR content. We propose a new Lagrangian multiplier that strikes a balance between the bit-rate and distortion of the HDR video. The updated Lagrange multiplier was implemented on the H.264/AVC reference software. Our experiment results show that the HDR-VDP2 quality scores of the videos encoded by the HDR-accustomed encoder are higher than the ones encoded with the reference encoder. Moreover, subjective tests confirmed that the visual quality of the compressed HDR videos using our proposed method is higher than the one encoded by the reference codec <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

We conducted a large-scale study of human perceptual quality judgments of High Dynamic Range (HDR) and Standard Dynamic Range (SDR) videos subjected to scaling and compression levels and viewed on three different display devices. While conventional expectations are that HDR quality is better than SDR quality, we have found subject preference of HDR versus SDR depends heavily on the display device, as well as on resolution scaling and bitrate. To study this question, we collected more than 23,000 quality ratings from 67 volunteers who watched 356 videos on OLED, QLED, and LCD televisions, and among many other findings, observed that HDR videos were often rated as lower quality than SDR videos at lower bitrates, particularly when viewed on LCD and QLED displays. Since it is of interest to be able to measure the quality of videos under these scenarios, e.g. to inform decisions regarding scaling, compression, and SDR vs HDR, we tested several well-known full-reference and no-reference video quality models on the new database. Towards advancing progress on this problem, we also developed a novel no-reference model called HDRPatchMAX, that uses a contrast-based analysis of classical and bit-depth features to predict quality more accurately than existing metrics.