Stereoscopic Image Quality Assessment by Considering Binocular Visual Mechanisms

A no-reference stereoscopic image quality assessment method is proposed that takes two primary binocular effects, visual discomfort, and binocular rivalry, into consideration. The adaptive segmentation, local energy, and visual saliency are employed for estimating binocular effects and constructing a quality baseline, which together composes a comprehensive image quality description. Moreover, a compact parallel framework is designed for efficiency. Experiments have validated the high accuracy and robustness of the proposed method.

Existing stereoscopic image quality assessment (SIQA) methods are mostly based on the luminance information, in which color information is not sufficiently considered. Actually, color is part of the important factors that affect human visual perception, and nonnegative matrix factorization (NMF) and manifold learning are in line with human visual perception. We propose an SIQA method based on learning binocular manifold color visual properties. To be more specific, in the training phase, a feature detector is created based on NMF with manifold regularization by considering color information, which not only allows parts-based manifold representation of an image, but also manifests localized color visual properties. In the quality estimation phase, visually important regions are selected by considering different human visual attention, and feature vectors are extracted by using the feature detector. Then the feature similarity index is calculated and the parts-based manifold color feature energy (PMCFE) for each view is defined based on the color feature vectors. The final quality score is obtained by considering a binocular combination based on PMCFE. The experimental results on LIVE I and LIVE Π 3-D IQA databases demonstrate that the proposed method can achieve much higher consistency with subjective evaluations than the state-of-the-art SIQA methods.

Stereoscopic image quality assessment by learning non-negative matrix factorization-based color visual characteristics and considering binocular interactions

This paper presents a full reference quality assessment metric for stereoscopic images based on perceptual binocular characteristics. To ensure that the predicted 3D quality of experience is as reliable and close as possible to 3D human perception, the proposed stereoscopic image quality assessment (SIQA) method is relying on the cyclopean image. Our approach is motivated by the fact that in case of asymmetric quality, 3D perception mechanisms place more emphasis on the view providing the most important and contrasted information. We integrated this psychophysical findings in the proposed 3D-IQA framework thanks to a weighting factor based on local information content. Add to that, to take into account the disparity/depth masking effect, we modulate the obtained quality score of each pixel of the cyclopean image according to its location in the scene. Experimental results show that the proposed metric correlates better with human judgement than the state-of-the-art metrics.

The degradation in quality of stereoscopic image was evaluated subjectively as part of an effort to develop a method for evaluating the quality of coded stereoscopic color images. The mean opinion score was analyzed to determine the effect of the attentive eye from the viewpoint of stereoscopic vision. The result of an evaluation experiment shows that the effect of the attentive eye was not significant for coded stereoscopic image. Moreover, the overall quality of the stereoscopic image was degraded when the qualities of the left and right images differed. This means that the overall quality of the coded stereoscopic image was not evaluated by the average value of qualities of the right image and the left image. Finally, the overall quality of the coded stereoscopic image was evaluated synthetically both of the block noise of the background portion which has no disparity information and the distortion of the stereoscopic edge portion of the object.

In contemporary society full of stereoscopic images, how to assess visual quality of 3D images has attracted an increasing attention in field of Stereoscopic Image Quality Assessment (SIQA). Compared with 2D-IQA, SIQA is more challenging because some complicated features of Human Visual System (HVS), such as binocular interaction and binocular fusion, must be considered. In this paper, considering both binocular interaction and fusion mechanisms of the HVS, a hierarchical no-reference stereoscopic image quality assessment network (StereoIF-Net) is proposed to simulate the whole quality perception of 3D visual signals in human cortex, including two key modules: BIM and BFM. In particular, Binocular Interaction Modules (BIMs) are constructed to simulate binocular interaction in V2-V5 visual cortex regions, in which a novel cross convolution is designed to explore the interaction details in each region. In the BIMs, different output channel numbers are designed to imitate various receptive fields in V2-V5. Furthermore, a Binocular Fusion Module (BFM) with automatic learned weights is proposed to model binocular fusion of the HVS in higher cortex layers. The verification experiments are conducted on the LIVE 3D, IVC and Waterloo-IVC SIQA databases and three indices including PLCC, SROCC and RMSE are employed to evaluate the assessment consistency between StereoIF-Net and the HVS. The proposed StereoIF-Net achieves almost the best results compared with advanced SIQA methods. Specifically, the metric values on LIVE 3D, IVC and WIVC-I are the best, and are the second-best on the WIVC-II.

© 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Stereoscopic image quality typically depends on two factors: i) the quality of the luminance image perception, and ii) the quality of depth perception. The effect of distortion on luminance perception and depth perception is usually different, even though depth is estimated from luminance images. Therefore, we present a full reference stereoscopic image quality assessment (FRSIQA) algorithm that rates stereoscopic images in proportion to the quality of individual luminance image perception and the quality of depth perception. The luminance and depth quality is obtained by applying the robust Multiscale-SSIM (MS-SSIM) index on both luminance and disparity maps respectively. We propose a novel multi-scale approach for combining the luminance and depth scores from the left and right images into a single quality score per stereo image. We also explained that a small amount of distortion does not significantly affect depth perception. Further, heavy distortion in stereo pairs will result in significant loss of depth perception. Our algorithm performs competitively over standard databases and is called the 3D-MS-SSIM index.

The quality assessment of stereoscopic images has attracted considerable attention and become an important issue in 3D multimedia applications. The 3D image quality assessment (IQA) encounters many challenges and simple extension of the 2D quality metrics to the 3D case is not satisfying. In this paper, we propose a new perceptual quality assessment scheme for stereoscopic 3D images by considering the local and global visual characteristics. The design of this scheme is motivated by studies on the perception of distorted stereoscopic images. To be more specific, after the log-Gabor filter processing, the local amplitude and phase from the left and right views of the reference and distorted 3D images are utilized as features in local quality evaluation. Meanwhile, the global structure changes of the left and right views are also incorporated into the final quality pooling. The overall 3D quality score is obtained by combining the local and global quality indexes together. The effectiveness of the designed metric is verified on publicly available 3D image quality assessment databases. Experimental results show that the proposed scheme exhibits better performance than other related algorithms in terms of consistency with subjective assessment of stereoscopic 3D images.

Measurement of visual quality is of fundamental importance for numerous stereoscopic image and video processing applications. The effect of distortions on the perceived stereoscopic images quality already been investigated. But few works focus on the weight preference between distortion types. This paper firstly analyzed the weight preference of stereoscopic images between distortion types. The results show that human rating on quality of stereoscopic image will be insensitive, when the quality of left view image keep constant and the right view is Gaussian blurred. Secondly, this paper investigates whether low-pass filtering on JPEG compressed stereoscopic image can enhance the stereoscopic perceptual quality. The results of the experiments show that low-pass filtering may be used to enhance the perceived quality of stereoscopic images in which the right view image is JPEG compressed at an extent of distortion. Furthermore, the enhancement is image content dependent.

As the demand and supply for 3D technologies grows, the quality assessment of stereoscopic content is both demanding and increasingly urgent. This paper proposes a novel 3D images quality assessment (IQA) algorithm which is based on the Binocular Rivalry model and the Summation and Difference model. Both the influence of distorted disparity information on perceptual quality and the binocular rivalry mechanism of HVS are considered in our algorithm. Since disparity map is not utilized in our algorithm, it saves the complex computation of disparity map and improves the robustness of our algorithm. Experiment results shows that our algorithm outperforms well-known 3D IQA metrics on LIVE database and delivers competitive performance on Waterloo database. The cross database validation indicates that our algorithm is stable on different database.

With continuous progress of Internet of Things, multimedia analysis in it has attracted more and more attention. Specially, stereoscopic display technology plays an important role in the multimedia analysis processing. In the Internet of Things system, the quality of stereoscopic image will be reduced in the transmission process. In this mode, it will have a great impact on multimedia analysis to judge whether the quality of stereoscopic image meets the requirements. In this paper, a new no-reference stereoscopic image quality assessment model for multimedia analysis towards Internet of Things is built, which is based on a deep learning model to learn from the class labels and image representations. In our framework, images are represented by natural scene statistics features that are extracted from discrete cosine transform domain, and a regression model is employed to shine upon the quality from the feature vector. The training process of the proposed model contains an unsupervised pretraining phase and a supervised fine-tuning phase, enabling it to generalize over the whole distortion types and severity. The proposed model greatly shows the correlation with subjective assessment as demonstrated by experiments on the LIVE 3-D Image Quality Database and IVC 3-D Image Quality Database.

Three-dimensional (3D) imaging technology has been growingly prevalent in today's world. But objective quality assessment of 3D images is a challenging task. In this paper, we try to investigate and develop the 2D image quality metrics into stereoscopic image quality assessment using a dual-weight model within the concept of free energy principle and human visual system. On the basis of a psychological measure, the free energy is a principle telling where supervises more and attracts human attention. We believe that the "surprise" can account for the binocular rivalry and introduce the inter-weight. Also, the intra-weight is computed according to the physiology of human visual system. We first do the intra-weighted 2D image quality assessment. With inter-weights, we then calculate the combining score of stereopairs. Experiments are performed on the symmetric LIVE3D-I and asymmetric LIVE3D-II databases. Results confirm that the proposed 3D IQA technique, appropriately computing and combining the inter- and intra-weights, is able to faithfully predict the visual quality of stereopairs.

© IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

No-reference stereoscopic image quality assessment using a multi-task CNN and registered distortion representation