ヒューマンインフォメーション 符号化されたステレオ静止画像における品質評価

Measurement of visual quality is of fundamental importance for numerous stereoscopic image and video processing applications. The effect of distortions on the perceived stereoscopic images quality already been investigated. But few works focus on the weight preference between distortion types. This paper firstly analyzed the weight preference of stereoscopic images between distortion types. The results show that human rating on quality of stereoscopic image will be insensitive, when the quality of left view image keep constant and the right view is Gaussian blurred. Secondly, this paper investigates whether low-pass filtering on JPEG compressed stereoscopic image can enhance the stereoscopic perceptual quality. The results of the experiments show that low-pass filtering may be used to enhance the perceived quality of stereoscopic images in which the right view image is JPEG compressed at an extent of distortion. Furthermore, the enhancement is image content dependent.

© IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

© 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

With the booming of 3-D image processing in the entertainment industry and 3-D multimedia applications todays, the technology for assessing the quality of stereoscopic image faces more challenging tasks than its 2-D counterparts, such as binocular combination, stereo matching, and binocular rivalry. In this paper, a novel stereoscopic image quality assessment method is proposed by jointly exploring binocular fusion and rivalry models. In the quality-aware feature extraction stage, multi-scale binocular combination and binocular rivalry energy responses are first generated from reference and distorted stereopairs. In addition, considering that the changes of luminance affect the quality of stereoscopic image greatly, multi-scale visual features relating to image quality are obtained from its luminance maps as another binocular combination and rivalry features. Then, the dissimilarity of quality-aware features between the reference stereopair and its distorted version is quantified. Finally, such dissimilarities are mapped into an objective score to represent the perceptual quality of stereoscopic image through the pooling strategy of support vector regression. Experiments conducted on LIVE 3-D databases demonstrate that the proposed method achieves 96.61% and 96.03% in terms of Pearson’s linear correlation coefficient on Database Phase I and Phase II, respectively, outperforming most of the state-of-the-art methods.

This paper presents a full reference quality assessment metric for stereoscopic images based on perceptual binocular characteristics. To ensure that the predicted 3D quality of experience is as reliable and close as possible to 3D human perception, the proposed stereoscopic image quality assessment (SIQA) method is relying on the cyclopean image. Our approach is motivated by the fact that in case of asymmetric quality, 3D perception mechanisms place more emphasis on the view providing the most important and contrasted information. We integrated this psychophysical findings in the proposed 3D-IQA framework thanks to a weighting factor based on local information content. Add to that, to take into account the disparity/depth masking effect, we modulate the obtained quality score of each pixel of the cyclopean image according to its location in the scene. Experimental results show that the proposed metric correlates better with human judgement than the state-of-the-art metrics.

We propose a new watermarking algorithm for stereoscopic image tamper detection and self-recovery in three-dimensional multimedia services. Initially, left and right views of stereoscopic image are divided into nonoverlapping 2×2 blocks in order to improve the accuracy of tamper localization in an image. As the left and right views of a stereoscopic image are not independent from each other but have an inter-view relationship, every block of a stereoscopic image is classified into matching block or nonmatching block and then block disparities are obtained. Both matching blocks in the left and right views have similar pixel values, so that fewer bits are allocated for recovery watermark generation, which can increase the quality of watermarked stereoscopic images. A hierarchical tamper-detection strategy with a four-level checkup is presented to improve the accuracy of tamper localization. Additionally, two copies of block (matching block and nonmatching block) information are embedded into the stereoscopic image, and it assures the quality of tampered recovery. For the nonmatching block recovery, two copies of the partner block are embedded into their chaotic mapping blocks, which supply the second chance for tamper recovery. For the matching block recovery, the inter-view relationship between tampers of left and right views supplies the third chance for tamper recovery. Experimental results show that the proposed algorithm can not only detect and locate tampers in stereoscopic image more accurately but also recover the tampered regions better, compared with other algorithms.

To consider the quality of service for stereoscopic image through the network, it is necessary to develop a quality evaluation method for coded stereoscopic image. We propose a quality evaluation model of the coded stereoscopic color image. This evaluation model considers not only the distortions of the edge region and smooth region but also the texture features of the left image. In addition, this model takes into account the disparity information between the left and right images. Instead of the disparity compensated coded image, we employ the JPEG coded image for the subjective assessment test. As the results, the evaluation model is useful for coded stereoscopic image.

Stereoscopic image quality typically depends on two factors: i) the quality of the luminance image perception, and ii) the quality of depth perception. The effect of distortion on luminance perception and depth perception is usually different, even though depth is estimated from luminance images. Therefore, we present a full reference stereoscopic image quality assessment (FRSIQA) algorithm that rates stereoscopic images in proportion to the quality of individual luminance image perception and the quality of depth perception. The luminance and depth quality is obtained by applying the robust Multiscale-SSIM (MS-SSIM) index on both luminance and disparity maps respectively. We propose a novel multi-scale approach for combining the luminance and depth scores from the left and right images into a single quality score per stereo image. We also explained that a small amount of distortion does not significantly affect depth perception. Further, heavy distortion in stereo pairs will result in significant loss of depth perception. Our algorithm performs competitively over standard databases and is called the 3D-MS-SSIM index.

Due to the deficient knowledge of binocular vision properties, how to effectively evaluate stereoscopic images still remains a challenging task. Inspired by multichannel processing of human visual system (HVS), we propose a blind method for stereoscopic image quality assessment (SIQA) by extracting quality related features in sub-bands of the image. First of all, we introduce the shearlet transform to decompose the left- and right-view images into multiple sub-bands content with diverse combinations of scales and orientations, and obtain the combined view based on energy-weighted summation of the corresponding sub-bands of two eye views. Then, natural scene statistics (NSS) of the original left and right images are obtained as quality-sensitive features, followed by extracting NSS features of the sub-bands of left, right and combined views. Moreover, we calculate the gradient similarity between each sub-band pair to denote the asymmetric distortion and disparity information. Finally, all the extracted features are mapped into a quality score by support vector regression (SVR). experimental results on multiple benchmark databases verify the superiority of our method.

Objective quality assessment of distorted stereoscopic images is a challenging problem, especially when the distortions in the left and right views are asymmetric. Existing studies suggest that simply averaging the quality of the left and right views well predicts the quality of symmetrically distorted stereoscopic images, but generates substantial prediction bias when applied to asymmetrically distorted stereoscopic images. In this paper, we first build a database that contains both single-view and symmetrically and asymmetrically distorted stereoscopic images. We then carry out a subjective test, where we find that the quality prediction bias of the asymmetrically distorted images could lean toward opposite directions (overestimate or underestimate), depending on the distortion types and levels. Our subjective test also suggests that eye dominance effect does not have strong impact on the visual quality decisions of stereoscopic images. Furthermore, we develop an information content and divisive normalization-based pooling scheme that improves upon structural similarity in estimating the quality of single-view images. Finally, we propose a binocular rivalry-inspired multi-scale model to predict the quality of stereoscopic images from that of the single-view images. Our results show that the proposed model, without explicitly identifying image distortion types, successfully eliminates the prediction bias, leading to significantly improved quality prediction of the stereoscopic images.

With continuous progress of Internet of Things, multimedia analysis in it has attracted more and more attention. Specially, stereoscopic display technology plays an important role in the multimedia analysis processing. In the Internet of Things system, the quality of stereoscopic image will be reduced in the transmission process. In this mode, it will have a great impact on multimedia analysis to judge whether the quality of stereoscopic image meets the requirements. In this paper, a new no-reference stereoscopic image quality assessment model for multimedia analysis towards Internet of Things is built, which is based on a deep learning model to learn from the class labels and image representations. In our framework, images are represented by natural scene statistics features that are extracted from discrete cosine transform domain, and a regression model is employed to shine upon the quality from the feature vector. The training process of the proposed model contains an unsupervised pretraining phase and a supervised fine-tuning phase, enabling it to generalize over the whole distortion types and severity. The proposed model greatly shows the correlation with subjective assessment as demonstrated by experiments on the LIVE 3-D Image Quality Database and IVC 3-D Image Quality Database.

Presence of depth and compression artifacts have been identified as major factors that influence subjective quality of experience and satisfaction of 3D imagery visualized on a portable auto-stereoscopic display. The juxtaposition between these two factors has been investigated through psychometric subjective experiment, where participants have evaluated the overall quality of stereoscopic images with varying both presences of depth and compression levels. The results reveal that compression artifacts dominate the quality of experience compared to varying depth range.

Chapter 10 - A study on the relationship between depth map quality and stereoscopic image quality using upsampled depth maps

Understanding the characteristics of high-quality stereoscopic 3D (S3D) images has great significance for S3D image classification, quality assessment, and quality enhancement. Existing works assess the quality of an S3D image from a single perspective and use databases with subjective opinion scores obtained by conducting subjective experiments in labs. So the performance of these works in real applications is unclear and questionable. In this paper, we propose an S3D image quality assessment index based on two important factors, namely DisTortion and DisParity (DTDP). Distortion reflects the extent of distortion of each view of an S3D image, respectively. Disparity is a distinguishing factor for an S3D image as compared with a monocular image. We design some features to represent these two factors and use a random forest regression (RFR) to learn the mapping between the features and subjective opinion scores. The database we choose is the NVIDIA 3D VISION LIVE Highest Rated database which is from real application. The images and the subjective opinion scores are directly come from the website viewers all over the world. Experimental results demonstrate a superior performance of the proposed DTDP index as compared to the existing methods.

An adaptive connectionist model was implemented to estimate structure from stereo by associating patterns of activity caused by the covariation of the left and right eye images to the surface structure of the environment. Neurallike elements tuned to different disparities and to different resolutions formed the units of the system. Artificial surfaces were computer synthesized and then illuminated to create left and right eye images. Pairs of image and depth maps were taught to the system by linear association and Widrow-Hoff error correction techniques. Extracting depth information about the environment from 2-D stereo image pairs has generally been considered as a two-part process. First, points on the left and right retinas that correspond to a single point in the world are matched, and second, the depth of the point is computed by the disparity information given by this correspondence. Our implementation implicitly incorporates both steps. The system was tested with stereo pairs that had not previously been learned and was able to reconstruct the depth maps with reasonable accuracy.