A Multiscale Approach to Deep Blind Image Quality Assessment.

Due to the rapid development of multimedia technologies, over the last decades image quality assessment (IQA) has become an important topic. As a consequence, a great research effort has been made to develop computational models that estimate image quality. Among the possible IQA approaches, blind IQA (BIQA) is of fundamental interest as it can be used in most multimedia applications. BIQA techniques measure the perceptual quality of an image without using the reference (or pristine) image. This paper proposes a new BIQA method that uses a combination of texture features and saliency maps of an image. Texture features are extracted from the images using the local binary pattern (LBP) operator at multiple scales. To extract the salient of an image, i.e. the areas of the image that are the main attractors of the viewers' attention, we use computational visual attention models that output saliency maps. These saliency maps can be used as weighting functions for the LBP maps at multiple scales. We propose an operator that produces a combination of multiscale LBP maps and saliency maps, which is called the multiscale salient local binary pattern (MSLBP) operator. To define which is the best model to be used in the proposed operator, we investigate the performance of several saliency models. Experimental results demonstrate that the proposed method is able to estimate the quality of impaired images with a wide variety of distortions. The proposed metric has a better prediction accuracy than state-of-the-art IQA methods.

Quality assessment of images with multiple distortions based on phase congruency and gradient magnitude

In practice, images available to consumers usually undergo several stages of processing including acquisition, compression, transmission, and presentation, and each stage may introduce certain type of distortion. It is common that images are simultaneously distorted by multiple types of distortions. Most existing objective image quality assessment (IQA) methods have been designed to estimate perceived quality of images corrupted by a single image processing stage. In this letter, we propose a no-reference (NR) IQA method to predict the visual quality of multiply-distorted images based on structural degradation. In the proposed method, a novel structural feature is extracted as the gradient-weighted histogram of local binary pattern (LBP) calculated on the gradient map (GWH-GLBP), which is effective to describe the complex degradation pattern introduced by multiple distortions. Extensive experiments conducted on two public multiply-distorted image databases have demonstrated that the proposed GWH-GLBP metric compares favorably with existing full-reference and NR IQA methods in terms of high accordance with human subjective ratings.

In this paper, we propose a new standard haze image database with nearly all kinds of haze situations. Our database includes haze-free images as well as different levels and situations of haze images, such as snowy and extremely serious haze images. Our database also records the related weather and air quality information. Moreover, the database offers a mean opinion score (MOS) for each image as the subjective evaluation of the haze severity. Our database provides ample haze images with various objective and subjective descriptions, and has significantly potential scientific value. Since haze is a reason leading to the degradation of the image quality, obtaining the quality of haze images is also necessary and meaningful. In this paper, a novel no-reference image quality assessment (IQA) method for haze images is also proposed. Our method analyzes the model of haze images to evaluate the quality. The experimental results on the haze database show that our method is consistent with the subjective evaluation.

An investigation of diseases using magnetic resonance (MR) imaging requires automatic image quality assessment methods able to exclude low-quality scans. Such methods can be also employed for an optimization of parameters of imaging systems or evaluation of image processing algorithms. Therefore, in this paper, a novel blind image quality assessment (BIQA) method for the evaluation of MR images is introduced. It is observed that the result of filtering using non-maximum suppression (NMS) strongly depends on the perceptual quality of an input image. Hence, in the method, the image is first processed by the NMS with various levels of acceptable local intensity difference. Then, the quality is efficiently expressed by the entropy of a sequence of extrema numbers obtained with the thresholded NMS. The proposed BIQA approach is compared with ten state-of-the-art techniques on a dataset containing MR images and subjective scores provided by 31 experienced radiologists. The Pearson, Spearman, Kendall correlation coefficients and root mean square error for the method assessing images in the dataset were 0.6741, 0.3540, 0.2428, and 0.5375, respectively. The extensive experimental evaluation of the BIQA methods reveals that the introduced measure outperforms related techniques by a large margin as it correlates better with human scores.

State of the art blind image quality assessment (IQA) methods generally extract perceptual features from the training images, and send them into support vector machine (SVM) to learn the regression model, which could be used to further predict the quality scores of the testing images. However, these methods need complicated training and learning, and the evaluation results are sensitive to image contents and learning strategies. In this paper, two novel blind IQA metrics without training and learning are firstly proposed. The new methods extract perceptual features, i.e., the shape consistency of conditional histograms, from the joint histograms of neighboring divisive normalization transform coefficients of distorted images, and then compare the length attribute of the extracted features with that of the reference images and degraded images in the LIVE database. For the first method, a cluster center is found in the feature attribute space of the natural reference images, and the distance between the feature attribute of the distorted image and the cluster center is adopted as the quality label. The second method utilizes the feature attributes and subjective scores of all the images in the LIVE database to construct a dictionary, and the final quality score is calculated by interpolating the subjective scores of nearby words in the dictionary. Unlike the traditional SVM based blind IQA methods, the proposed metrics have explicit expressions, which reflect the relationships of the perceptual features and the image quality well. Experiment results in the publicly available databases such as LIVE, CSIQ and TID2008 had shown the effectiveness of the proposed methods, and the performances are fairly acceptable.

Screen content image quality assessment based on convolutional neural networks

High Dynamic Range (HDR) imaging captures a wider range of luminance and color than Standard Dynamic Range (SDR), enabling more faithful representations of real-world scenes. Since most displays only support SDR, HDR content must be converted to SDR, often resulting in perceptual distortions like unnatural contrast, loss of detail or color artifacts. This paper evaluates No-Reference Image Quality Assessment (NR-IQA) methods on SDR images derived from HDR using the ESPL-LIVE HDR Image Database, which includes tone-mapped images and images created by multi-exposure fusion, together with associated mean opinion scores (MOS) and standard deviations, providing a reliable ground truth for evaluation. Seven NR-IQA methods, both traditional and Deep Learning (DL)-based, were tested and compared. DL algorithms performed best, with prediction errors below the standard deviation of MOS. Despite moderate correlation values (around 0.5), the results demonstrate the potential of these methods for practical image quality monitoring.

The visual quality of images resulting from Super Resolution (SR) techniques is predicted with blind image quality assessment (BIQA) models trained on a database(s) of human rated distorted images and associated human subjective opinion scores. Such opinion-aware (OA) methods need a large amount of training samples with associated human subjective scores, which are scarce in the field of SR. By contrast, opinion distortion unaware (ODU) methods do not need human subjective scores for training. This paper presents an opinion-unaware BIQA measure of super resolved images based on optimally extracted perceptual features. This set of features was selected using a floating forward search whose objective function is the correlation with human judgment. The proposed BIQA method does not need any distorted images nor subjective quality scores for training, yet the experiments demonstrate its superior quality-prediction performance relative to state-of-the-art opinion-unaware BIQA methods, and that it is competitive to state-of-the-art opinion-aware BIQA methods.

Blind light field image quality assessment by analyzing angular-spatial characteristics

Image quality is related to image content and distortion information. Most learning-based image quality assessment (IQA) methods extract quality-oriented features with auxiliary tasks like detecting the distortion type and level. However, the perceptual quality degradation caused by the same distortion type and level varies substantially for different content in an image. To deal with this problem, in this article, we propose a blind IQA method based on Deep Response fEAture decoMposition and aggregation (DREAM), which considers two factors affecting the image quality simultaneously. First, we use a convolutional neural network (CNN) to extract the basic features from the input image. Second, several parallel fully connected (FC) layers are employed to decompose these basic features into response features related to the image content, distortion type, and distortion level. Third, the graph attention network (GAT) is leveraged to aggregate these response features corresponding to the visual quality. Finally, a regression network is used to predict the quality score. The success of our method lies in the feature decomposition to obtain the response features of different content to a specific distortion in the given distorted image and the quality-oriented features obtained by feature aggregation using the internal relation of these response features. Experimental results indicate that our proposed DREAM achieves state-of-the-art (SOTA) performance on both synthetic and authentic distortion IQA datasets.

In this paper, a universal blind image quality assessment (IQA) algorithm is proposed that works in presence of various distortions. The proposed algorithm is a Blind Image Quality metric based on Wakeby Statistics (BIQWS) which extracts local mean subtraction and contrast normalization (MSCN) coefficients in spatial domain from input image. The MSCN coefficients are used for generating a Wakeby distribution statistical model to extract quality-aware features. The statistical studies indicate that the MSCN coefficients histogram is altered in the presence of various distortions with different severities. These changes are regular and can be used to estimate the type of the distortion and its severity. We extended our previous studies to extract efficient Wakeby distribution model parameters which are more sensitive to changes in MSCN coefficients. These parameters are used to form a quality-aware feature vector. This feature vector is then fed to an SVM (support vector machine) regression model with a nonlinear Kernel to predict the quality score of the input image without any information about the distortion type or reference image. Experimental results show that the image quality index obtained by the proposed method has higher correlation with respect to human perceptual opinions and it is superior in some distortions when compared to some full-reference and other state-of-the-art blind image quality assessment methods.

No-reference image quality assessment (NR-IQA), which aims to predict image quality without accessing to reference images, is a fundamental and challenging problem in the field of image processing. Nevertheless, there are few researches about contrast-distorted images and results of the existing NR-IQA methods which cannot be in accordance with the subjective assessment results further. Therefore, an effective NR-IQA method for contrast-distorted image is proposed in this paper. Firstly, the proposed method extracts five features of all images from the database based on the natural scene statistics (NSS) model. Then the curve fitting method is subsequently represented to calculate values of natural image features. Finally, an improved Support Vector Regression (SVR) learning method based on grid search is proposed to establish the mapping between image feature values and the quality score of a test image. Experiments proved that the proposed method is effective when compared with other related state-of-the-art image quality assessment (IQA) methods based on three standard databases.

The current rate of success in launching satellites and advances in onboard and ground image processing have led to a dramatic increase in the scale of remote sensing image data. This has resulted in considerable research on how to provide the best quality of experience to end users. However, subjective image quality assessment (IQA) is time-consuming, cumbersome, expensive and cannot be implemented automatically using computers. Thus, subjective IQA may not be suitable for application requirements. In this paper, we design and construct a usability-based subjective remote sensing IQA database. A corresponding no-reference IQA method is also proposed. The new IQA method uses scale-invariant feature transforms to form a dictionary and then a support vector machine to obtain an IQA model. The experimental results show that the new subjective IQA database is highly suited to the task, and the quality predictions of the new IQA method correlate well with human subjective scores in the new database.

In recent years, deep learning has achieved significant progress in many respects. However, unlike other research fields with millions of labeled data such as image recognition, only several thousand labeled images are available in image quality assessment (IQA) field for deep learning, which heavily hinders the development and application for IQA. To tackle this problem, in this paper, we proposed an error self-learning semi-supervised method for no-reference (NR) IQA (ESSIQA), which is based on deep learning. We employed an advanced full reference (FR) IQA method to expand databases and supervise the training of network. In addition, the network outputs of expanding images were used as proxy labels replacing errors between subjective scores and objective scores to achieve error self-learning. Two weights of error back propagation were designed to reduce the impact of inaccurate outputs. The experimental results show that the proposed method yielded comparative effect.