Blind Image Quality Assessment via Deep Response Feature Decomposition and Aggregation

A good distortion representation is crucial for the success of deep blind image quality assessment (BIQA). However, most previous methods do not effectively model the relationship between distortions or the distribution of samples with same distortion type but different distortion levels. In this work, we start from the analysis of the relationship between perceptual image quality and distortion-related factors, such as distortion types and levels. Then, we propose a Distortion Graph Representation (DGR) learning framework for IQA, named GraphIQA, in which each distortion is represented as a graph, i.e., DGR. One can distinguish distortion types by learning the contrast relationship between these different DGRs, and infer the ranking distribution of samples from different levels in a DGR. Specifically, we develop two sub-networks to learn the DGRs: a) Type Discrimination Network (TDN) that aims to embed DGR into a compact code for better discriminating distortion types and learning the relationship between types; b) Fuzzy Prediction Network (FPN) that aims to extract the distributional characteristics of the samples in a DGR and predicts fuzzy degrees based on a Gaussian prior. Experiments show that our GraphIQA achieves the state-of-the-art performance on many benchmark datasets of both synthetic and authentic distortions. The code is available at http://staff.ustc.edu.cn/~chenzhibo/resources/2021/GraphIQA.html.

No-reference screen content image quality assessment based on multi-region features

To monitor objects of interest, such as wildlife and people, image-capturing devices are used to collect a large number of images with and without objects of interest. As we are recording valuable information about the behavior and activity of objects, the quality of images containing objects of interest should be better than that of images without objects of interest, even if the former exhibits more severe distortion than the latter. However, according to current methods, quality assessments produce the opposite results. In this study, we propose an end-to-end model, named DETR-IQA (detection transformer image quality assessment), which extends the capability to perform object detection and blind image quality assessment (IQA) simultaneously by adding IQA heads comprising simple multi-layer perceptrons at the top of the DETRs (detection transformers) decoder. Using IQA heads, DETR-IQA carried out blind IQAs based on the weighted fusion of the distortion degree of the region of objects of interest and the other regions of the image; the predicted quality score of images containing objects of interest was generally greater than that of images without objects of interest. Currently, the subjective quality score of all public datasets is in accordance with the distortion of images and does not consider objects of interest. We manually extracted the images in which the five predefined classes of objects were the main contents of the largest authentic distortion dataset, KonIQ-10k, which was used as the experimental dataset. The experimental results show that with slight degradation in object detection performance and simple IQA heads, the values of PLCC and SRCC were 0.785 and 0.727, respectively, and exceeded those of some deep learning-based IQA models that are specially designed for only performing IQA. With the negligible increase in the computation and complexity of object detection and without a decrease in inference speeds, DETR-IQA can perform object detection and IQA via multi-tasking and substantially reduce the workload.

In this paper, a universal blind image quality assessment (IQA) algorithm is proposed that works in presence of various distortions. The proposed algorithm is a Blind Image Quality metric based on Wakeby Statistics (BIQWS) which extracts local mean subtraction and contrast normalization (MSCN) coefficients in spatial domain from input image. The MSCN coefficients are used for generating a Wakeby distribution statistical model to extract quality-aware features. The statistical studies indicate that the MSCN coefficients histogram is altered in the presence of various distortions with different severities. These changes are regular and can be used to estimate the type of the distortion and its severity. We extended our previous studies to extract efficient Wakeby distribution model parameters which are more sensitive to changes in MSCN coefficients. These parameters are used to form a quality-aware feature vector. This feature vector is then fed to an SVM (support vector machine) regression model with a nonlinear Kernel to predict the quality score of the input image without any information about the distortion type or reference image. Experimental results show that the image quality index obtained by the proposed method has higher correlation with respect to human perceptual opinions and it is superior in some distortions when compared to some full-reference and other state-of-the-art blind image quality assessment methods.

Due to the rapid development of multimedia technologies, over the last decades image quality assessment (IQA) has become an important topic. As a consequence, a great research effort has been made to develop computational models that estimate image quality. Among the possible IQA approaches, blind IQA (BIQA) is of fundamental interest as it can be used in most multimedia applications. BIQA techniques measure the perceptual quality of an image without using the reference (or pristine) image. This paper proposes a new BIQA method that uses a combination of texture features and saliency maps of an image. Texture features are extracted from the images using the local binary pattern (LBP) operator at multiple scales. To extract the salient of an image, i.e. the areas of the image that are the main attractors of the viewers' attention, we use computational visual attention models that output saliency maps. These saliency maps can be used as weighting functions for the LBP maps at multiple scales. We propose an operator that produces a combination of multiscale LBP maps and saliency maps, which is called the multiscale salient local binary pattern (MSLBP) operator. To define which is the best model to be used in the proposed operator, we investigate the performance of several saliency models. Experimental results demonstrate that the proposed method is able to estimate the quality of impaired images with a wide variety of distortions. The proposed metric has a better prediction accuracy than state-of-the-art IQA methods.

An optimized CNN-based quality assessment model for screen content image

Image quality assessment (IQA) is a fundamental yet constantly developing task for computer vision and image processing. Most IQA evaluation mechanisms are based on the pertinence of subjective and objective estimation. Each image distortion type has its own property correlated with human perception. However, this intrinsic property may not be fully exploited by existing IQA methods. In this paper, we make two main contributions to the IQA field. First, a novel IQA method is developed based on a local linear model that examines the distortion between the reference and the distorted images for better alignment with human visual experience. Second, a distortion-specific compensation strategy is proposed to offset the negative effect on IQA modeling caused by different image distortion types. These score offsets are learned from several known distortion types. Furthermore, for an image with an unknown distortion type, a convolutional neural network-based method is proposed to compute the score offset automatically. Finally, an integrated IQA metric is proposed by combining the aforementioned two ideas. Extensive experiments are performed to verify the proposed IQA metric, which demonstrate that the local linear model is useful in human perception modeling, especially for individual image distortion, and the overall IQA method outperforms several state-of-the-art IQA approaches.

The convolutional neural network (CNN) has achieved great success in many visual tasks. However, it has limited progress on image quality assessment (IQA) due to the lacking of IQA-oriented CNN framework which can efficiently represent the hierarchical quality degradation. In this paper, inspired by the hierarchical perception mechanism (from local structure to global semantics) in the human visual system, we design an end-to-end cascaded CNN framework for blind IQA (BIQA), in which multilevel features are extracted and concatenated to represent the hierarchical quality degradation. By jointly optimizing the feature extraction, hierarchical degradation integration, and quality prediction in an end-to-end manner, the novel cascaded CNN with hierarchical feature integration (CaHFI) for BIQA is designed. Experimental results on five benchmark IQA databases demonstrate that the proposed CaHFI achieves the state-of-the-art. And experiments on cross-database evaluation further prove the high generalization ability of the proposed CaHFI.

Faithful measurement of perceptual quality is of significant importance to various multimedia applications. By fully utilizing reference images, full-reference image quality assessment (FR-IQA) methods usually achieves better prediction performance. On the other hand, no-reference image quality assessment (NR-IQA), also known as blind image quality assessment (BIQA), which does not consider the reference image, makes it a challenging but important task. Previous NR-IQA methods have focused on spatial measures at the expense of information in the available frequency bands. In this paper, we present a multiscale deep blind image quality assessment method (BIQA, M.D.) with spatial optimal-scale filtering analysis. Motivated by the multi-channel behavior of the human visual system and contrast sensitivity function, we decompose an image into a number of spatial frequency bands by multiscale filtering and extract features for mapping an image to its subjective quality score by applying convolutional neural network. Experimental results show that BIQA, M.D. compares well with existing NR-IQA methods and generalizes well across datasets.

Models for image quality assessment (IQA) are generally optimized and tested by comparing to human ratings, which are expensive to obtain. Here, we develop a blind IQA (BIQA) model, and a method of training it without human ratings. We first generate a large number of corrupted image pairs, and use a set of existing IQA models to identify which image of each pair has higher quality. We then train a convolutional neural network to estimate perceived image quality along with the uncertainty, optimizing for consistency with the binary labels. The reliability of each IQA annotator is also estimated during training. Experiments demonstrate that our model outperforms state-of-the-art BIQA models in terms of correlation with human ratings in existing databases, as well in group maximum differentiation (gMAD) competition.

Late gadolinium-enhanced (LGE) MRI has become a widely used technique to non-invasively image the left atrium prior to catheter ablation. However, LGE-MRI images are prone to variable image quality, with quality metrics that do not necessarily correlate to the image's diagnostic quality. In this study, we aimed to define consistent clinically relevant metrics for image and diagnostic quality in 3D LGE-MRI images of the left atrium, have multiple observers assess LGE-MRI image quality to identify key features that measure quality and intra/inter-observer variabilities, and train and test a CNN to assess image quality automatically. We identified four image quality categories that impact fibrosis assessment in LGE-MRI images and trained individuals to score 50 consecutive pre-ablation atrial fibrillation LGE-MRI scans from the University of Utah hospital image database. The trained individuals then scored 146 additional scans, which were used to train a convolutional neural network (CNN) to assess diagnostic quality. There was excellent agreement among trained observers when scoring LGE-MRI scans, with inter-rater reliability scores ranging from 0.65 to 0.76 for each category. When the quality scores were converted to a binary diagnostic/non-diagnostic, the CNN achieved a sensitivity of and a specificity of . The use of a training document with reference examples helped raters achieve excellent agreement in their quality scores. The CNN gave a reasonably accurate classification of diagnostic or non-diagnostic 3D LGE-MRI images of the left atrium, despite the use of a relatively small training set.

An investigation of diseases using magnetic resonance (MR) imaging requires automatic image quality assessment methods able to exclude low-quality scans. Such methods can be also employed for an optimization of parameters of imaging systems or evaluation of image processing algorithms. Therefore, in this paper, a novel blind image quality assessment (BIQA) method for the evaluation of MR images is introduced. It is observed that the result of filtering using non-maximum suppression (NMS) strongly depends on the perceptual quality of an input image. Hence, in the method, the image is first processed by the NMS with various levels of acceptable local intensity difference. Then, the quality is efficiently expressed by the entropy of a sequence of extrema numbers obtained with the thresholded NMS. The proposed BIQA approach is compared with ten state-of-the-art techniques on a dataset containing MR images and subjective scores provided by 31 experienced radiologists. The Pearson, Spearman, Kendall correlation coefficients and root mean square error for the method assessing images in the dataset were 0.6741, 0.3540, 0.2428, and 0.5375, respectively. The extensive experimental evaluation of the BIQA methods reveals that the introduced measure outperforms related techniques by a large margin as it correlates better with human scores.

In the domain of image and multimedia processing, image quality is a critical factor, as it directly influences the performance of subsequent tasks such as compression, transmission, and content analysis. Reliable assessment of image quality is therefore essential not only for benchmarking algorithms but also for ensuring user satisfaction in real-world multimedia applications. The most advanced Blind image quality assessment (BIQA) methods are typically built upon deep learning models and rely on complex architectures that, while effective, require substantial computational resources and large-scale training datasets. This complexity can limit their scalability and practical deployment, particularly in resource-constrained environments. In this paper, we revisit a model inspired by one of the early applications of convolutional neural networks (CNNs) in BIQA and demonstrate that by leveraging recent advancements in machine learning—such as Bayesian hyperparameter optimization and widely used stochastic optimization methods (e.g., Adam)—it is possible to achieve competitive performance using a simpler, more scalable, and lightweight architecture. To evaluate the proposed approach, we conducted extensive experiments on widely used benchmark datasets, including TID2013 and KADID-10k. The results show that the proposed model achieves competitive performance while maintaining a substantially more efficient design. These findings suggest that lightweight CNN-based models, when combined with modern optimization strategies, can serve as a viable alternative to more elaborate frameworks, offering an improved balance between accuracy, efficiency, and scalability.

The direct use of a deep convolutional neural network (CNN) in no-reference image quality assessment (NR-IQA) usually struggles for a good performance due to a lack of training data, which can be alleviated by transfer learning. However, depending on the similarity between the source and target tasks, the final performance differs vastly. In particular, various kinds of distortion types exist in IQA, which requires different kinds of features to predict visual quality. In this paper, to make the transferred model robust to various distortion types, we propose a Multiple-level Feature-based Image Quality Assessor (MFIQA) which considers multiple levels of features simultaneously. Through rigorous experiments, we prove that MFIQA consistently yields state-of-the-art performance regardless of the distortion types including synthetic and authentic corruption.

Since the introduction of 360-degree images, a significant number of deep learning based image quality assessment (IQA) models have been introduced. Most of them are based on multichannel architectures where several convolutional neural networks (CNNs) are used together. Despite the competitive results, these models come with a higher cost in terms of complexity. To significantly reduce the complexity and ease the training of the CNN model, this paper proposes a patch-based scheme dedicated to 360-degree IQA. Our framework is developed including patches selection and extraction based on latitude to account for the importance of the equatorial region, data normalization, CNN-based architecture and a weighted average pooling of predicted local qualities. We evaluate the proposed model on two widely used databases and show the superiority to state-of-the-art models, even multichannel ones. Furthermore, the cross-database assessment revealed the good generalization ability, demonstrating the robustness of the proposed model.