Deep feature importance awareness based no-reference image quality prediction

Shift-insensitive perceptual feature of quadratic sum of gradient magnitude and LoG signals for image quality assessment and image classification

The rapid progress of multi-modal large language models (MLLMs) has boosted the task of image quality assessment (IQA). However, a key challenge arises from the inherent mismatch between the discrete token outputs of MLLMs and the continuous nature of quality scores required by IQA tasks. This discrepancy significantly hinders the performance of MLLM-based IQA methods. Previous approaches that convert discrete token predictions into continuous scores often suffer from conversion errors. Moreover, the semantic confusion introduced by level tokens (e.g., “good”) further constrains the performance of MLLMs on IQA tasks and degrades their original capabilities to related tasks. To tackle these problems, we provide a theoretical analysis of the errors inherent in previous approaches and, motivated by this analysis, propose a simple yet effective framework, Q-Scorer. This framework incorporates a lightweight regression module and IQA-specific score tokens into the MLLM pipeline. Extensive experiments demonstrate that Q-Scorer achieves state-of-the-art performance across multiple IQA benchmarks, generalizes well to mixed datasets, and further improves combined with other methods.

Algorithms for image quality assessment (IQA) aim to predict the qualities of images in a manner that agrees with subjective quality ratings. Over the last several decades, the major impetus in IQA research has focused on improving predictive performance; very few studies have focused on analyzing and improving the runtime performance of IQA algorithms. This paper is the first to examine IQA algorithms from the perspective of their interaction with the underlying hardware and microarchitectural resources, and to perform a systematic performance analysis using state-of-the-art tools and techniques from other computing disciplines. We implemented four popular full-reference IQA algorithms (most apparent distortion, multiscale structural similarity, visual information fidelity, and visual signal-to-noise ratio) and two no-reference algorithms (blind image integrity notator using DCT statistics and blind/referenceless image spatial quality evaluator) in C++ based on the code provided by their respective authors. We then conducted a hotspot analysis to identify sections of code that were performance bottlenecks and performed microarchitectural analysis to identify the underlying causes for these bottlenecks. Despite the fact that all six algorithms share common algorithmic operations (e.g., filterbanks and statistical computations), our results revealed that different IQA algorithms overwhelm different microarchitectural resources and give rise to different types of bottlenecks. Based on these results, we propose microarchitectural-conscious coding techniques and custom hardware recommendations for performance improvement.

With the rise in research on applications of medical image processing, the evaluation of parameters and techniques required for measurement of medical image quality is the need of the hour. The effective, yet automatic methods for measurement of quality of a medical image are of particular interest. This chapter is an overview of different medical imaging technologies, and the related image quality assessment (IQA) algorithms. The main focus is on objective assessment (OA), rather than subjective assessment (SA). Three types of OA-based IQA algorithms are presented in detail: full reference-based IQA (FR-IQA) algorithms; no reference-based IQA (NR-IQA) algorithms and reduced reference-based IQA (RR-IQA) algorithms.

The current rate of success in launching satellites and advances in onboard and ground image processing have led to a dramatic increase in the scale of remote sensing image data. This has resulted in considerable research on how to provide the best quality of experience to end users. However, subjective image quality assessment (IQA) is time-consuming, cumbersome, expensive and cannot be implemented automatically using computers. Thus, subjective IQA may not be suitable for application requirements. In this paper, we design and construct a usability-based subjective remote sensing IQA database. A corresponding no-reference IQA method is also proposed. The new IQA method uses scale-invariant feature transforms to form a dictionary and then a support vector machine to obtain an IQA model. The experimental results show that the new subjective IQA database is highly suited to the task, and the quality predictions of the new IQA method correlate well with human subjective scores in the new database.

Image Quality Assessment (IQA) algorithms evaluate the perceptual quality of an image using evaluation scores that assess the similarity or difference between two images. We propose a new low-level feature based IQA technique, which applies filter-bank decomposition and center-surround methodology. Differing from existing methods, our model incorporates color intensity adaptation and frequency scaling optimization at each filter-bank level and spatial orientation to extract and enhance perceptually significant features. Our computational model exploits the concept of object detection and encapsulates characteristics proposed in other IQA algorithms in a unified architecture. We also propose a systematic approach to review the evolution of IQA algorithms using unbiased test datasets, instead of looking at individual scores in isolation. Experimental results demonstrate the feasibility of our approach.

Lung nodule image quality assessment under iterative model reconstruction

An optimized CNN-based quality assessment model for screen content image

Objective Image Quality Assessment (IQA) of low-dose CT holds significant importance, yet it faces substantial challenges such as the lack of reference images, multiple non-specific types of distortions, the absence of a recognized database, and inadequate simulation of Human Visual System (HVS) understanding. This paper focuses on the characteristics of low-dose CT images and conducts research on constructing an IQA model based on a multi-task U-Net and the human eye’s attention mechanism. Firstly, a multi-task U-Net model is established to simultaneously perform image region segmentation and image quality assessment tasks. By extracting and sharing features from the effective abdomen region, two related tasks reinforce each other. Secondly, considering the human visual attention mechanism, an attention weight map is obtained based on phase congruency, it is introduced into the multi-task U-Net IQA model to explore a general no-reference deep learning IQA model that can more comprehensively and effectively simulate the human perception process. Finally, the consistency of the proposed model with subjective evaluations is validated on international open-source databases LDCTIQAC2023. This paper provides new insights and tools for objective IQA of medical images, ensuring the quality of CT clinical diagnosis and treatment. The research findings are also easily extensible to other tissue organs and modalities of medical images, with potential applications in various fields, demonstrating extensive value.

Recently, various objective image quality assessment (IQA) metrics have been proposed for measuring the quality of a captured image. Moreover, IQA databases have been made available to enable researchers develop and test IQA metrics. For instance, the TID2013 database was used in the experiments conducted in this study. It contains many test images obtained from 25 reference images, each having 24 types of distortion and five levels of noise for each distortion. However, estimating a subjective evaluation score for many types of noise is a difficult task. Therefore, this study proposes an objective IQA framework for evaluating noise clusters using image features. Experimental results confirmed the effectiveness of the proposed framework.

영상 처리 및 컴퓨터 비전 분야에 있어서, 평균 제곱 오차(Mean Squared Error: MSE)는 좋은 수학적 특성(예를 들어, 척도성(metricability), 미분가능성(differentiability) 및 볼록 성질(convexity))을 가짐으로 인해 많은 영상 화질 최적화 문제의 객관적 척도로 사용되어 왔다. 그러나 MSE가 영상의 왜곡 신호에 대한 시각적 인지 화질과 상관도가 높지 않다는 것이 알려지면서, 이를 해결하기 위해 위에서 언급한 좋은 수학적 특성과 높은 영상 화질 예측 성능을 동시에 가지는 객관적 영상 화질 측정(Image Quality Assessment: IQA)척도가 활발히 연구되어 왔다. 비록 최근 제안된 좋은 수학적 성질을 만족시키는 IQA 척도들은 MSE와 비교하여 매우 향상된 주관적 화질 예측 성능을 보이지만, 상대적으로 높은 계산 복잡도를 가진다. 본 논문은 이를 해결하기 위해, 단순 라플라스 연산자를 이용한 좋은 수학적 특성을 가지는 새로운 IQA 척도를 제안한다. 제안 IQA 방법에 도입한 단순 라플라스 연산자는 인간 시각 체계의 망막에서의 광도 자극에 대한 시신경 반응을 효과적으로 모사할 뿐만 아니라 계산이 매우 단순하기 때문에, 제안 IQA 척도는 단순 라플라스 연산자를 사용하여 매우 빠른 계산 속도와 높은 주관적 화질 점수 예측력을 확보하였다. 제안 IQA 척도의 효과를 검증하기 위해, 최신 IQA 척도들과 광범위한 성능비교 실험을 수행하였다. 실험 결과, 제안하는 IQA 척도는 모든 테스트 IQA 척도들 중 MSE를 제외하고 가장 빠른 처리 속도를 보였을 뿐만 아니라, 가장 높은 주관적 화질예측 성능을 보였다. In image processing and computer vision fields, mean squared error (MSE) has popularly been used as an objective metric in image quality optimization problems due to its desirable mathematical properties such as metricability, differentiability and convexity. However, as known that MSE is not highly correlated with perceived visual quality, much effort has been made to develop new image quality assessment (IQA) metrics having both the desirable mathematical properties aforementioned and high prediction performances for subjective visual quality scores. Although recent IQA metrics having the desirable mathematical properties have shown to give some promising results in prediction performance for visual quality scores, they also have high computation complexities. In order to alleviate this problem, we propose a new fast IQA metric using a simple Laplace operator. Since the Laplace operator used in our IQA metric can not only effectively mimic operations of receptive fields in retina for luminance stimulus but also be simply computed, our IQA metric can yield both very fast processing speed and high prediction performance. In order to verify the effectiveness of the proposed IQA metric, our method is compared to some state-of-the-art IQA metrics. The experimental results showed that the proposed IQA metric has the fastest running speed compared the IQA methods except MSE under comparison. Moreover, our IQA metric achieves the best prediction performance for subjective image quality scores among the state-of-the-art IQA metrics under test.

Measurement of visual quality is of fundamental importance to numerous image and video processing applications. The goal of quality assessment (QA) research is to design algorithms that can automatically assess the quality of images or videos in a perceptually consistent manner. Image QA algorithms generally interpret image quality as fidelity or similarity with a "reference" or "perfect" image in some perceptual space. Such "full-reference" QA methods attempt to achieve consistency in quality prediction by modeling salient physiological and psychovisual features of the human visual system (HVS), or by signal fidelity measures. In this paper, we approach the image QA problem as an information fidelity problem. Specifically, we propose to quantify the loss of image information to the distortion process and explore the relationship between image information and visual quality. QA systems are invariably involved with judging the visual quality of "natural" images and videos that are meant for "human consumption." Researchers have developed sophisticated models to capture the statistics of such natural signals. Using these models, we previously presented an information fidelity criterion for image QA that related image quality with the amount of information shared between a reference and a distorted image. In this paper, we propose an image information measure that quantifies the information that is present in the reference image and how much of this reference information can be extracted from the distorted image. Combining these two quantities, we propose a visual information fidelity measure for image QA. We validate the performance of our algorithm with an extensive subjective study involving 779 images and show that our method outperforms recent state-of-the-art image QA algorithms by a sizeable margin in our simulations. The code and the data from the subjective study are available at the LIVE website.

We present a no-reference (NR) image quality assessment (IQA) algorithm that is inspired by the representation of visual scenes in the primary visual cortex of the human visual system. Specifically, we use the sparse coding model of the area V1 to construct an overcomplete dictionary for sparsely representing pristine (undistorted) natural images. First, we empirically demonstrate that the distribution of the sparse representation coefficients of natural images have sharp peaks and heavy tails, and can therefore be modeled using a Univariate Generalized Gaussian Distribution (UGGD). We then show that the UGGD model parameters form good features for distortion estimation and formulate our no-reference IQA algorithm based on this observation. Subsequently, we find UGGD model parameters that are representative of the class of pristine natural images. This is achieved using a training set of undistorted natural images. The perceptual quality of a test image is then defined to be the likelihood of its sparse coefficients being generated from the pristine UGGD model. We show that the proposed algorithm correlates well with subjective evaluation over several standard image databases. Further, the proposed method allows us to construct a distortion map that has several useful applications like distortion localization, adaptive rate allocation etc. Finally and importantly, the proposed NR-IQA algorithm does not make use of any distortion information or subjective scores during the training process.

With the rapid development of Internet technology, images on the Internet are used in various aspects of people’s lives. The security and authorization of images are strongly dependent on image quality. Some potential problems have also emerged, among which the quality assessment and denoising of images are particularly evident. This paper proposes a novel NR-IQA method based on the dual convolutional neural network structure, which combines saliency detection with the human visual system (HSV), used as a weighting function to reflect the important distortion caused by the local area. The model is trained using gray and color features in the HSV space. It is applied to the parameter selection of an image denoising algorithm. The experiment proves that our proposed method can accurately evaluate image quality in the process of denoising. It provides great help in parameter optimization iteration and improves the performance of the algorithm. Through experiments, we obtain both improved image quality and a reasonable result of subject assessment when the cascaded algorithm is applied in image security and authorization.

No-reference image quality assessment based on spatial and spectral entropies