Classification of priors and regularization techniques appurtenant to single image super-resolution

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.

The single image super resolution recovers missing high resolution details so as to reconstruct a high resolution image from a single low resolution image. This paper proposes a novel directionally adaptive, learning-based, single image super resolution method using multiple direction wavelet transform, called directionlets. Here, critically sampled directionlets are used to capture directional features effectively and to extract edge information along different directions from a set of available high resolution images. This information is used as the training set for super resolving a low resolution input image. The directionlet coefficients at finer scales of its high resolution image are learned locally from this training set and the inverse directionlet transform recovers the super resolved high resolution image. The simulation results showed that the proposed directionlet approach outperforms standard interpolation techniques like cubic spline interpolation as well as standard wavelet-based learning, both visually and in terms of the mean squared error (MSE) values. The SNR scores for cubic spline interpolation, wavelet and directionlet method are 13.6998 dB, 23.8324 dB and 30.8654 dB respectively for Barbara.

Efficient deep learning based methods have achieved significant performance in single image super-resolution. Recent research on efficient super-resolution has mainly focused on reducing the number of parameters and computational complexity through various network designs, enabling models to be better deployed on resource constrained devices. In this work, we propose a novel and effective super-resolution model based on attention mechanism and structural re-parameterization, called Dual Path Attention and Re-parameterization Network DPARN , which uses a hybrid attention mechanisms to balance the running speed and reconstruction quality of the model. Specifically, we utilize grouped convolution to introduce both parameter free and enhanced spatial attention to improve the feature extraction capability of student networks. Meanwhile, we ado-pted a novel lightweight network training strategy that first uses knowledge distillation for initial training, during which structured knowledge from the teacher network is transmitted to the student network. Then, multiple loss functions are combined to fine-tuning the student network, in order to preserve high-frequency details and avoid excessive smoothing caused by pixel loss. Finally, extensive experiments conducted on four benchmark datasets demonstrated the effectiveness and efficiency of proposed DPARN. Our method achieves PSNR SSIM performance comparable to state-of-the-art efficient super-resolution models, with faster inference speed and fewer network parameters.

Deep convolutional neural networks (CNNs), trained on corresponding pairs of high- and low-resolution images, achieve state-of-the-art performance in single-image super- resolution and surpass previous signal-processing based approaches. However, their performance is limited when applied to real photographs. The reason lies in their train- ing data: low-resolution (LR) images are obtained by bicu- bic interpolation of the corresponding high-resolution (HR) images. The applied convolution kernel significantly differs from real-world camera-blur. Consequently, while current CNNs well super-resolve bicubic-downsampled LR images, they often fail on camera-captured LR images. To improve generalization and robustness of deep super- resolution CNNs on real photographs, we present a ker- nel modeling super-resolution network (KMSR) that incor- porates blur-kernel modeling in the training. Our pro- posed KMSR consists of two stages: we first build a pool of realistic blur-kernels with a generative adversarial net- work (GAN) and then we train a super-resolution network with HR and corresponding LR images constructed with the generated kernels. Our extensive experimental vali- dations demonstrate the effectiveness of our single-image super-resolution approach on photographs with unknown blur-kernels.

Single image super-resolution (SISR) is of great importance as a low-level computer vision task. The fast development of Generative Adversarial Network (GAN) based deep learning architectures realises an efficient and effective SISR to boost the spatial resolution of natural images captured by digital cameras. However, the SISR for medical images is still a very challenging problem. This is due to (1) compared to natural images, in general, medical images have lower signal to noise ratios, (2) GAN based models pre-trained on natural images may synthesise unrealistic patterns in medical images which could affect the clinical interpretation and diagnosis, and (3) the vanilla GAN architecture may suffer from unstable training and collapse mode that can also affect the SISR results. In this paper, we propose a novel lesion focused SR (LFSR) method, which incorporates GAN to achieve perceptually realistic SISR results for brain tumour MRI images. More importantly, we test and make comparison using recently developed GAN variations, e.g., Wasserstein GAN (WGAN) and WGAN with Gradient Penalty (WGAN-GP), and propose a novel multi-scale GAN (MS-GAN), to achieve a more stabilised and efficient training and improved perceptual quality of the super-resolved results. Based on both quantitative evaluations and our designed mean opinion score, the proposed LFSR coupled with MS-GAN has performed better in terms of both perceptual quality and efficiency.

Special Section on Enabling Technologies for Network-Based Applications

Scene text image super-resolution (STISR), often considered a preliminary step for scene text recognition, refers to the task of enhancing the resolution of text embedded in natural scene images and plays a vital role in various applications. Most of the existing STISR methods either leverage deep convolutional neural networks by regarding text images as natural scene images or use a text recognizer’s feedback as guidance to the STISR process. However, since the text recognition is initially done on low-resolution images, it is mostly inaccurate, more so as the length of the words increases, thus degrading the super-resolution process. In this paper, we introduce DEPP which utilizes dictionary embedding (DE) based probabilistic priors calculated from a large English text corpus consisting of both alphabets and digits. The initial state and the bigram probabilities obtained are fused with the probability obtained from the recognizer, before passing it onto a single image super-resolution (SISR) block. By integrating DE as a prior and implementing a modified perceptual loss, the method effectively captures the contextual information of text, enabling more accurate super-resolution and visually pleasing results. Experimental results on the benchmark TextZoom dataset demonstrate that our DEPP framework achieves superior performance compared to most existing approaches, particularly for medium and long-length words, as measured by text recognition accuracy. Since DEPP uses the text recognition attributes to rectify or guide the super-resolution process, it makes our method more domain-inspired and task-aware, compared to usual black box deep learners.

Super-resolution (SR) techniques present a suitable solution to increase the image resolution acquired using an ultrasound device characterized by a low image resolution. This can be particularly beneficial in low-resource imaging settings. This work surveys advanced SR techniques applied to enhance the resolution and quality of fetal ultrasound images, focusing Dual back-projection based internal learning (DBPISR) technique, which utilizes internal learning for blind super-resolution, as opposed to blind super-resolution generative adversarial network (BSRGAN), real-world enhanced super-resolution generative adversarial network (Real-ESRGAN), swin transformer for image restoration (SwinIR) and SwinIR-Large. The dual back-projection approach enhances SR by iteratively refining downscaling and super-resolution processes through a dual network training method, achieving high accuracy in kernel estimation and image reconstruction. Real-ESRGAN uses synthetic data to simulate complex real-world degradations, incorporating a U-shaped network (U-Net) discriminator to improve training stability and visual performance. BSRGAN addresses the limitations of traditional degradation models by introducing a realistic and comprehensive degradation process involving blur, downsampling, and noise, leading to superior real-world SR performance. Swin models (SwinIR and SwinIR_large) employ a Swin Transformer architecture for image restoration, excelling in capturing long-range dependencies and complex structures, resulting in an outstanding performance in PSNR, SSIM, NIQE, and BRISQUE metrics. The tested images, sourced from five developing countries and often of lower quality, enabled us to show that these approaches can help enhance the quality of the images. Evaluations on fetal ultrasound images reveal that these methods significantly enhance image quality, with DBPISR, Real-ESRGAN, BSRGAN, SwinIR, and SwinIR-Large showing notable improvements in PSNR and SSIM, thereby highlighting their potential for improving the resolution and diagnostic utility of fetal ultrasound images. We evaluated the five aforementioned Super-Resolution models, analyzing their impact on both image quality and classification tasks. Our findings indicate that these techniques hold great potential for enhancing the evaluation of medical images, particularly in development countries. Among the models tested, Real-ESRGAN consistently enhanced both image quality and diagnostic accuracy, even when challenged by limited and variable datasets. This finding was further supported by deploying the ConvNext-base classifier, which demonstrated improved performance when applied to the super-resolved images. Real-ESRGAN’s capacity to enhance image quality, and in turn, classification accuracy, highlights its potential to address the resource constraints often encountered in these settings.

Depth image super-resolution based on joint sparse coding

An existing multi-scale residual network (MSRN) has demonstrated its success on conducting the single image super-resolution (SISR) task. The MSRN consists of a number of multi-scale residual blocks (MSRBs), and each MSRB performs convolutions by exploiting two different sizes of windows for conducting multi-scale feature extraction. The smaller window is used to extract image features at a low scale, while the larger one is used for a high scale. To significantly reduce the number of parameters involved in the MSRB, a new feature extraction module, called the asynchronous multi-scale block (AMB), is proposed in this paper. It is based on the fact that the larger window used in the MSRB can be replaced by two smaller windows without affecting the original MSRB's function. Consequently, by replacing each MSRB with our AMB, an asynchronous multi-scale network (AMNet) is then constructed, which can yield a significant reduction on computational complexity. This means that more AMBs can be used in our AMNet to deliver superior SISR performance, while maintaining the same or comparable computational complexity to that of the MSRN. To consolidate all image features generated from all scales, a new fusion scheme, called the adaptive feature fusion block (AFFB), is proposed that weights the extracted features according to their importance for further increasing SISR's performance. Extensive experimental results have clearly shown the superiority of our proposed AMNet when compared with multiple state-of-the-arts.

We explore in this paper an efficient algorithmic solution to single image super-resolution (SR). We propose the gCLSR, namely graph-Constrained Least Squares Regression, to super-resolve a high-resolution (HR) image from a single low-resolution (LR) observation. The basic idea of gCLSR is to learn a projection matrix mapping the LR image patch to the HR image patch space while preserving the intrinsic geometric structure of the original HR image patch manifold. Even if gCLSR resembles other manifold learning-based SR methods in preserving the local geometric structure of HR and LR image patch manifolds, the innovation of gCLSR lies in that it preserves the intrinsic geometric structure of the original HR image patch manifold rather than the LR image patch manifold, which may be contaminated by image degeneration (e.g., blurring, down-sampling and noise). Upon acquiring the projection matrix, the target HR image can be simply super-resolved from a single LR image without the need of HR-LR training pairs, which favors resource-limited applications. Experiments on images from the public database show that gCLSR method can achieve competitive quality as state-of-the-art methods, while gCLSR is much more efficient in computation than some state-of-the-art methods.

We propose a new model called iterative collaborative representation (ICR) for image super-resolution (SR). Most of popular SR approaches extract low-resolution (LR) features from the given LR image directly to recover its corresponding high-resolution (HR) features. However, they neglect to utilize the reconstructed HR image for further image SR enhancement. Based on this observation, we extract features from the reconstructed HR image to progressively upscale LR image in an iterative way. In the learning phase, we use the reconstructed and the original HR images as inputs to train the mapping models. These mapping models are then used to upscale the original LR images. In the reconstruction phase, mapping models and LR features extracted from the LR and reconstructed image are then used to conduct image SR in each iteration. Experimental results on standard images demonstrate that our ICR obtains state-of-the-art SR performance quantitatively and visually, surpassing recently published leading SR methods.KeywordsIterative collaborative representationSuper-resolution

Recently deep convolutional neural network (CNN) has made significant achievement in Single Image Super-Resolution (SISR). Most CNN-based SISR methods used the default L2 norm of the error. However, for Hyperspectral Image (HSI), this loss function may bring spectral inconsistencies. The main reason is that most methods did not pay much attention to spectral loss. To HSI, the loss function should capture not only spatial information but also spectral consistency. In this paper, a Multi-Losses Function Network (MLFN) simultaneously considering spatial and spectral information is proposed, and is composed of two parts: one is Concatenate Dense Residual Network (CDRN), and the other is Loss Network (LN). CDRN is an image reconstruction network which can utilize the hierarchical features extracted from the low-resolution image. LN includes pixel-wise spatial loss and spectral loss which drive the learning of the entire reconstruction model. The experimental results prove that the proposed MLFN can enhance spatial resolution with the consistency of the spectrum of HSI preserved.

High-resolution (HR) image reconstruction from single low-resolution (LR) image is one of the important vision applications. Despite numerous algorithms have been successfully proposed in recent years, efficient and robust single-image super-resolution (SR) reconstruction is still challenging by several factors, such as inherent ambiguous mapping between the HR-LR images, necessary huge exemplar images, and computational load. In this paper, we proposed a new learning-based method of single-image SR. Inspired by simple mapping functions method, a mapping matrix table of HR-LR feature patches is calculated in the training phase. Each atom of dictionary learned from LR feature patches is corresponding to a mapping matrix in the mapping matrix table. Combining this mapping table with sparse coding, high quality and HR images are reconstructed in reconstruction phase. The effectiveness and efficiency of this method is validated with experiments on the training datasets. Compared with state-of-art methods, jagged and blurred artifacts are depressed effectively and high reconstruction quality is acquired with less exemplar images.

Self-supervised cycle-consistent learning for scale-arbitrary real-world single image super-resolution