Lightweight Image Super-Resolution with Information Multi-distillation Network

Lightweight single image super-resolution with attentive residual refinement network

MSAR-Net: Multi-scale attention based light-weight image super-resolution

Single image super-resolution (SISR) with deep convolutional neural networks has recently attracted increasing attention due to its potentials to generate rich details. To obtain better fidelity and visual quality, most of existing methods are of heavy design with the depth of network. However, the lack of high-frequency information in the deep network and the limit of non-local context relations of pixels remain the core issue. In addition, there are many arbitrary-scale image super-resolution tasks, so it is a focus that provides an effective and efficient model for super resolution with arbitrary-scale. To this end, we propose a Feature Preserving and Enhancing Network (FPEN) based on implicit representation, which is aims to preserving high-frequency information in deep network and enhancing non-local contextual features of pixels to output more realistic images at arbitrary scales. In particular, our proposed High Frequency Preserving Block (HFPB) can divide the features into high-frequency components and low-frequency components, and allocate more operations to high-frequency components to ensure that high-frequency components can be preserved in the network. Since high-frequency information contains the details and texture of the image, it can restore the finer details of the image. Moreover, Pixel Continuity Attention (PiCA) module our proposed utilizes visual cues observed from pixels to adaptively recalibrate the pixel range that needs attention in the image to better achieve feature enhancement and generate smoother images. Extensive experiments conducted on benchmark SISR models and datasets show that Feature Preserving and Enhancing Network can be employed for various SISR tasks with arbitrary-scale to obtain the better visual quality than other state-of-the-art SR algorithms.

The external example-driven single image super-resolution (SISR) method that uses a deep convolutional neural network (CNN) has exhibited superior performance as compared to previously developed SISR methods. However, the advantages of jointly using external and internal examples on a deep CNN framework have not been sufficiently investigated. In this letter, we present a novel method for single image super-resolution by exploiting a complementary relation between external and internal example-based SISR methods. The proposed deep CNN model consists of two subnetworks, a global residual network and a self-residual network, to utilize the advantages of both external and internal examples. In contrast with conventional joint SISR methods, the proposed method is the first deep CNN-based SISR method that does not require a retraining process, which tends to be inefficient. The proposed method outperformed existing methods in both quantitative and qualitative evaluations.

Lightweight image super-resolution with feature enhancement residual network

Traditional works have shown that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Make full use of these multi-scale information can improve the image restoration performance. However, the current proposed deep learning based restoration methods do not take the multi-scale information into account. In this paper, we propose a dilated convolution based inception module to learn multi-scale information and design a deep network for single image super-resolution. Different dilated convolution learns different scale feature, then the inception module concatenates all these features to fuse multi-scale information. In order to increase the reception field of our network to catch more contextual information, we cascade multiple inception modules to constitute a deep network to conduct single image super-resolution. With the novel dilated convolution based inception module, the proposed end-to-end single image super-resolution network can take advantage of multi-scale information to improve image super-resolution performance. Experimental results show that our proposed method outperforms many state-of-the-art single image super-resolution methods.

In this study, a novel single image super-resolution (SR) method, which uses a generated dictionary from pairs of high-resolution (HR) images and their corresponding low-resolution (LR) representations, is proposed. First, HR and LR dictionaries are created by dividing HR and LR images into patches Afterwards, when performing SR, the distance between every patch of the input LR image and those of available LR patches in the LR dictionary are calculated. The minimum distance between the input LR patch and those in the LR dictionary is taken, and its counterpart from the HR dictionary will be passed through an illumination enhancement process resulting in consistency of illumination between neighbour patches. This process is applied to all patches of the LR image. Finally, in order to remove the blocking effect caused by merging the patches, an average of the obtained HR image and the interpolated image is calculated. Furthermore, it is shown that the stabe of dictionaries is reducible to a great degree. The speed of the system is improved by 62.5%. The quantitative and qualitative analyses of the experimental results show the superiority of the proposed technique over the conventional and state-of-the-art methods.

Convolutional neural network (CNN)-based single image super-resolution (SR) methods have achieved superior performance on some discrete-scaling factors, including 2, 3, and 4. However, the scaling factors for SR should be continuous and not discrete in practical applications. Previous CNN-based SR models usually yield poor results for non-integer-scaling factors and are sometimes even worse than results derived from the conventional bicubic method. To extend CNN-based SR models to continuous scale, this paper proposes a multiple-scaling-based SR (MSSR) method that combines an integer-scaling-factor SR and once or twice non-integer-scaling-factor SR without retraining networks. For a non-integer-scaling factor, the MSSR method first computes an optimal integer-scaling factor according to the data similarity and choose the corresponding pre-trained model for the next stage. Then, an existing CNN-based model is used to perform the integer-scaling-factor SR. Finally, the output is scaled to the target size. The proposed MSSR method can extend a variety of existing CNN-based SR models from discrete to continuous-scaling factors. Experimental results with six CNN-based SR models demonstrated that the MSSR method could effectively improve the performance of existing CNN-based SR models for continuous-scaling-factor SR without retraining networks. Furthermore, the comparison with a magnification-arbitrary method, called Meta-SR, shows that the proposed MSSR method usually outperforms Meta-SR for scaling factors greater than or equal to 2.

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.

The single image super resolution recovers missing high resolution details so as to reconstruct a high resolution image from a single low resolution image. This paper proposes a novel directionally adaptive, learning-based, single image super resolution method using multiple direction wavelet transform, called directionlets. Here, critically sampled directionlets are used to capture directional features effectively and to extract edge information along different directions from a set of available high resolution images. This information is used as the training set for super resolving a low resolution input image. The directionlet coefficients at finer scales of its high resolution image are learned locally from this training set and the inverse directionlet transform recovers the super resolved high resolution image. The simulation results showed that the proposed directionlet approach outperforms standard interpolation techniques like cubic spline interpolation as well as standard wavelet-based learning, both visually and in terms of the mean squared error (MSE) values. The SNR scores for cubic spline interpolation, wavelet and directionlet method are 13.6998 dB, 23.8324 dB and 30.8654 dB respectively for Barbara.

Image super resolution is a vital research topic in the field of computer vision. It aims to reconstruct high resolution images from low resolution images. Although the conventional image super resolution methods have achieved good performance and effect, there are still have some issues, e.g., the high-frequency details information is insufficient, and the reconstruction process will bring additional noise, and most basic interpolation techniques produce blurry results. To settle the problems mentioned above, we consider combining the deep learning method with the frequency domain fusion method. In this paper, a novel single image super resolution method based on SRFeat network and wavelet fusion is proposed. First, the training image is taken as the input of the backbone SRFeat network, then the generative adversarial network training is carried out. Then, the up-sampling is utilized to obtain the coarse super resolved image. Finally, the output image after the network training is combined with the up-sampling image of the low-resolution image by Wavelet fusion to obtain the final result. Without increasing the depth of the network and the redundant parameters, the proposed method can achieve better reconstruct result. The experimental results show that the proposed method can not only reduce the probability of image distortion, but recover the global information of the reconstructed image and remove the noise brought by the reconstruction process. The PSNR value of the proposed method is improved 0.3 dB, and the SSIM is improved 0.02.

Sparse representation-based single image super-resolution (SISR) methods use a coupled overcomplete dictionary trained from high-resolution images/image patches. Since remote sensing (RS) satellites capture images of large areas, these images usually have poor spatial resolution and obtaining an effective dictionary as such would be very challenging. Moreover, traditional patch-based sparse representation models for reconstruction tend to give unstable sparse solution and produce visual artefact in the recovered images. To mitigate these problems, in this article, we have proposed an adaptive joint sparse representation-based SISR method that is dependent only on the input low-resolution image for dictionary training and sparse reconstruction. The new model combines patch-based local sparsity and group sparse representation-based nonlocal sparsity in a single framework, which helps in stabilizing the sparse solution and improve the SISR results. The experimental results are evaluated both visually and quantitatively for several RGB and multispectral RS datasets, where the proposed method shows improvements in peak signal-to-noise ratio by 1–4 dB and 2–3 dB over the state-of-the-art sparse representation- and deep learning-based SR methods, respectively. Land cover classification applied on the super-resolved images further validate the advantages of the proposed method. Finally, for practical RS applications, we have performed parallel implementation in general purpose graphics processing units and achieved significant speed ups (30–40×) in the execution time.

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

Single image super-resolution based on compressive sensing and improved TV minimization sparse recovery

In this paper, we propose a novel single image super-resolution (SR) method based on low-rank sparse representation with self-similarity learning. Sparse representation is known as a promising method for SR. However, the sparse codes for low resolution (LR) patches gained by conventional method are not faithful to those for the original high resolution (HR) ones. To overcome this defect, we explore the structures of sparse representation for nonlocal similar patches in natural images by low-rank strategy. It assumes that the sparse codes for nonlocal similar patches should be low-rank. By low-rank constraint, similar components of sparse codes are shared and coding noises are removed, which improves coding accuracy and SR performance. Furthermore, we utilize self-similarity learning framework to generate a self-examples dictionary compatible to the low-rank sparse representation based SR. Experimental results demonstrate that our proposed method can recover good SR results both quantitatively and perceptually.