Lightweight Dual-Stream Residual Network for Single Image Super-Resolution

For a given image, the self-attention mechanism aims to capture dependencies for each pixel. It has been proved that the performance of neural networks which employ self-attention is superior in various image processing tasks. However, the performance of self-attention has extensively correlated with the amount of computation. The vast majority of works tend to use local attention to capture local information to reduce the amount of calculation when using self-attention. The ability to capture information from the entire image is easily weakened on this occasion. In this paper, a local-global attention block (LGAB) is proposed to enhance both the local features and global features with low calculation complexity. To verify the performance of LGAB, a lightweight local-global attention network (LGAN) for single image super-resolution (SISR) is proposed and evaluated. Compared with other lightweight state-of-the-arts (SOTAs) of SISR, the superiority of our LGAN is demonstrated by extensive experimental results. The source code can be found at https://github.com/songzijiang/LGAN.

An efficient lightweight network for single image super-resolution

Kernel-attended residual network for single image super-resolution

Recent research on single image super-resolution (SISR) using convolutional neural networks (CNNs) with the utilization of residual structures and attention mechanisms to utilize image features has demonstrated excellent performance. However, previous SISR techniques mainly integrated extracted image features within a deep or wide network architecture, ignoring the interaction between multiscale features and the diversity of features. At the same time, SISR is also a typical ill-posed problem in that it allows for several predictions for a given LR image. These problems limit the great learning ability of CNNs. To solve these problems, we propose a closed-loop residual attention network (CLRAN) to extract and interact with all the available diversity of features features efficiently and limit the space of possible function solutions. Specifically, we design an enhanced residual attention block (ERA) to extract features, and it dynamically assigns weight to the internal attention branches. The ERA combines multi-scale block (MSB) and enhanced attention mechanism (EAM) base on the residual module. The MSB adaptively detects multiscale image features of different scales by using different 3 × 3 convolution kernels. The EAM combines multi-spectral channel attention (MSCA) and spatial attention (SA). Therefore, the EAM extracts different frequency component information and spatial information to utilize the diversity features. Furthermore, we apply the progressive network architecture and learn an additional map for model monitoring, which forms a closed-loop with the mapping already learned by the LR to HR function. Extensive experiments demonstrate that our CLRAN outperforms the state-of-the-art SISR methods on public datasets for both ×4 and ×8, proving its accuracy and visual perception.

Recently, deep convolutional neural networks (CNNs) are widely used in single image super-resolution (SISR) and have recorded impressive performance. However, most of the existing CNNs architectures can not fully utilize the correlation of feature maps in the middle layers, and abundant features of different levels are lost. Furthermore, convolution operation is limited by processing one local neighborhood at a time, which lacks global information. To address these issues, we propose the nested dense attention network (NDAN) for generating more refined and structured high-resolution images. Specifically, we propose nested dense structure (NDS) to better integrate features of different levels extracted from different layers. Besides that, in order to capture inter-channel dependencies more efficiently, we propose the adaptive channel attention module (ACAM) to adaptively rescale channel-wise features by automatically adjusting the weights of different receptive fields. Furthermore, to better explore the global-level context information, we design hybrid non-local module (HNLM) and hybrid non-local up-sampler (HNLU) to upscale the images by capturing spatial-wise long-distance dependencies and channel-wise long-distance correlation. Numerous experiments demonstrate the effectiveness of our model by achieving higher PSNR and SSIM scores and generating images with better structures against the state-of-the-art methods.

Deep convolutional neural networks (CNNs) have recently achieved great success for single image super-resolution (SISR) task due to their powerful feature representation capabilities. The most recent deep learning based SISR methods focus on designing deeper / wider models to learn the non-linear mapping between low-resolution (LR) inputs and high-resolution (HR) outputs. These existing SR methods do not take into account the image observation (physical) model and thus require a large number of network's trainable parameters with a great volume of training data. To address these issues, we propose a deep Iterative Super-Resolution Residual Convolutional Network (ISRResCNet) that exploits the powerful image regularization and large-scale optimization techniques by training the deep network in an iterative manner with a residual learning approach. Extensive experimental results on various super-resolution benchmarks demonstrate that our method with a few trainable parameters improves the results for different scaling factors in comparison with the state-of-art methods.

Image restoration, such as single image super-resolution (SISR), is a long-established low-level vision issue that intends to regenerate high-resolution (HR) images from low-resolution (LR) input counterparts. While state-of-the-art image super-resolution models are based on the well-known convolutional neural network (CNN), many self-attention-based or transformer-based experiment attempts have been conducted. They have shown promising performance on vision problems. A powerful baseline model based on the swin transformer adopts the shifted window approach. It enhances the capability by restricting the model to compute the self-attention function only on non-superimpose local windows while enabling cross-window relations. However, the architecture design is manually fixed. Therefore, the results are not achieving optimal performance. This work presents a genetic algorithm-based deep multi-route self-attention network for single image super-resolution (GA-MRSA). The genetic algorithm (GA) is introduced to discover the more suitable number of filters and layers. Experimental results demonstrate that the proposed optimization technique can produce an SR image with a maximum progressive PSNR of 0.14 dB and an average of 0.06 dB in the testing datasets compared to the state-of-the-art.

Although deep convolutional neural networks (CNNs) have obtained outstanding performance in image super-resolution (SR), their computational cost increases geometrically as CNN models get deeper and wider. Meanwhile, the features of intermediate layers are treated equally across the channel, thus hindering the representational capability of CNNs. In this paper, we propose an attention-aware linear depthwise network for single image SR, named ALDNet. Specifically, linear depthwise convolution allows CNN-based SR models to preserve useful information for reconstructing a super-resolved image while reducing computational burden. Furthermore, we design an attention-aware branch to enhance the representation ability of depthwise convolution layers by making full use of depthwise filter interdependencies. Experiments on publicly available benchmark datasets show that ALDNet achieves superior performance to traditional depthwise separable convolutions in terms of quantitative measurements and visual quality.

Traditional works have shown that patches in a natural image tend to redundantly recur many times inside the image, both within the same scale, as well as across different scales. Make full use of these multi-scale information can improve the image restoration performance. However, the current proposed deep learning based restoration methods do not take the multi-scale information into account. In this paper, we propose a dilated convolution based inception module to learn multi-scale information and design a deep network for single image super-resolution. Different dilated convolution learns different scale feature, then the inception module concatenates all these features to fuse multi-scale information. In order to increase the reception field of our network to catch more contextual information, we cascade multiple inception modules to constitute a deep network to conduct single image super-resolution. With the novel dilated convolution based inception module, the proposed end-to-end single image super-resolution network can take advantage of multi-scale information to improve image super-resolution performance. Experimental results show that our proposed method outperforms many state-of-the-art single image super-resolution methods.

Deep convolutional neural networks have been widely used in single image super-resolution (SISR) with great success. However, the performance and efficiency of such models need to be improved for practical applications. In this paper, a novel deep quantification down-plain-upsampling (QDPU) network for SISR is proposed. In the framework, a down-plain-upsampling (DPU) residual block based on U-Net is firstly designed to reduce the computational cost by transforming the spatial scale of feature maps without sacrificing the reconstruction performance. Then, to better transmit low-level features to the reconstruction layer, we construct quantification skip-connection modules to integrate the outputs of the DPU residual blocks. Finally, QDPU is developed by stacking the DPU residual blocks with multiple skip-connections to reconstruct high-resolution images and reduce the computational burden. Quantitative and qualitative evaluations of the reconstruction results on four benchmark datasets show that the proposed method can achieve better performance compared with several state-of-the-art SISR methods.

The success of modern single image super-resolution (SISR) algorithms is inspired by the development of deep convolutional neural networks (CNNs). However, these CNN-based methods require considerable computation and complexity, making it impossible for these methods to perform real-time calculations in edge devices. Thus, lightweight model design has become a development trend in the super-resolution field, including pruning, quantization, and other methods. The 1-bit quantization is an extreme lightweight method which can reduce the calculation amount of the model in an extreme manner and is friendly to hardware such as edge devices. Most existing binary quantization approaches lead to a large information loss during forward propagation, especially in detailed color information (e.g., edge, texture, and contrast). The loss of color information makes modern binary methods unsuitable for SISR tasks. We think the loss occurs because these methods typically utilize a uniform threshold to quantize the weights and activations. Thus, in this article, we thoroughly analyze the difference between normal classification tasks and SISR tasks, and present a binarization scheme based on local means. The proposed method can maintain more detailed information in feature maps using dynamic thresholds during quantization. Specifically, each value in the full precision activations has a corresponding threshold during the quantization process, and those thresholds are determined by the full precision values of the surroundings. In addition, a gradient approximator is introduced to adaptively optimize the gradient for updating binary weights. We then verify the effectiveness of our method for training binary networks on several SISR benchmarks including VDSR and SRResNet. Experimental results show that the proposed method can outperform the state-of-the-art algorithms to obtain binary networks for image super-resolution with better peak signal-to-noise ratio (PSNR) values and visual quality.

Deep convolutional neural networks (CNN) have achieved excellent performance in the single image super-resolution (SISR) task. Blindly stacking network depth cannot effectively utilize the shallow features and equal treatment of features from different stages seriously hinders network’s representational ability. To address the problems, we present a novel network structure called pyramid channel attention combined adaptive multi-column network (PCA-AMCN) for SISR. Specifically, adaptive multi-column block (AMCB) is proposed to extract local features and adaptively handle the interlayer and interlayer features, which contains three columns made up from different convolution kernels, and each column is weighted by a learnable adaptive parameter in the same receptive field and eventually fused by residual skip connection. Furthermore, pyramid channel attention (PCA) is designed to implement channel attention mechanism, which can progressively infer attention weights more accurately through the pyramid structure. The experimental results on benchmark datasets demonstrate the superiority of PCA-AMCN in comparison with other state-of-the-art methods.

A problem of single image super-resolution is considered, where the goal is to recover one high-resolution image from one low-resolution image. Whereas this problem has been successfully solved so far by the known VDSR network, such an approach still cannot give an overall beneficial effect compared to bicubic interpolation. This is so due to the fact that the image reconstruction quality has been estimated separately by three subjective factors. Moreover, the original VDSR network consisting of 20 convolutional layers is apparently not optimal by its depth. This is why here those factors are aggregated, and the network performance is deemed by a single estimator. Then the depth is tried to be decreased (truncation) along with adjusting the learning rate drop factor. Finally, a plausible improvement of the VDSR network is confirmed. The best truncated network, performing by almost 3.2 % better than bicubic interpolation, occupies less memory space and is about 1.44 times faster than the original VDSR network for images of a medium size.

Single-image super-resolution (SR) is vital in all areas of computer vision, due to the capability of the technology to generate high-resolution (HR) images. Conventional SR approaches do not consider high-frequency detail information during the reconstruction, resulting in high-frequency details of the image unreal, distorted in the reconstructed SR image. In this study, a novel detail-enhanced wavelet residual network (DeWRNet) is proposed to individually deal with the low- and high-frequency of sub-images and resolve the problem of the details over smooth with a novel low-to-high frequency transmission (L2HFT) and detail enhancement (DE) mechanism. Unlike traditional SR approaches, which directly predict high-resolution images, the proposed DeWRNet decomposes the image into low- and high-frequency ones through stationary wavelet transform, and trains low- and high-frequency sub-images with different models. Furthermore, while reconstructing high-frequency details, low-frequency structure is also provided to further restore and enhance high-frequency details by the proposed L2HFT and DE mechanism. Finally, the joint-loss function is used to optimize low- and high-frequency results in different degree of weighting. In addition to correct restoration, image details are further enhanced by adjusting different hyperparameters during training. Compared with the state-of-the-art approaches, the experimental results indicate that the proposed DeWRNet achieves a better performance and has excellent visual presentation, especially in image edges and texture details.

Single image super-resolution (SISR) with deep convolutional neural networks has recently attracted increasing attention due to its potentials to generate rich details. To obtain better fidelity and visual quality, most of existing methods are of heavy design with the depth of network. However, the lack of high-frequency information in the deep network and the limit of non-local context relations of pixels remain the core issue. In addition, there are many arbitrary-scale image super-resolution tasks, so it is a focus that provides an effective and efficient model for super resolution with arbitrary-scale. To this end, we propose a Feature Preserving and Enhancing Network (FPEN) based on implicit representation, which is aims to preserving high-frequency information in deep network and enhancing non-local contextual features of pixels to output more realistic images at arbitrary scales. In particular, our proposed High Frequency Preserving Block (HFPB) can divide the features into high-frequency components and low-frequency components, and allocate more operations to high-frequency components to ensure that high-frequency components can be preserved in the network. Since high-frequency information contains the details and texture of the image, it can restore the finer details of the image. Moreover, Pixel Continuity Attention (PiCA) module our proposed utilizes visual cues observed from pixels to adaptively recalibrate the pixel range that needs attention in the image to better achieve feature enhancement and generate smoother images. Extensive experiments conducted on benchmark SISR models and datasets show that Feature Preserving and Enhancing Network can be employed for various SISR tasks with arbitrary-scale to obtain the better visual quality than other state-of-the-art SR algorithms.