MSDBPN: multi-column smoothed dilated convolution based back projection network for stereo image super-resolution

With the rapid growth of Internet video image information, there is a large amount of redundancy in image data. Use less data stream information to transfer the image or the amount of information contained in the image. Its purpose is to reduce the redundancy of images, so as to store them at low bit rate and reduce the data storage space. In the general image compression method, the hybrid coding framework is adopted. Each algorithm adopts a fixed algorithm mode through a specific design algorithm, without global optimization. Image compression is mainly divided into prediction, transformation, quantization, digital entropy coding and other steps. At present, there are many researches on super-resolution network based on deep learning technology. The main function is to reconstruct high-resolution image replace image magnification low-resolution images such as linear interpolation, which has a great performance improvement image resolution, noise reduction, deblurring and so on, but there is no effective way to use super-resolution network applications to improve quality of compression reconstructed image quality. This paper involves a new method that using image super-resolution residual learning network to improve quality of compression image, our method, the reduced image is encoded into a content stream and a transmission corresponding parameter is encoded into a model stream. Firstly, the original image is scaled down 1/2 size of source image, then encode the small image into content stream with the existing codec. Secondly, the residual learning super-resolution (SR) network is used for image filtering to scale up reconstructed image with decode image resizing method and boost the quality of edge feature extraction of image. Our results show that there is significant performance improvement of h265 in low resolution reconstructed image (bits-per-pixel less than 0.1).

Adversarially trained deep neural networks have significantly improved performance of single image super resolution, by hallucinating photorealistic local textures, thereby greatly reducing the perception difference between a real high resolution image and its super resolved (SR) counterpart. However, application to medical imaging requires preservation of diagnostically relevant features while refraining from introducing any diagnostically confusing artifacts. We propose using a deep convolutional super resolution network (SRNet) trained for (i) minimising reconstruction loss between the real and SR images, and (ii) maximally confusing learned relativistic visual Turing test (rVTT) networks to discriminate between (a) pair of real and SR images (T1) and (b) pair of patches in real and SR selected from region of interest (T2). The adversarial loss of T1 and T2 while backpropagated through SRNet helps it learn to reconstruct pathorealism in the regions of interest such as white blood cells (WBC) in peripheral blood smears or epithelial cells in histopathology of cancerous biopsy tissues, which are experimentally demonstrated here. Experiments performed for measuring signal distortion loss using peak signal to noise ratio (pSNR) and structural similarity (SSIM) with variation of SR scale factors, impact of rVTT adversarial losses, and impact on reporting using SR on a commercially available artificial intelligence (AI) digital pathology system substantiate our claims.

Nonhomogeneous haze removal is a challenging problem, which does not follow the physical scattering model of haze. Numerous existing methods focus on homogeneous haze removal by generating transmission map of the image, which is not suitable for nonhomogeneous dehazing tasks. Some methods use end-to-end model but are also designed for homogeneous haze. Inspired by Knowledge Transfer Dehazing Network and Trident Dehazing Network, we propose a model with super-resolution method and knowledge transfer method. Our model consists of a teacher network, a dehaze network and a super-resolution network. The teacher network provides the dehaze network with reliable prior, the dehaze network focuses primarily on haze removal, and the super-resolution network is used to capture details in the hazy image. Ablation study shows that the super-resolution network has significant benefit to image quality. And comparison shows that our model out- performs previous state-of-the-art methods in terms of perceptual quality on NTIRE2021 NonHomogeneous Dehazing Challenge dataset, and also performs well on other datasets.

In many practical situations, images are not only down sampled but also compressed for efficient transmission and storage. JPEG and MPEG-2 compressions often introduce blocking artifacts because they process the data as 8 × 8 blocks. Many of the existing super resolution (SR) methods assume low resolution images as a down sampled version of high resolution (HR) image, and neglect the degradation due to compression. This exacerbates artifacts in the SR image and reduces the user experience. To address the joint deblocking and SR (DbSR), a novel deep network with dense skip connections and dilated convolutions is proposed in this paper, and we name it as DenseDbSR. Recently, many researchers have proposed deeper networks and achieved improvement in the SR performance. However, training deeper networks is very challenging because of the problem of vanishing gradients. Simply increasing the depth of the network leads to cumbersome computational costs. To enlarge the field-of-view (FOV) without increasing the computational cost, the dilated convolution is used. The dilated convolution exponentially expands the FOV and helps to exploit the contextual information efficiently. Moreover, the dense skip connections create short paths for gradients to be back-propagated efficiently and alleviates the problem of vanishing gradients. Furthermore, the network is relieved from the training burden by learning residuals of the SR image instead of learning raw images. From the conducted extensive experimentation, the proposed DenseDbSR network produced better performance in terms of PSNR and SSIM than the compared state-of-the-art methods.

Abstract. As deep learning models advance, their use in Super Resolution (SR) tasks has become pivotal for enhancing remote sensing low-resolution (LR) satellite images. However, the decision making processes within these models remain opaque, especially in remote sensing applications where transparency is critical. This paper focuses on applying Explainable Artificial Intelligence (XAI) techniques, particularly Local Attribution Maps (LAMs), to analyze and interpret the internal behavior of both general purpose and remote sensing specific SR neural networks. General purpose models like Generative Adversarial Network for Super Resolution (SRGAN), Enhanced Deep Super-Resolution Network (EDSR), Efficient Super-Resolution Transformer (ESRT), and Hybrid-Attention Transformer (HAT), although highly effective in SR tasks, were originally designed for broader image enhancement challenges; in contrast, Hybrid-Scale Self-Similarity Exploitation Network (HSENet) and Multi-scale Enhanced Network (MEN) are tailored for the unique complexities of remote sensing, such as varied textures and intricate scene features. By leveraging LAMs, we highlight how different networks prioritize and process features such as edges, textures, and high frequency details to generate super resolved outputs. The comparative study between general purpose and remote sensing specific networks outlines each model’s strengths and weaknesses in managing the data present in remote sensing imagery. This approach addresses the previously unexplored application of LAMs in remote sensing for SR, contributing to the research in this field and providing deeper insights into the interpretability and transparency of widely used SR models. Furthermore, by drawing parallels between feature attribution in SR and classification tasks, we suggest new pathways for integrating semantic information to refine model transparency and performance in remote sensing applications.

Despite breakthrough advances in image super-resolution (SR) with convolutional neural networks (CNNs), SR has yet to enjoy ubiquitous applications due to the high computational complexity of SR networks. Quantization is one of the promising approaches to solve this problem. However, existing methods fail to quantize SR models with a bit-width lower than 8 bits, suffering from severe accuracy loss due to fixed bit-width quantization applied everywhere. In this work, to achieve high average bit-reduction with less accuracy loss, we propose a novel Content-Aware Dynamic Quantization (CADyQ) method for SR networks that allocates optimal bits to local regions and layers adaptively based on the local contents of an input image. To this end, a trainable bit selector module is introduced to determine the proper bit-width and quantization level for each layer and a given local image patch. This module is governed by the quantization sensitivity that is estimated by using both the average magnitude of image gradient of the patch and the standard deviation of the input feature of the layer. The proposed quantization pipeline has been tested on various SR networks and evaluated on several standard benchmarks extensively. Significant reduction in computational complexity and the elevated restoration accuracy clearly demonstrate the effectiveness of the proposed CADyQ framework for SR. Codes are available at https://github.com/Cheeun/CADyQ.

In this work, a simple and compact-structured convolutional neural network (CNN) entitled as Denoising and Super-Resolution (DnSR) network for single image denoising and super resolution (SISR) is presented. Using sparsity property (a smaller number of large intensity values and a larger number of small intensity values), the training of DnSR is made to converge early. We obtain the sparsity by learning residuals of image pixels instead of learning raw image pixels particularly for SR. We use a smaller receptive field of the image patch for faster training. We use gradient clipping to train DnSR net with an extremely large learning rate to overcome gradient inconsistency during training. An end-to-end mapping is done by learning the complex mapping between low-resolution (LR) image patches and the corresponding residual of high resolution (∆HR) image patches using DnSR net. The proposed DnSR net uses a smaller number of parameters (approximately 3 times lesser) than the existing state-of-the-art (SOTA) SRCNN method. The proposed DnSR provides superior reconstruction accuracy with less artifacts when compared to other SOTA methods. The performance of the DnSR is assessed qualitatively and quantitatively using different image quality metrics.

An efficient multi-scale learning method for image super-resolution networks

Efficient image super-resolution (SR) has witnessed rapid progress thanks to novel lightweight architectures or model compression techniques (e.g., neural architecture search and knowledge distillation). Nevertheless, these methods consume considerable resources or/and neglect to squeeze out the network redundancy at a more fine-grained convolution filter level. Network pruning is a promising alternative to overcome these shortcomings. However, structured pruning is known to be tricky when applied to SR networks because the extensive residual blocks demand the pruned indices of different layers to be the same. Besides, the principled determination of proper layerwise sparsities remains challenging too. In this paper, we present Global Aligned Structured Sparsity Learning (GASSL) to resolve these problems. GASSL has two major components: Hessian-Aided Regularization (HAIR) and Aligned Structured Sparsity Learning (ASSL). HAIR is a regularization-based sparsity auto-selection algorithm with Hessian considered implicitly. A proven proposition is introduced to justify its design. ASSL is for physically pruning SR networks. Particularly, a new penalty term Sparsity Structure Alignment (SSA) is proposed to align the pruned indices of different layers. With GASSL, we design two new efficient single image SR networks of different architecture genres, pushing the efficiency envelope of SR models one step forward. Extensive results demonstrate the merits of GASSL over other recent counterparts.

Efficient masked feature and group attention network for stereo image super-resolution

B2GAN: Bidirectional-branch generative adversarial network for text image super-resolution with structure preservation

Compared to single-modality magnetic resonance (MR) image super-resolution (SR) methods, multi-modality MR image methods can utilize high-resolution reference modality (e.g., T1 modality) to provide valuable complementary information for low-resolution target modality (e.g., T2 modality) in SR reconstruction, which can further improve the quality of the SR images. Although they have achieved impressive results, these methods still suffer from the following drawbacks: (1) They can only handle fixed integer upsampling factors, such as 2X, 3X, and 4X, and require training and storing corresponding models for each upsampling factor, which is infeasible in clinical practice; (2) They only perform feature extraction and reconstruction in the image domain. However, the aliasing artifacts produced in the image domain are structural and non-local. Therefore, using only the image domain cannot effectively reconstruct high-quality aliasing-free SR images. To address these issues, we develop a brand-new Dual-Domain Implicit Network (DuDoINet) for multi-modality MR image arbitrary-scale SR. Specifically, we propose a dual-domain learning scheme for multi-modality MR image SR, which allows the network to sufficiently exploit the frequency and image domain information in MR images. In addition, we design implicit attention to achieve arbitrary-scale upsampling of MR images, which utilizes a continuously differentiable function that generates pixel values from pixel coordinates. Furthermore, we designed a deformable cross-modality attention mechanism that can adaptively transfer high-frequency details from the T1 to the T2 modality, better integrating valuable complementary information from the T1 modality. Extensive and comprehensive experiments on healthy subjects and patient datasets demonstrate that our DuDoINet outperforms SOTA methods, demonstrating its great potential for clinical practice.

DHBSR: A deep hybrid representation-based network for blind image super resolution

Super-resolution (SR) has attracted increasing attention as it can improve the quality of video satellite images. Most previous studies only consider several integer magnification factors and focus on obtaining a specific SR model for each scale factor. However, in the real world, it is a common requirement to zoom the videos arbitrarily by rolling the mouse wheel. In this article, we propose a unified network for arbitrary scale SR (ASSR) of video satellite images. The proposed ASSR consists of two modules, i.e., feature learning module and arbitrary upscale module. The feature learning module accepts multiple low-resolution (LR) frames and extracts useful features of those frames by using many 3-D residual blocks. The arbitrary upscale module takes the extracted features as input and enhances the spatial resolution by subpixel convolution and bicubic-based adjustment. Different from existing video satellite image SR methods, ASSR can continuously zoom LR video satellite images with arbitrary integer and noninteger scale factors in a single model. Experiments have been conducted on real video satellite images acquired by Jilin-1 and OVS-1. Quantitative and qualitative results have demonstrated that ASSR has superior reconstruction performance compared with the state-of-the-art SR methods.

Due to the limitation of the imaging system, it is hard to get Hyperspectral Image (HSI) with very high spatial resolution. Super-Resolution (SR) is a handling missing data technology to restore high-frequency information from the low-resolution image, can be used to solve this problem. Recently, Deep Learning (DL) has achieved great performance in computer vision, including SR. However, most DL-based HSI SR methods neglect the spectral disorder caused by normal 2D convolution. This paper proposes a novel end–end deep learning-based network named Separable-Spectral and Inception Network (SSIN) for HSI SR. In SSIN, the feature extraction module independently extracts features of each band image, and then these features are fused together to further exploit residual image by using feature fusion module. In reconstruction module, a multi-path connection is built to obtain features of different levels to restore high spatial resolution image in a coarse-to-fine manner. Experiments are implemented on two datasets include both indoor and airborne HSIs, and the performances of SSIN are evaluated in different conditions. Experimental results show that adding several separable spectral convolutions and multi-path connection in a deep network can greatly improve the SR performance, and SSIN achieves higher accuracy and better visualization compare with other methods.