Investigating single image super-resolution algorithm with deep learning using convolutional neural network for chest digital tomosynthesis

Special Section on Enabling Technologies for Network-Based Applications

The success of modern single image super-resolution (SISR) algorithms is inspired by the development of deep convolutional neural networks (CNNs). However, these CNN-based methods require considerable computation and complexity, making it impossible for these methods to perform real-time calculations in edge devices. Thus, lightweight model design has become a development trend in the super-resolution field, including pruning, quantization, and other methods. The 1-bit quantization is an extreme lightweight method which can reduce the calculation amount of the model in an extreme manner and is friendly to hardware such as edge devices. Most existing binary quantization approaches lead to a large information loss during forward propagation, especially in detailed color information (e.g., edge, texture, and contrast). The loss of color information makes modern binary methods unsuitable for SISR tasks. We think the loss occurs because these methods typically utilize a uniform threshold to quantize the weights and activations. Thus, in this article, we thoroughly analyze the difference between normal classification tasks and SISR tasks, and present a binarization scheme based on local means. The proposed method can maintain more detailed information in feature maps using dynamic thresholds during quantization. Specifically, each value in the full precision activations has a corresponding threshold during the quantization process, and those thresholds are determined by the full precision values of the surroundings. In addition, a gradient approximator is introduced to adaptively optimize the gradient for updating binary weights. We then verify the effectiveness of our method for training binary networks on several SISR benchmarks including VDSR and SRResNet. Experimental results show that the proposed method can outperform the state-of-the-art algorithms to obtain binary networks for image super-resolution with better peak signal-to-noise ratio (PSNR) values and visual quality.

Convolutional neural networks (CNN) have given rise to a new generation of video super-resolution (SR) technique. However, most existing CNN-based video SR algorithms treat the consecutive frames as a series of feature maps, just as the procedure performed in single image SR algorithms. We propose an end-to-end three-dimensional (3-D) CNN video SR framework. The input frames are considered as a cube in our framework. 3-D convolution is performed on it to extract features along spatial and temporal dimension. Image prior knowledge, such as optical flows, is introduced in reconstruction. A combination of mean square error loss and multiscale structure similarity index (MS-SSIM) loss is used to optimize the model. Experimental results show that the proposed method reconstructs high-resolution frames with more accurate and visually pleasant structures compared with state-of-the-art video SR algorithms. We also achieve comparable PSNR/SSIM results with less computation time.

Image super-resolution based on half quadratic splitting

Existing single image super-resolution (SR) algorithms based on convolution neural networks (CNNs) have achieved usable visual results.However, they often encounter artifacts and blurring, particularly at large scaling factors (e.g., 4, 8), due to significant loss of high-frequency information.To address these challenges, we propose a novel high-low frequency differentiation dynamic Laplacian pyramid network (HFDDLap).Our approach introduces a learnable high-low frequency differentiation convolution (HLC) within high-low frequency differentiation residual channel attention blocks (HL-RCAB) to effectively capture and differentiate high-and low-frequency components, enhancing detail preservation.Additionally, we employ a dynamic deconvolution (DDC) that adaptively generates upsampling kernels based on input features, improving reconstruction accuracy by reducing feature distortion.Extensive experiments on 4 and 8 SR demonstrate that our proposed method effectively reconstructs edge details, produces satisfactory SR results and outperforms some state-of-the-art (SOTA) methods in terms of evaluation metrics.

Single image super-resolution (SR) algorithms based on joint dictionaries and sparse representations of image patches have received significant attention in the literature and deliver the state-of-the-art results. Recently, Gaussian mixture models (GMMs) have emerged as favored prior for natural image patches in various image restoration problems. In this paper, we approach the single image SR problem by using a joint GMM learnt from concatenated vectors of high and low resolution patches sampled from a large database of pairs of high resolution and the corresponding low resolution images. Covariance matrices of the learnt Gaussian models capture the inherent correlations between high and low resolution patches, which are utilized for inferring high resolution patches from given low resolution patches. The proposed joint GMM method can be interpreted as the GMM analogue of joint dictionary-based algorithms for single image SR. We study the performance of the proposed joint GMM method by comparing with various competing algorithms for single image SR. Our experiments on various natural images demonstrate the competitive performance obtained by the proposed method at low computational cost.

In recent years, keypoints-based features, such as speeded up robust features (SURF), capture the attention of researchers in the area of image copy-move forgery detection. They are fast to compute and are quite robust to most inter-processing and post-processing operations. However, they will fail to detect the forgery if not enough keypoints are detected, and that is the case when the forgery size is small. A small size forgery is possible if the object is far away from the camera, or if the image is scaled down as a post-processing step. In this paper, our focus is on detecting forgeries of small size using an improved SURF based copy-move forgery detection (CMFD) method. Our method improves keypoints detection by preprocessing the image using a single image super resolution (SISR) algorithm. The proposed approach has been evaluated and compared against original SURF through a comprehensive set of experiments using a dataset of small size forgeries. Experimental results indicate that our method outperforms SURF especially when the forgery size is small.

Many remote sensing applications require a high-resolution hyperspectral image. However, resolutions of most hyperspectral imagers are limited to tens of meters. Existing resolution enhancement techniques either acquire additional multispectral band images or use a pan band image. The former poses hardware challenges, whereas the latter has limited performance. In this paper, we present a new resolution enhancement method that only requires a color image. Our approach integrates two newly developed techniques in the area: (1) A hybrid color mapping algorithm, and (2) A Plug-and-Play algorithm for single image super-resolution. Comprehensive experiments using real hyperspectral images are conducted to validate and evaluate the proposed method.

This paper proposes a single image super resolution algorithm with the aim of satisfying three desirable characteristics, namely, high quality of the produced images, adaptability to image contents...

Nowadays, satellite images are used in various governmental applications, such as urbanization and monitoring the environment. Spatial resolution is an element of crucial impact on the usage of remote sensing imagery. As such, increasing the spatial resolution of an image is an important pre-processing step that can improve the performance of various image processing tasks, such as segmentation. Once a satellite is launched, the more practical solution to improve the resolution of its captured images is to use Single Image Super Resolution (SISR) techniques. In the recent years, Deep Convolutional Neural Networks (DCNNs) have been recognized as a highly effective tool to reconstruct a High Resolution (HR) image from its Low Resolution (LR) counterpart, which is an open problem due to the inherent difficulty of estimating the missing high frequency components. The aim of this research paper is to design and implement a satellite image SISR algorithm by estimating high frequency details through training Deep Convolutional Neural Network (DCNNs) with respect to wavelet analysis. The goal is to improve the spatial resolution of multispectral remote sensing images captured by DubaiSat-2 satellite. The accuracy of the proposed algorithm is assessed using several metrics such as Peak Signal-to-Noise Ratio (PSNR), Wavelet-based Signal-to-Noise Ratio (WSNR) and Structural Similarity Index Measurement (SSIM).

Most of the state of the art super-resolution methods use deep networks with large filter sizes. Therefore, they need to train and store a correspondingly large number of parameters, thereby making their use difficult for mobile devices applications such as recognition of individuals from selfie images. To achieve an efficient super-resolution method, we propose an Efficient Single Image Super-Resolution (ESISR) algorithm, which takes into account a trade-off among the efficiency of the deep neural network, the size of the filters, and the sharpness of the images. To that end, the method implements a novel loss function based on the Sharpness metric. This metric turns out to be more suitable for recovering the quality of the eye images. Our method drastically reduces the number of parameters when compared with Deep CNNs with Skip Connection and Network (DCSCN): from 1,754,942 to 27,209 parameters when the image size is increased by a factor of 2 (x2), from 2,170,142 to 28,654 parameters when increased by 3 (x3), and from 2,087,102 to 64,201 parameters when increased by 4 (x4). Furthermore, the proposed method maintains the sharpness quality of the images.

Gradient boosting for single image super-resolution

In this paper, we propose a novel single image super-resolution algorithm that integrates a model-based approach with self-learning deep networks. The proposed method can be adapted to low-resolution (LR) images obtained with real acquisition devices where the point spread function is Gaussian-like. By modelling natural image lines as piece-wise smooth functions and approximating the blurring kernel with B-splines, an intermediate high-resolution (HR) image can be first obtained based on Finite Rate of Innovation theory. A self-supervised deep recursive residual network is then applied to further enhance the reconstruction quality. From the simulation results, our algorithm outperforms other self-learning algorithms and achieves state-of-the-art performance.

Image inpainting, a technique of completing missing or corrupted image regions in undetected form, is an open problem in digital image processing. Inpainting of large regions using Deep Convolutional Generative Adversarial Nets (DCGAN) is a new and powerful approach. In described approaches the size of generated image and size of input image should be the same. In this paper we propose a new method where the size of input image with corrupted region can be up to 4 times larger than generated image.

Depth images play an important role and are popularly used in many computer vision tasks recently. However, the limited resolution of the depth image has been hindering its further applications. To address this problem, we propose a novel dual sparsity model based single depth image super resolution algorithm, with a single low-resolution depth image as input. We formulate this problem by combining the recently developed analysis model and synthesis model exploiting the sparsity of analyzed vectors and the sparse coefficients respectively. The analysis operator and dictionaries are trained over extensive samples separately. We show that our model clearly outperforms state-of-the-art methods on the widely used Middlebury 2007 datasets both quantitatively and visually.