Image Super-Resolution Using Deformable Convolutional Network

The topic of super-resolution image reconstruction has recently received considerable attention within the research area. Super-resolution image reconstruction methods attempt to create a single high-resolution image from a number of low-resolution images (or a video sequence). The Bayesian maximum a posteriori (MAP) estimation method is widely used for super-resolution image reconstruction. So far, existing super-resolution reconstruction methods are all confronted with the problem of slow convergence and expensive computation. To satisfy the requirement of real-time application, we propose a high-efficiency super-resolution reconstruction algorithm that solves two key bottlenecks in the multi-frame MAP framework. The first breakthrough is to select the Armijo rule to identify the step length instead of the exact line search. The second one is to approximately compute the gradient of the MAP objective function using analytic representation instead of numerical calculation. Experimental results show that the proposed algorithm is applicable to real-time super-resolution reconstruction of low-resolution image sequences.

Raman spectroscopy visualization is a challenging task due to the interference of complex background noise and the number of selected measurement points. In this paper, a super-resolution image reconstruction algorithm for Raman spectroscopy is studied to convert raw Raman data into pseudo-color super-resolution imaging. Firstly, the Raman spectrum data of a single measurement point is measured multiple times to calculate the mean value to remove the random background noise, and innovatively introduce the Retinex algorithm and the median filtering algorithm which improve the signal-to-noise ratio. The novel method of using deep neural network performs a super-resolution reconstruction operation on the gray image. An adaptive guided filter that automatically adjusts the filter radius and penalty factor is proposed to highlight the contour of the cell, and the super-resolution reconstruction of the pseudo-color image of the Raman spectrum is realized. The average signal-to-noise ratio of the reconstructed pseudo-color image sub-band reaches 14.29 db, and the average value of information entropy reaches 4.30 db. The results show that the Raman-based cell pseudo-color image super-resolution reconstruction algorithm is an effective tool to effectively remove noise and high-resolution visualization. The contrast experiments show that the pseudo-color image Kullback–Leiber (KL) entropy of the color image obtained by the method is small, the boundary is obvious, and the noise is small, which provide technical support for the development of sophisticated single-cell imaging Raman spectroscopy instruments.

Coping with the limited spatial resolution of images, which is caused by the physical limitations of image sensors, is the objective of this thesis. Initially, an effort to model the scanner function when generating a document copy by means of simple models is made. In a task of scanner function simulation the proposed model should be preferred over the Gaussian and Cauchy models met in bibliography as it is equivalent in performance, simpler in implementation and does not present any dependence on certain scanner characteristics. Afterwards, new methods for improving images spatial resolution are formulated. A nonuniform interpolation method for Super-Resolution (SR) image reconstruction is proposed. Experimentation proves that the proposed method employing Kriging interpolation predominates over the method which creates the high-resolution grid by means of the weighted nearest neighbor interpolation that is a conventional interpolation technique. Also, three new methods for stochastic regularized SR image reconstruction are presented. The Tukey error norm in combination with the Bilateral Total Variation (BTV) regularization, the Lorentzian error norm in combination with the BTV regularization and the Huber error norm combined with the BTV regularization are the three proposed methods. An additional novelty is the direct comparison of the three estimators Tukey, Lorentzian and Huber in the task of super-resolution image reconstruction, thus in rejecting outliers. The performance of the proposed methods proves superior to that of a regularized SR technique met in bibliography. Experimental results verify the robust statistics theory. Moreover, a novel study which considers the effect of each one of the data-fidelity and regularization terms on the SR image reconstruction result is carried out. The conclusions reached help to select an effective SR image reconstruction method, among several potential ones, for a given low-resolution sequence of frames. Finally, an image interpolation method employing a neural network is proposed. The presented training procedure results in the network learning the scanner point spread function. Experimental results prove that the proposed technique predominates over the classical algorithms of bicubic and spline interpolation. The proposed method is novel as it treats, for the first time, the issue of the training data presentation order to the neural network input.

It is difficult for laser scanning confocal microscopy to obtain high- or ultra-high-resolution laser confocal images directly, which affects the deep mining and use of the embedded information in laser confocal images and forms a technical bottleneck in the in-depth exploration of the microscopic physiological and biochemical processes of plants. The super-resolution reconstruction model (SRGAN), which is based on a generative adversarial network and super-resolution reconstruction model (SRResNet), which is based on a residual network, was used to obtain single and secondary super-resolution reconstruction images of laser confocal images of the root cells of the hyperaccumulator Solanum nigrum. Using the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and mean opinion score (MOS), the models were evaluated by the image effects after reconstruction and were applied to the recognition of endocytic vesicles in Solanum nigrum root cells. The results showed that the single reconstruction and the secondary reconstruction of SRGAN and SRResNet improved the resolution of laser confocal images. PSNR, SSIM, and MOS were clearly improved, with a maximum PSNR of 47.690. The maximum increment of PSNR and SSIM of the secondary reconstruction images reached 21.7% and 2.8%, respectively, and the objective evaluation of the image quality was good. However, overall MOS was less than that of the single reconstruction, the perceptual quality was weakened, and the time cost was more than 130 times greater. The reconstruction effect of SRResNet was better than that of SRGAN. When SRGAN and SRResNet were used for the recognition of endocytic vesicles in Solanum nigrum root cells, the clarity of the reconstructed images was obviously improved, the boundary of the endocytic vesicles was clearer, and the number of identified endocytic vesicles increased from 6 to 9 and 10, respectively, and the mean fluorescence intensity was enhanced by 14.4% and 7.8%, respectively. Relevant research and achievements are of great significance for promoting the application of deep learning methods and image super-resolution reconstruction technology in laser confocal image studies.

Spatial resolution of image is a critical index of image quality, and also is an important parameter in the area of image application. However, there are many factors which can lower and degenerate the quality of images in the process of capturing images. An effective solution to this problem is the super-resolution image reconstruction, that means reconstructing a high resolution (HR) image from some low resolution (LR) images, and eliminating the additive noise and blur which is caused by the size of the detector and optical at the same time. In this paper,one of project of convex set method for reconstructing a super-resolution image from multiframe undersampling images is introduced.First,a very important factor-the model of the process of imaging in spatial domain iterative method for super-resolution image reconstruction is mentioned. Next the reconstruction model and its steps are introduced through introducing the theory of project of convex set method. At last, the technique is available by experiments. The result indicates the project of convex set method for super-resolution image reconstruction has preferable effect on multiframe undersampling remote sensing images, and the technique of super-resolution image reconstruction can used in lots of fields such as remote sensing, computer vision, public security, physic imaging and so on.

Medical image super-resolution reconstruction algorithms based on deep learning: A survey

Current image super-resolution (SR) reconstruction methods based on deep learning focus more on using the learning capabilities of deeper networks to obtain better reconstruction results, but the increase in depth will increase the complexity of the network structure and the difficulty of training. To address this problem, considering the advantages of residual learning that can overcome the problem of low learning efficiency, and the ability of multiscale networks that can extract global and local features at different scales, this paper proposes a multi-scale residual block (MSRB) for feature learning. Based on the constructed MSRB, a cascaded multi-scale residual network (CMSRN) is developed for image SR reconstruction. In the network, to reconstruct richer image texture details, multiple multi-scale residual blocks are cascaded to construct the residual feature learning part. Experimental results on four datasets show that the proposed network can obtain better reconstruction results, and is superior to state-of-the-art SR reconstruction methods in terms of subjective observation and objective quantitative evaluation.

Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network

A novel image super-resolution algorithm based on multi-scale dense recursive fusion network

Research shows that deep learning algorithms can effectively improve a single image's super-resolution quality. However, if the algorithm is solely focused on increasing network depth and the desired result is not achieved, difficulties in the training process are more likely to arise. Simultaneously, the function space that can be transferred from a low-resolution image to a high-resolution image is enormous, making finding a satisfactory solution difficult. In this paper, we propose a deep learning method for single image super-resolution. The MDRN network framework uses multi-scale residual blocks and dual learning to fully acquire features in low-resolution images. Finally, these features will be sent to the image reconstruction module to restore high-quality images. The function space is constrained by the closed-loop formed by dual learning, which provides additional supervision for the super-resolution reconstruction of the image. The up-sampling process includes residual blocks with short-hop connections, so that the network focuses on learning high-frequency information, and strives to reconstruct images with richer feature details. The experimental results of ×4 and ×8 super-resolution reconstruction of the image show that the quality of the reconstructed image with this method is better than some existing experimental results of image super-resolution reconstruction in subjective visual effects and objective evaluation indicators.

Image super-resolution (SR) reconstruction technology can improve the quality of low-resolution (LR) images. There are many available deep learning networks different from traditional machine learning algorithms. However, these networks are usually prone to poor performance on complex computation, vanishing gradients, and loss of useful information. In this work, we propose a sub-pixel convolutional neural network (SPCNN) for image SR reconstruction. First, to reduce the strong correlation, the RGB mode was translated into YCbCr mode, and the Y channel data was chosen as the input LR image. Meanwhile, the LR image was chosen as the network input to reduce computation instead of the interpolation reconstructed image as used in the super-resolution convolutional neural network (SRCNN). Then, two convolution layers were built to obtain more features, and four non-linear mapping layers were used to achieve different level features. Furthermore, the residual network was introduced to transfer the feature information from the lower layer to the higher layer to avoid the gradient explosion or vanishing gradient phenomenon. Finally, the sub-pixel convolution layer based on up-sampling was designed to reduce the reconstruction time. Experiments on three different data sets proved that the proposed SPCNN performs superiorly to the Bicubic, sparsity constraint super-resolution (SCSR), anchored neighborhood regression (ANR), and SRCNN methods on reconstruction precision and time consumption.

With the presentation of super-resolution convolutional neural network, deep learning approach was applied to image super-resolution reconstruction for the first time. By using convolutional neural network, the deep learning approaches can directly learn the mapping relationship between the low-resolution image and high-resolution image, and have achieved better reconstruction effects than the traditional image super-resolution reconstruction methods. Subsequently, a series of improved deep learning approaches have been proposed, and the reconstruction effects have been improved continuously. This paper systematically summa rizes the image super-resolution reconstruction approaches based on deep learning, analyzes the characteristics of different models, and compares the main deep learning models based on the experiments. Furthermore, based on deep learning model, the future research directions of the image super-resolution reconstruction methods based on deep learning models are reasonably predicted.

Due to the factors like processing power limitations and channel capabilities images are often down sampled and transmitted at low bit rates resulting in a low resolution compressed image. High resolution images can be reconstructed from several blurred, noisy and down sampled low resolution images using a computational process know as super resolution reconstruction. Super-resolution is the process of combining multiple aliased low-quality images to produce a high resolution, high-quality image. The problem of recovering a high resolution image progressively from a sequence of low resolution compressed images is considered. In this paper we propose a novel DCT based progressive image display algorithm by stressing on the encoding and decoding process. At the encoder we consider a set of low resolution images which are corrupted by additive white Gaussian noise and motion blur. The low resolution images are compressed using 8 by 8 blocks DCT and noise is filtered using our proposed novel zonal filter. Multiframe fusion is performed in order to obtain a single noise free image. At the decoder the image is reconstructed progressively by transmitting the coarser image first followed by the detail image. And finally a super resolution image is reconstructed by applying our proposed novel adaptive interpolation technique. We have performed both objective and subjective analysis of the reconstructed image, and the resultant image has better super resolution factor, and a higher ISNR and PSNR. A comparative study done with Iterative Back Projection (IBP) and Projection on to Convex Sets (POCS),Papoulis Grechberg, FFT based Super resolution Reconstruction shows that our method has out performed the previous contributions.

Despite advances in digital education making online oil painting lessons feasible, clarity and accuracy are still issues. Videos of low resolution or inadequate clarity make brushwork and texture, two essential elements of oil painting, difficult to discern. This work suggests an innovative deep learning-based online oil painting teaching framework (IDL-OOPTF) with picture super-resolution reconstruction to address this problem. The technique enhances real-time painting demos and student participation by utilizing a Convolutional Neural Network (CNN) model for semantic scene interpretation and an enhanced super-resolution generative adversarial network (SRGAN). Because of this invention, students can see finer details and color gradients in low-resolution television broadcasts. The proposed model outperforms baseline SR techniques and traditional interpolation on PSNR, SSIM, and customer satisfaction measures using a carefully chosen collection of oil painting pictures and instructional videos. Notable results include 35% more student engagement and 28% more detail recognition. Ultimately, this method enhances the pedagogical and aesthetic quality of online oil painting classes, addressing the issue of distant art education on a large scale. Customization using reinforcement learning and adaptive feedback are being developed.

Aiming at the problems of texture distortion and fuzzy details in the existing image super-resolution reconstruction methods, a super-resolution reconstruction network based on multi-channel attention mechanism is proposed. The texture extraction module designs an extremely lightweight multi-channel attention module in the network structure. Combined with one-dimensional convolution, it realizes cross-channel information interaction, focusing on important feature information; The texture restoration module introduces dense remaining blocks to restore some high-frequency texture details, improve the model performance, and generate high-quality reconstructed images. The proposed network can not only effectively improve the visual effect of the image, but the results on the benchmark data set CUFED5 are similar to the classic super-resolution (SRCNN) reconstruction method based on convolutional neural network. The peak signal-to-noise ratio (PSNR) and structure are similar. Degrees (SSIM) increased by 1.76dB and 0.062 respectively. Experimental results show that the designed network can improve the accuracy of texture migration and can effectively improve the quality of the generated image.