Context based super resolution image reconstruction

In this paper a novel direction adaptive super-resolution (SR) image reconstruction method is proposed. The proposed maximum a-posteriori (MAP) based estimator uses gradient direction for optimal noise reduction while preserving the edges. Compared to the other edge-preserving methods, the proposed algorithm uses gradient direction in addition to the gradient amplitude for optimum regularization. The method comprises a gradient amplitude and direction estimation stage where a gradient direction map is obtained. This map guides the SR reconstruction stage through iterations. Three variations of the proposed method are compared against other edge-preserving super resolution methods. PSNR (Peak signal-to-noise-ratio), SSIM (Structural similarity index measure) values, and illustrations show that the proposed method has better performance especially on image pixel values where a strong gradient is present.

Coping with the limited spatial resolution of images, which is caused by the physical limitations of image sensors, is the objective of this thesis. Initially, an effort to model the scanner function when generating a document copy by means of simple models is made. In a task of scanner function simulation the proposed model should be preferred over the Gaussian and Cauchy models met in bibliography as it is equivalent in performance, simpler in implementation and does not present any dependence on certain scanner characteristics. Afterwards, new methods for improving images spatial resolution are formulated. A nonuniform interpolation method for Super-Resolution (SR) image reconstruction is proposed. Experimentation proves that the proposed method employing Kriging interpolation predominates over the method which creates the high-resolution grid by means of the weighted nearest neighbor interpolation that is a conventional interpolation technique. Also, three new methods for stochastic regularized SR image reconstruction are presented. The Tukey error norm in combination with the Bilateral Total Variation (BTV) regularization, the Lorentzian error norm in combination with the BTV regularization and the Huber error norm combined with the BTV regularization are the three proposed methods. An additional novelty is the direct comparison of the three estimators Tukey, Lorentzian and Huber in the task of super-resolution image reconstruction, thus in rejecting outliers. The performance of the proposed methods proves superior to that of a regularized SR technique met in bibliography. Experimental results verify the robust statistics theory. Moreover, a novel study which considers the effect of each one of the data-fidelity and regularization terms on the SR image reconstruction result is carried out. The conclusions reached help to select an effective SR image reconstruction method, among several potential ones, for a given low-resolution sequence of frames. Finally, an image interpolation method employing a neural network is proposed. The presented training procedure results in the network learning the scanner point spread function. Experimental results prove that the proposed technique predominates over the classical algorithms of bicubic and spline interpolation. The proposed method is novel as it treats, for the first time, the issue of the training data presentation order to the neural network input.

It is well known that an arbitrary graphical model of statistical inference defined on a tree,i.e. on a graph without loops, is solved exactly and efficiently by an iterative beliefpropagation (BP) algorithm convergent to the unique minimum of the so-called Bethe freeenergy functional. For a general graphical model on a loopy graph, the functional may showmultiple minima, the iterative BP algorithm may converge to one of the minimaor may not converge at all, and the global minimum of the Bethe free energyfunctional is not guaranteed to correspond to the optimal maximum likelihood (ML)solution in the zero-temperature limit. However, there are exceptions to this generalrule, discussed by Kolmogorov and Wainwright (2005) and by Bayati et al (2006,2008) in two different contexts, where the zero-temperature version of the BPalgorithm finds the ML solution for special models on graphs with loops. Thesetwo models share a key feature: their ML solutions can be found by an efficientlinear programming (LP) algorithm with a totally uni-modular (TUM) matrix ofconstraints. Generalizing the two models, we consider a class of graphical modelsreducible in the zero-temperature limit to LP with TUM constraints. Assuming thata gedanken algorithm, g-BP, for finding the global minimum of the Bethe freeenergy is available, we show that in the limit of zero temperature, g-BP outputsthe ML solution. Our consideration is based on equivalence established betweengapless linear programming (LP) relaxation of the graphical model in the T → 0 limit and the respective LP version of the Bethe free energy minimization.

In this study, we propose a CNN-GAN-based real-time processing technique for filtering images of underwater cables used in power systems. This addresses the excessive interference impurities that are frequently observed in images captured by remotely operated vehicles (ROVs). The process begins with the input of the original image into the convolutional neural network (CNN). Subsequently, the training outcomes, which serve as input parameters for the generative adversarial network (GAN), facilitate the filtering process. The system also calculates both the structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR), performing model updates via backward propagation. This technique utilizes deep learning technologies to achieve rapid, real-time filtering of underwater cable images. The experimental results reveal that the loss function of the CNN reaches 0.16 with an accuracy of 97.5%, while the loss function of the adversarial GAN network approaches 0.05. Compared with traditional methods such as DDN, JORDOR, RESCAN, and PRENet, the proposed CNN-GAN algorithm exhibits superior performance, as evidenced by the higher PSNR and SSIM values. Specifically, for clear water images, the PSNR reaches 29.86 dB and the SSIM is 0.9045. For severely polluted images, the PSNR is 28.67 dB and the SSIM is 0.8965, while for unevenly illuminated images, the PSNR and SSIM values are 24.37 dB and 0.88, respectively. These enhancements significantly benefit the monitoring and maintenance of power systems.

The topic of super-resolution image reconstruction has recently received considerable attention within the research area. Super-resolution image reconstruction methods attempt to create a single high-resolution image from a number of low-resolution images (or a video sequence). The Bayesian maximum a posteriori (MAP) estimation method is widely used for super-resolution image reconstruction. So far, existing super-resolution reconstruction methods are all confronted with the problem of slow convergence and expensive computation. To satisfy the requirement of real-time application, we propose a high-efficiency super-resolution reconstruction algorithm that solves two key bottlenecks in the multi-frame MAP framework. The first breakthrough is to select the Armijo rule to identify the step length instead of the exact line search. The second one is to approximately compute the gradient of the MAP objective function using analytic representation instead of numerical calculation. Experimental results show that the proposed algorithm is applicable to real-time super-resolution reconstruction of low-resolution image sequences.

The occlusion problem is a major challenge in the field of computer vision. Synthetic aperture imaging (SAI) is often used for surface reconstruction of occluded objects. However, SAI usually relies on high-speed information transmission devices. In addition, a large amount of information in the scene is lost, when handling low-resolution input images. This limitation results in unclear reconstructed regions in the synthetic aperture image and thus hinders the application of SAI in downstream tasks. We propose a multi-view super-resolution SAI method. It aims to generate high-resolution synthetic aperture images using images acquired by few of low-resolution acquisition devices. The main contributions of this paper are: (1) a multi-view super-resolution algorithm is proposed. It can generate clear synthetic aperture images in an array with a limited number of cameras. (2) By exploiting the correlation between views, the proposed algorithm can generate super-resolution synthetic aperture images with more accurate image structure and sharper image edges. (3) A feature extraction module is proposed. It can effectively extract the complementary relationship between pictures from different perspectives. The experimental results show that the proposed method can generate a reconstructed image of the occluded object surface with clear edges and accurate structure. Compared to conventional SAI, our method improves 5.7 % / 21.1 % on peak signal-to-noise ratio (PSNR)/structure similarity index measure (SSIM) and 4.4 % / 9.2 % on PSNR/SSIM respectively on two datasets compared to other state-of-the-art super-resolution methods.

The thermal camera systems can be used in all kinds of applications that require the detection of heat change, but thermal imaging systems are highly costly systems. In recent years, developments in the field of deep learning have increased the success by obtaining quality results compared to traditional methods. In this paper, thermal images of neonates (healthy - unhealthy) obtained from a high-resolution thermal camera were used and these images were evaluated as high resolution (ground truth) images. Later, these thermal images were downscaled at 1/2, 1/4, 1/8 ratios, and three different datasets consisting of low-resolution images in different sizes were obtained. In this way, super-resolution applications have been carried out on the deep network model developed based on generative adversarial networks (GAN) by using three different datasets. The successful performance of the results was evaluated with PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). In addition, healthy - unhealthy classification application was carried out by means of a classifier network developed based on convolutional neural networks (CNN) to evaluate the super-resolution images obtained using different datasets. The obtained results show the importance of combining medical thermal imaging with super-resolution methods.

With the presentation of super-resolution convolutional neural network, deep learning approach was applied to image super-resolution reconstruction for the first time. By using convolutional neural network, the deep learning approaches can directly learn the mapping relationship between the low-resolution image and high-resolution image, and have achieved better reconstruction effects than the traditional image super-resolution reconstruction methods. Subsequently, a series of improved deep learning approaches have been proposed, and the reconstruction effects have been improved continuously. This paper systematically summa rizes the image super-resolution reconstruction approaches based on deep learning, analyzes the characteristics of different models, and compares the main deep learning models based on the experiments. Furthermore, based on deep learning model, the future research directions of the image super-resolution reconstruction methods based on deep learning models are reasonably predicted.

Multi-frame Super-Resolution (SR) image reconstruction creates a single High-Resolution (HR) image from a sequence of Low-Resolution (LR) frames. Apart from resolution increment, blurring and noise removal is also achieved. In stochastic regularized methods, the SR problem is formulated by means of two terms, the data-fidelity term and the regularization term. In the present work, a novel estimator named Var − norm has been proposed for utilization in the data-fidelity term. This estimator presents a simple mathematical form based on the variance of the SR estimation error, i.e. on the difference between the estimated LR frame and the corresponding measured LR frame. The introduced Var − norm estimator is combined with the Bilateral Total Variation (BTV) regularization to formulate a novel SR method. The SR performance of the proposed method is directly compared with that of two SR techniques existing in the literature. Experimentation proves that the proposed method outperforms the existing methods.

<bold>Background</bold>Synchrotron radiation nano-computed tomography (nano-CT) is widely applied in materials science, energy research, and geology due to its capability to provide high-resolution three-dimensional structural information at the nanoscale. However, environmental interference, equipment limitations, and light source fluctuations often degrade image quality, leading to reduced signal-to-noise ratio, blurring, and loss of critical details, which severely impact quantitative analysis and segmentation accuracy for microstructures such as cracks and pores. While existing super-resolution (SR) techniques for natural images have significant advantages, their adaptation to nano-CT remains challenged by dataset scarcity and stringent reconstruction accuracy requirements.<bold>Purpose</bold>This study aims to develop a dedicated super-resolution reconstruction network for synchrotron radiation nano-CT images to enhance the clarity and detail restoration of low-quality CT images and investigate its efficacy in optimizing downstream tasks such as crack segmentation.<bold>Methods</bold>Firstly, a Transformer-based U-shaped symmetric network for SR (UTSR) was proposed, integrating a Double Convolutional Feedforward Network (DCFFN) and a Grouped Multi-scale Window Self-Attention (GMSA) mechanism to achieve efficient reconstruction through multi-level feature extraction and hierarchical processing. Then, a transfer learning strategy was adopted, leveraging natural image datasets for pre-training and fine-tuning the model with nano-CT data from the Shanghai Synchrotron Radiation Facility (SSRF) to address dataset scarcity. Finally, a Sobel edge loss function was introduced to prioritize texture preservation, and a two-stage training framework (general image pre-training and nano-CT fine-tuning) was implemented to optimize model performances in terms of Natural Image Quality Evaluator (NIQE), Learned Perceptual Image Patch Similarity (LPIPS), Deep Image Structure and Texture Similarity (DISTS), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM).<bold>Results</bold>Comparison results show that UTSR demonstrates superior performance on both simulated and nano-CT datasets: it achieves lower NIQE (9.38) and LPIPS (0.312) scores compared to SwinIR and Real-ESRGAN, alongside reduced computational complexity (45.72G FLOPs). When applied to crack segmentation, UTSR-processed images attain an accuracy of 99.3%, with a 24.5% improvement in recall and a 19.1% increase in F1 Score over original images. Ablation experiments confirm the effectiveness of the DCFFN module, where its combination with GMSA further reduces NIQE to 10.99.<bold>Conclusions</bold>The UTSR network effectively addresses the challenges of low-quality synchrotron radiation nano-CT images and scarce datasets through its innovative architecture and transfer learning strategy, providing a robust preprocessing solution for microstructural analysis, hence expands the application of deep learning-based super-resolution in scientific imaging, offering substantial academic and engineering value.

In this paper, trade-off between effect of registration error and number of images used in the process of super resolution image reconstruction is studied. Super Resolution image reconstruction is three phase process, of which registration is of at most importance. Super resolution image reconstruction uses set of low resolution images to reconstruct high resolution image during registration. The study demonstrates the effects of registration error and benefit of more number of low resolution images on the quality of reconstructed image. Study reveals that the registration error degrades the reconstructed image and without better registration methodology, a better super resolution method is still not of any use. It is noticed that without further improvement in the registration technique, not much improvement can be achieved by increasing number of input low resolution images.

Super-resolution of cardiac magnetic resonance images using Laplacian Pyramid based on Generative Adversarial Networks

Feedback attention network for cardiac magnetic resonance imaging super-resolution

This work investigates three penalized-likelihood expectation maximization (EM) algorithms for image reconstruction with Poisson data where the images are known ap riorito be sparse in the space domain. The penalty functions considered are the � 1 norm, the � 0 “norm,” and a penalty function based on the sum of logarithms of pixel values, R(x )= � n p j=1 log � xj δ +1 � . Our results show that the � 1 penalized algorithm reconstructs scaled versions of the maximum-likelihood (ML) solution, which does not improve the sparsity over the traditional ML estimate. Due to the singularity of the Poisson log-likelihood at zero, the � 0 penalized EM algorithm is equivalent to the maximum-likelihood EM algorithm. We demonstrate that the penalty based on the sum of logarithms produces sparser images than the ML solution. We evaluated these algorithms using experimental data from a position-sensitive Compton-imaging detector, where the spatial distribution of photon-emitters is known to be sparse.

Spatial resolution of image is a critical index of image quality, and also is an important parameter in the area of image application. However, there are many factors which can lower and degenerate the quality of images in the process of capturing images. An effective solution to this problem is the super-resolution image reconstruction, that means reconstructing a high resolution (HR) image from some low resolution (LR) images, and eliminating the additive noise and blur which is caused by the size of the detector and optical at the same time. In this paper,one of project of convex set method for reconstructing a super-resolution image from multiframe undersampling images is introduced.First,a very important factor-the model of the process of imaging in spatial domain iterative method for super-resolution image reconstruction is mentioned. Next the reconstruction model and its steps are introduced through introducing the theory of project of convex set method. At last, the technique is available by experiments. The result indicates the project of convex set method for super-resolution image reconstruction has preferable effect on multiframe undersampling remote sensing images, and the technique of super-resolution image reconstruction can used in lots of fields such as remote sensing, computer vision, public security, physic imaging and so on.