Image Super-Resolution via Generative Adversarial Network Using an Orthogonal Projection

Higher performance is the eternal purpose for super-resolution (SR) methods to pursue. Since the deep convolution neural network is introduced into this issue successfully, many SR methods have achieved impressive results. To further improve the accuracy that current SR methods have achieved, we propose a high-accuracy deep convolution network (HDCN). In this article, deeper network structure is deployed for reconstructing images with a fixed upscaling factor and the magnification becomes alternative by cascading HDCN. \(L_2\) loss function is substituted by a more robust one for reducing the blurry prediction. In addition, gradual learning is adopted for accelerating the rate of convergence and compacting the training process. Extensive experiment results prove the effectiveness of these ingenious strategies and demonstrate the higher-accuracy of proposed model among state-of-the-art SR methods.

The thermal camera systems can be used in all kinds of applications that require the detection of heat change, but thermal imaging systems are highly costly systems. In recent years, developments in the field of deep learning have increased the success by obtaining quality results compared to traditional methods. In this paper, thermal images of neonates (healthy - unhealthy) obtained from a high-resolution thermal camera were used and these images were evaluated as high resolution (ground truth) images. Later, these thermal images were downscaled at 1/2, 1/4, 1/8 ratios, and three different datasets consisting of low-resolution images in different sizes were obtained. In this way, super-resolution applications have been carried out on the deep network model developed based on generative adversarial networks (GAN) by using three different datasets. The successful performance of the results was evaluated with PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). In addition, healthy - unhealthy classification application was carried out by means of a classifier network developed based on convolutional neural networks (CNN) to evaluate the super-resolution images obtained using different datasets. The obtained results show the importance of combining medical thermal imaging with super-resolution methods.

In recent years, research on increasing the spatial resolution and enhancing the quality of satellite images using the deep learning-based super-resolution (SR) method has been actively conducted. In a remote sensing field, conventional SR methods required high-quality satellite images as the ground truth. However, in most cases, high-quality satellite images are difficult to acquire because many image distortions occur owing to various imaging conditions. To address this problem, we propose an adaptive image quality modification method to improve SR image quality for the KOrea Multi-Purpose Satellite-3 (KOMPSAT-3). The KOMPSAT-3 is a high performance optical satellite, which provides 0.7-m ground sampling distance (GSD) panchromatic and 2.8-m GSD multi-spectral images for various applications. We proposed an SR method with a scale factor of 2 for the panchromatic and pan-sharpened images of KOMPSAT-3. The proposed SR method presents a degradation model that generates a low-quality image for training, and a method for improving the quality of the raw satellite image. The proposed degradation model for low-resolution input image generation is based on Gaussian noise and blur kernel. In addition, top-hat and bottom-hat transformation is applied to the original satellite image to generate an enhanced satellite image with improved edge sharpness or image clarity. Using this enhanced satellite image as the ground truth, an SR network is then trained. The performance of the proposed method was evaluated by comparing it with other SR methods in multiple ways, such as edge extraction, visual inspection, qualitative analysis, and the performance of object detection. Experimental results show that the proposed SR method achieves improved reconstruction results and perceptual quality compared to conventional SR methods.

In contrast to sophisticated means of visual su-per resolution (SR), not much work has been done in the tactile SR field. Existing tactile SR algorithms for taxel-based sensors mainly focus on enhancing the localization accuracy, and generally associate with a specific type of hardware, sometimes not applicable to generic taxel-based tactile sensors. Inspired by image SR, we investigate the tactile pattern SR in this paper, and present how to transform successful image SR schemes, e.g. Convolutional Neural Network (CNN) and Generative Adversarial Network (GAN) to serve the tactile SR. We propose two tactile SR models, i.e. TactileSRCNN and TactileSRGAN, and establish a new tactile pattern SR dataset for model learning. The ground truth of high resolution (HR) tactile patterns in the dataset is obtained via multi-sampling (i.e. overlapping reception) and registration of low resolution (LR) sensor. One key contribution of this research lies in achieving ×100 (from 3×4×4 to 40×40) times tactile pattern SR with a one-time tapping of 3-axis taxel-based sensor. Different from existing tactile SR algorithms which improves the localization accuracy of a single contact point, the proposed scheme can provide multi-point contact detection to robotic applications.

Super-resolution (SR) has become more vital, because of its capability to generate high-quality ultra-high definition (UHD) high-resolution (HR) images from low-resolution (LR) input images. Conventional SR methods entail high computational complexity, which makes them difficult to be implemented for up-scaling of full-high-definition input images into UHD-resolution images. Nevertheless, our previous super-interpolation (SI) method showed a good compromise between Peak-Signal-to-Noise Ratio (PSNR) performances and computational complexity. However, since SI only utilizes simple linear mappings, it may fail to precisely reconstruct HR patches with complex texture. In this paper, we present a novel SR method, which inherits the large-to-small patch conversion scheme from SI but uses global regression based on local linear mappings (GLM). Thus, our new SR method is called GLM-SI. In GLM-SI, each LR input patch is divided into 25 overlapped subpatches. Next, based on the local properties of these subpatches, 25 different local linear mappings are applied to the current LR input patch to generate 25 HR patch candidates, which are then regressed into one final HR patch using a global regressor. The local linear mappings are learned cluster-wise in our off-line training phase. The main contribution of this paper is as follows: Previously, linear-mapping-based conventional SR methods, including SI only used one simple yet coarse linear mapping to each patch to reconstruct its HR version. On the contrary, for each LR input patch, our GLM-SI is the first to apply a combination of multiple local linear mappings, where each local linear mapping is found according to local properties of the current LR patch. Therefore, it can better approximate nonlinear LR-to-HR mappings for HR patches with complex texture. Experiment results show that the proposed GLM-SI method outperforms most of the state-of-the-art methods, and shows comparable PSNR performance with much lower computational complexity when compared with a super-resolution method based on convolutional neural nets (SRCNN15). Compared with the previous SI method that is limited with a scale factor of 2, GLM-SI shows superior performance with average 0.79 dB higher in PSNR, and can be used for scale factors of 3 or higher.

As an important display mode of underwater environments, the sonar image has limitations on the resolution, which often leads to problems with low resolution of underwater objects. Therefore, the image super-resolution algorithm is needed to transform the images from low-resolution to high-resolution. It can improve the visual effect and contribute to subsequent processing such as 3D reconstruction and object recognition. This paper proposes a method for sonar image super-resolution based on generative adversarial networks (GAN). By comparing the super-resolution effects of various interpolation and convolutional neural network algorithms on sonar images, a Residual-in-Residual Dense Block network is employed as the generator of GAN since it has the low distortion and high perceptual quality. Because the sonar image training set does not have enough data, the generator utilizes the transfer learning on the sonar images to produce an optimized network model which is more suitable for super-resolution of sonar image. The VGG19 network is employed as the discriminator. In addition, the perceptual loss is introduced into the loss function of $$\hbox {S}^2\hbox {RGAN}$$ to further improve the perceptual quality of super-resolution images. The experimental results indicate that the proposed $$\hbox {S}^2\hbox {RGAN}$$ shows excellent performance. The generated super-resolution images of $$\hbox {S}^2\hbox {RGAN}$$ have the remarkable advantages of both lower distortion and higher perceptual quality comparing with other methods. Because $$\hbox {S}^2\hbox {RGAN}$$ focuses more on the reality and overall visual effect of super-resolution sonar images, it is suitable for various underwater situations.

Color images have a wider range of applications than gray images. There are two ways to extend the traditional super-resolution reconstruction method to color images: Super resolution reconstructs each channel of the color image individually; Change the RGB color bands into YCrCb color bands, then super-resolution reconstructs the luminance component and interpolates the chrominance components.These algorithms cannot effectively utilize the property that the edges and textures are similar in the RGB channels, and the results of those methods may lead to color artifacts. Aiming to solve these problems, we propose a new super-resolution method based on cross channel prior. First, a cross channel prior is proposed to describe the similarity of gradient in RGB channels. Then, a new super-resolution method is proposed for color images via combination of the cross channel prior and the traditional super-resolution methods. Finally, the proposed method reconstructs the color channels alternately. The experimental results show that the proposed method could effectively suppress the generation of color artifacts and improve the quality of the reconstructed images.

paper, we focus on exploring the relationship between weave patterns and their mechanical properties in woven fiber composites through Machine Learning. Specifically, we explore the interactions between woven architectures and in-plane stiffness properties through Deep Convolutional Neural Network (DCNN) and Generative Adversarial Network (GAN). Our research is important for exploring how woven composite’s pattern is related to its mechanical properties and accelerating woven composite design as well as optimization. We focus on two tasks: (1) Stiffness prediction: Predicting in-plane stiffness properties for given weave patterns. Our DCNN extracts high-level features through several convolutional and fully connected layers to determine the final predictions. (2) Weave pattern prediction: Predicting weave patterns for target stiffness properties, which can be treated as the reverse task of the first one. Due to many-to-one mapping between weave patterns and the composite properties, we utilize a Decoder Neural Network as our baseline model and compare its performance with GAN and Genetic Algorithm. We represent the weave patterns as 2D checkerboard models and use finite element analysis (FEA) to determine in-plane stiffness properties, which serve as input data for our ML framework. We show that: (1) for stiffness prediction, DCNN can predict stiffness values for a given weave pattern with relatively high accuracy (above 93%); (2) for weave pattern prediction, the GAN model gives the best prediction accuracy (above 92%) while Decoder Neural Network has the best time efficiency. HAOTIAN FENG

Rectified linear units (ReLU) are known to be effective in many deep learning methods. Inspired by linear-mapping technique used in other super-resolution (SR) methods, we reinterpret ReLU into point-wise multiplication of an identity mapping and a switch, and finally present a novel nonlinear unit, called a selection unit (SU). While conventional ReLU has no direct control through which data is passed, the proposed SU optimizes this on-off switching control, and is therefore capable of better handling nonlinearity functionality than ReLU in a more flexible way. Our proposed deep network with SUs, called SelNet, was top-5th ranked in NTIRE2017 Challenge, which has a much lower computation complexity compared to the top-4 entries. Further experiment results show that our proposed SelNet outperforms our baseline only with ReLU (without SUs), and other state-of-the-art deep-learning-based SR methods.

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.

Deep Learning image processing methods are gradually gaining popularity in a number of areas including medical imaging. Classification, segmentation, and denoising of images are some of the most demanded tasks. In this study, we aim at enhancing optic nerve head images obtained by Optical Coherence Tomography (OCT). However, instead of directly applying noise reduction techniques, we use multiple state-of-the-art image Super-Resolution (SR) methods. In SR, the low-resolution (LR) image is upsampled to match the size of the high-resolution (HR) image. With respect to image enhancement, the upsampled LR image can be considered as low quality, noisy image, and the HR image would be the desired enhanced version of it. We experimented with several image SR architectures, such as super-resolution Convolutional Neural Network (SRCNN), very deep Convolutional Network (VDSR), deeply recursive Convolutional Network (DRCN), and enhanced super-resolution Generative Adversarial Network (ESRGAN). Quantitatively, in terms of peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), the SRCNN, VDSR, and DRCN significantly improved the test images. Although the ERSGAN showed the worst PSNR and SSIM, qualitatively, it was the best one.

Brain tumors are formed due to the abnormal growth of the cells that get multiplied and become in uncontrollable perspective. Tumors can damage brain cells by pressing down the skull, which consociate begins in, which negatively affects human health. In advanced stages, a brain tumor is a more dangerous infection that cannot be relieved. Brain tumor detection and early prevention are necessary in today's world. An extreme learning machine (ELM) is a widely adopted algorithm in machine learning. It is proposed to use classification models in brain tumor imaging. This classification is based on the techniques implemented: Convolutional Neural Networks(CNN) and Generative Adversarial Networks(GAN). CNN efficiently solves the convex optimization problem and is faster, requiring less human effort. The algorithmic architecture of GAN uses two neural networks, pocking one against the other. These networks are implemented in various fields to classify brain Tumor images. The present study mainly aims to introduce a new proposed classification system of preschool children brain imaging with Hybrid Convolutional Neural Networks and with the techniques of GAN. The proposed technique is compared with the existing hybrid-CNN and hybrid-GAN techniques. The outcomes are encouraging because the loss is deduced, and the accuracy facet increases. The proposed system achieved a training accuracy of 97.8% and a validation accuracy of 89%. The outcomes of the studies show that ELM in the platform of GAN for preschool children brain imaging classification has achieved higher predictive performance than the traditional classification mechanisms in increasingly complex scenarios. Time elapsed for training brain images samples finds inference value for training samples and time elapsed value increased by 28.9855%. Approximation ratio for cost based on probability, finding Approximation ratio for low probability range is increased by 88.1%. The combination of CNN, GAN and hybrid-CNN, hybrid-GAN, and hybrid CNN + GAN, compared with the proposed hybrid system, increased Detection latency for low range learning rate by 3.31%.

The article presents an analysis of neural networks that can be used to generating web page program code from input image. As part of the research of the possibilities of modification of algorithms for generating code, the method of generating code simultaneous using the generative adversarial and recurrent neural networks based on the input image is proposed. Methods for creating source code using generative adversarial network (GAN) and recurrent neural networks (RNN) are analyzed. Using domain-specific language (DSL) for tree generation code method, it was possible to use GAN and RNN together, achieving on the average 10% productivity increase. Using GRID’s mesh, the generated code is adaptable. To implement the proposed method, software with a graphical user interface is created. Classes of GAN and RNN were considered, among which the feasibility of using a neural network with long-term short-term memory (LSTM) for analysis and forecasting of tags, blocks and HTML elements of the language in the input image was selected and substantiated. The stages of the research are presented. For the experiment datasets made on the basis of existing sites from open sources for 2018–2019 are used. The estimation of the effectiveness of the proposed method is described.

In recent years, advances in image restoration have prominently featured Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs).Specifically, in seismic image denoising, the DnCNN model, rooted in CNN, employs residual learning combined with batch normalization for a streamlined approach to denoising.Alternatively, SRGAN, leveraging GAN and super-resolution techniques, trains its generative model using a perceptual loss function, accentuating perceptual disparities between generated and authentic images to boost visual quality.Recognizing the importance of preserving salient seismic data, the Constrained-DnCNN model has been introduced, refining seismic data interpretation.Additionally, the CR-SRGAN model, targeting super-resolution of artifacts and color restoration, deviates from conventional training datasets acquired via high-resolution image interpolation and downsampling.Upon comparative analysis, the DCGAN, amalgamating features of CNN and GAN, stands out for harnessing CNN's robust feature extraction.This bolsters the model's capacity to adeptly fit real data distributions.DCGAN's potential spans aspects like multimodal fusion, model interpretability, trustworthiness, and image restoration quality, marking it a promising research avenue.

Deep convolutional neural networks (CNNs), trained on corresponding pairs of high- and low-resolution images, achieve state-of-the-art performance in single-image super- resolution and surpass previous signal-processing based approaches. However, their performance is limited when applied to real photographs. The reason lies in their train- ing data: low-resolution (LR) images are obtained by bicu- bic interpolation of the corresponding high-resolution (HR) images. The applied convolution kernel significantly differs from real-world camera-blur. Consequently, while current CNNs well super-resolve bicubic-downsampled LR images, they often fail on camera-captured LR images. To improve generalization and robustness of deep super- resolution CNNs on real photographs, we present a ker- nel modeling super-resolution network (KMSR) that incor- porates blur-kernel modeling in the training. Our pro- posed KMSR consists of two stages: we first build a pool of realistic blur-kernels with a generative adversarial net- work (GAN) and then we train a super-resolution network with HR and corresponding LR images constructed with the generated kernels. Our extensive experimental vali- dations demonstrate the effectiveness of our single-image super-resolution approach on photographs with unknown blur-kernels.