Combination of single image super resolution and digital Inpainting algorithms based on GANS for robust image completion

Investigating single image super-resolution algorithm with deep learning using convolutional neural network for chest digital tomosynthesis

Special Section on Enabling Technologies for Network-Based Applications

In recent years, keypoints-based features, such as speeded up robust features (SURF), capture the attention of researchers in the area of image copy-move forgery detection. They are fast to compute and are quite robust to most inter-processing and post-processing operations. However, they will fail to detect the forgery if not enough keypoints are detected, and that is the case when the forgery size is small. A small size forgery is possible if the object is far away from the camera, or if the image is scaled down as a post-processing step. In this paper, our focus is on detecting forgeries of small size using an improved SURF based copy-move forgery detection (CMFD) method. Our method improves keypoints detection by preprocessing the image using a single image super resolution (SISR) algorithm. The proposed approach has been evaluated and compared against original SURF through a comprehensive set of experiments using a dataset of small size forgeries. Experimental results indicate that our method outperforms SURF especially when the forgery size is small.

Most of the state of the art super-resolution methods use deep networks with large filter sizes. Therefore, they need to train and store a correspondingly large number of parameters, thereby making their use difficult for mobile devices applications such as recognition of individuals from selfie images. To achieve an efficient super-resolution method, we propose an Efficient Single Image Super-Resolution (ESISR) algorithm, which takes into account a trade-off among the efficiency of the deep neural network, the size of the filters, and the sharpness of the images. To that end, the method implements a novel loss function based on the Sharpness metric. This metric turns out to be more suitable for recovering the quality of the eye images. Our method drastically reduces the number of parameters when compared with Deep CNNs with Skip Connection and Network (DCSCN): from 1,754,942 to 27,209 parameters when the image size is increased by a factor of 2 (x2), from 2,170,142 to 28,654 parameters when increased by 3 (x3), and from 2,087,102 to 64,201 parameters when increased by 4 (x4). Furthermore, the proposed method maintains the sharpness quality of the images.

Many remote sensing applications require a high-resolution hyperspectral image. However, resolutions of most hyperspectral imagers are limited to tens of meters. Existing resolution enhancement techniques either acquire additional multispectral band images or use a pan band image. The former poses hardware challenges, whereas the latter has limited performance. In this paper, we present a new resolution enhancement method that only requires a color image. Our approach integrates two newly developed techniques in the area: (1) A hybrid color mapping algorithm, and (2) A Plug-and-Play algorithm for single image super-resolution. Comprehensive experiments using real hyperspectral images are conducted to validate and evaluate the proposed method.

Image super-resolution based on half quadratic splitting

This paper proposes a single image super resolution algorithm with the aim of satisfying three desirable characteristics, namely, high quality of the produced images, adaptability to image contents...

This research was conducted to optimize the use of the convolutional neural network (CNN) method in classification of krasak river sand and white river sand in the Muntilan area, Central Java with the aim of knowing or recognizing two types of sand based on digital images, the number of datasets collected amounted to 1,305 consisting of 697 krasak river sand image data and 608 white river sand image data, which then after going through Data Pre-processing process obtained 1200 image data that could be used in this study, each consisting of 600 data from each type of sand. This study uses 5 different input image sizes namely 32x32x3, 60x60x3, 122x122x3, 175x175x3, 225x225x3, to find out which input image size can produce the most effective accuracy data, the results found that the 122x122x3 input image size with epoch 10 and 3 layers CNN architecture is the best input image size with an accuracy of 99, 58% in recognizing and classifying Krasak river sand and white river sand, which then results of the most accurate input image were implemented in form of a Graphical User Interface (GUI) in facilitating the recognition of the classification of Krasak river sand and white river sand.

Single image super-resolution (SR) algorithms based on joint dictionaries and sparse representations of image patches have received significant attention in the literature and deliver the state-of-the-art results. Recently, Gaussian mixture models (GMMs) have emerged as favored prior for natural image patches in various image restoration problems. In this paper, we approach the single image SR problem by using a joint GMM learnt from concatenated vectors of high and low resolution patches sampled from a large database of pairs of high resolution and the corresponding low resolution images. Covariance matrices of the learnt Gaussian models capture the inherent correlations between high and low resolution patches, which are utilized for inferring high resolution patches from given low resolution patches. The proposed joint GMM method can be interpreted as the GMM analogue of joint dictionary-based algorithms for single image SR. We study the performance of the proposed joint GMM method by comparing with various competing algorithms for single image SR. Our experiments on various natural images demonstrate the competitive performance obtained by the proposed method at low computational cost.

Depth image super-resolution is a challenging problem, since normally high upscaling factors are required (e.g., 16×), and depth images are often noisy. In order to achieve large upscaling factors and resilience to noise, we propose a Robust Algorithm for Depth imAge super Resolution (RADAR) that combines the power of finite rate of innovation (FRI) theory with multimodal dictionary learning. Given a low-resolution (LR) depth image, we first model its rows and columns as piece-wise polynomials and propose an FRI-based depth upscaling (FDU) algorithm to super-resolve the image. Then, the upscaled moderate quality (MQ) depth image is further enhanced with the guidance of a registered high-resolution (HR) intensity image. This is achieved by learning multimodal mappings from the joint MQ depth and HR intensity pairs to the HR depth, through a recently proposed triple dictionary learning (TDL) algorithm. Moreover, to speed up the super-resolution process, we introduce a new projection-based rapid upscaling (PRU) technique that pre-calculates the projections from the joint MQ depth and HR intensity pairs to the HR depth. Compared with the state-of-the-art deep learning-based methods, our approach has two distinct advantages: we need a fraction of training data but can achieve the best performance, and we are resilient to mismatches between training and testing datasets. The extensive numerical results show that the proposed method outperforms other state-of-the-art methods on either noise-free or noisy datasets with large upscaling factors up to 16× and can handle unknown blurring kernels well.

Convolutional neural networks (CNN) have given rise to a new generation of video super-resolution (SR) technique. However, most existing CNN-based video SR algorithms treat the consecutive frames as a series of feature maps, just as the procedure performed in single image SR algorithms. We propose an end-to-end three-dimensional (3-D) CNN video SR framework. The input frames are considered as a cube in our framework. 3-D convolution is performed on it to extract features along spatial and temporal dimension. Image prior knowledge, such as optical flows, is introduced in reconstruction. A combination of mean square error loss and multiscale structure similarity index (MS-SSIM) loss is used to optimize the model. Experimental results show that the proposed method reconstructs high-resolution frames with more accurate and visually pleasant structures compared with state-of-the-art video SR algorithms. We also achieve comparable PSNR/SSIM results with less computation time.

The success of modern single image super-resolution (SISR) algorithms is inspired by the development of deep convolutional neural networks (CNNs). However, these CNN-based methods require considerable computation and complexity, making it impossible for these methods to perform real-time calculations in edge devices. Thus, lightweight model design has become a development trend in the super-resolution field, including pruning, quantization, and other methods. The 1-bit quantization is an extreme lightweight method which can reduce the calculation amount of the model in an extreme manner and is friendly to hardware such as edge devices. Most existing binary quantization approaches lead to a large information loss during forward propagation, especially in detailed color information (e.g., edge, texture, and contrast). The loss of color information makes modern binary methods unsuitable for SISR tasks. We think the loss occurs because these methods typically utilize a uniform threshold to quantize the weights and activations. Thus, in this article, we thoroughly analyze the difference between normal classification tasks and SISR tasks, and present a binarization scheme based on local means. The proposed method can maintain more detailed information in feature maps using dynamic thresholds during quantization. Specifically, each value in the full precision activations has a corresponding threshold during the quantization process, and those thresholds are determined by the full precision values of the surroundings. In addition, a gradient approximator is introduced to adaptively optimize the gradient for updating binary weights. We then verify the effectiveness of our method for training binary networks on several SISR benchmarks including VDSR and SRResNet. Experimental results show that the proposed method can outperform the state-of-the-art algorithms to obtain binary networks for image super-resolution with better peak signal-to-noise ratio (PSNR) values and visual quality.

The authors address a parallel algorithm for handwriting feature extraction on a RPM (reconfigurable parallel machine). Introducing the novel parallel architecture RPM, the authors have developed a very effective algorithm for feature extraction. The feature of handwriting is represented as a graph called a pattern graph. The proposed parallel feature extraction algorithm executes in O(1) time, while conventional algorithms require O(N/sup 2/) time, where N /spl times/ N is the size of input image. Dramatic improvement in execution time is due to the pixel per PE (processing element) mapping and the distributed programmable interconnection network. The number of PEs can be reduced by multiplexing a group of virtual PEs onto one PE.

In this paper, a deep learning classification model is proposed for automatically detecting the marine oil spill in Lanset-7 and Lanset-8 images, which can combine fully convolutional network (FCN) with Resnet and Googlenet respectively. The classification algorithms, i.e. FCN-Googlenet and FCN-ResNet are compared to the state-of-the-art Support Vector Machine (SVM) method. The experimental results show that our FCN-Googlenet and FCN-ResNet models outperform other approaches with a significant improvement. Moreover, our methods are more flexible in that no restriction on the size of input image is required in our algorithmic setups, which is more suitable in real applications.

Gradient boosting for single image super-resolution