Text Image Super-Resolution by Image Matting and Text Label Supervision

Natural scene text images captured by handheld devices usually cause low resolution (LR) problems, thus making sub-sequent detection and recognition tasks more challenging. To address this problem, LR text images are generally super-resolution (SR) processed first. In this paper, we propose a novel low-resolution text image super-resolution method. This method adopts the residual-in-residual dense network (RRDN) to extract deeper high-frequency features than the residual dense network (RDN). Then, enhances the spatial and channel features with an attention mechanism. According to the characteristics of the text, we added gradient loss to adversarial learning. Experiments show that our method performs well in both qualitative and quantitative aspects of the latest public text image super-resolution dataset. Similarly, the proposed super-resolution method for text images of natural scenes also achieves the latest results.

B2GAN: Bidirectional-branch generative adversarial network for text image super-resolution with structure preservation

TSRGAN: Real-world text image super-resolution based on adversarial learning and triplet attention

In the realm of image reconstruction, deep learning-based super-resolution (SR) has established itself as a prevalent technique, particularly in the domain of text image restoration. This study aims to address notable deficiencies in existing research, including constraints imposed by restricted datasets and challenges related to model generalization. Specifically, the goal is to enhance the super-resolution network’s reconstruction of scene text image super-resolution and utilize the generated degenerate dataset to alleviate issues associated with poor generalization due to the sparse scene text image super-resolution dataset. The methodology employed begins with the degradation of images from the MJSynth dataset, using a stochastic degradation process to create eight distinct degraded versions. Subsequently, a blank image is constructed, preserving identical dimensions to the low-resolution image, with each pixel sourced randomly from the corresponding points across the eight degraded images. Following several iterations of training via Finetune, the LR-HR method is applied to the TextZoom dataset. The pivotal metric for assessment is optical character recognition (OCR) accuracy, recognized for its fundamental role in gauging the pragmatic effectiveness of this approach. The experimental findings reveal a notable enhancement in OCR accuracy when compared to the TBSRN model, yielding improvements of 2.4%, 2.3%, and 4.8% on the TextZoom dataset. This innovative approach, founded on pixel-level degradation, not only exhibits commendable generalization capabilities but also demonstrates resilience in confronting the intricate challenges inherent to text image super-resolution.

In text images, there are some frequently used characters repeating more than others. Likewise, some characters have common strokes. This characteristic is used in this paper for machine-printed te...

Scene text image super-resolution aims to improve readability by recovering text shapes from low-resolution degraded text images. Although recent developments in deep learning have greatly improved super-resolution (SR) techniques, recovering text images with irregular shapes, heavy noise, and blurriness is still challenging. This is because networks with Convolutional Neural Network (CNN)-based backbones cannot sufficiently capture the global long-range correlations of text images or detailed sequential information about the text structure. In order to address this issue, this paper proposes a Multi-task learning-based Text Super-resolution (MTSR) Network to approach this problem. MTSR is a multi-task architecture for image reconstruction and SR. It uses transformer-based modules to transfer complementary features of the reconstruction model, such as noise removal capability and text structure information, to the SR model. In addition, another transformer-based module using 2D positional encoding is used to handle irregular deformations of the text. The feature maps generated from these two transformer-based modules are fused to attempt improvement of the visual quality of images with heavy noise, blurriness, and irregular deformations. Experimental results on the TextZoom dataset and several scene text recognition benchmarks show that our MTSR significantly improves the accuracy of existing text recognizers.

Given a pair of stream images of a dynamic scene with multiple moving object, how does the scene look like from a new viewing position in between two reference cameras? We tackle this interpolation problem using layered representation of the scene in this paper. The dynamic scene is separated into two different layers; a static background layer and a dynamically changing foreground layer. The foreground layer is further separated into sublayers according to the correspondences of each object in the scene. Different interpolation methods are used for the background and foreground layers. The joint view triangulation method is used for the background layer and interpolation using a local transformation of each object is used for the foreground layer. View interpolation of a dynamic scene is achieved by combining these separately interpolated background and foreground layers. Obtained photo-realistic interpolated images are presented.

Scene text image super-resolution (STISR), often considered a preliminary step for scene text recognition, refers to the task of enhancing the resolution of text embedded in natural scene images and plays a vital role in various applications. Most of the existing STISR methods either leverage deep convolutional neural networks by regarding text images as natural scene images or use a text recognizer’s feedback as guidance to the STISR process. However, since the text recognition is initially done on low-resolution images, it is mostly inaccurate, more so as the length of the words increases, thus degrading the super-resolution process. In this paper, we introduce DEPP which utilizes dictionary embedding (DE) based probabilistic priors calculated from a large English text corpus consisting of both alphabets and digits. The initial state and the bigram probabilities obtained are fused with the probability obtained from the recognizer, before passing it onto a single image super-resolution (SISR) block. By integrating DE as a prior and implementing a modified perceptual loss, the method effectively captures the contextual information of text, enabling more accurate super-resolution and visually pleasing results. Experimental results on the benchmark TextZoom dataset demonstrate that our DEPP framework achieves superior performance compared to most existing approaches, particularly for medium and long-length words, as measured by text recognition accuracy. Since DEPP uses the text recognition attributes to rectify or guide the super-resolution process, it makes our method more domain-inspired and task-aware, compared to usual black box deep learners.

In this paper, we proposed improved single image super resolution using multiple re-evaluation Process model for use in various image processing fields. The proposed method generates the first super resolution using the input image, and analyzes the change for each region by comparing the features of previous image and super resolution result. According to the analyzed features, the feature map for generate n-th super resolution was selected for improved detail. After then, next generate super resolution using previous super resolution result as input image. This process is repeated for final result. The existing single image super resolution method has two areas to be improved. First, it minimizes artifacts or staircases, which are unnecessary details that can be created during the super resolution process. Second, it is necessary to consider the input image because it affects the result depending on the quality of input image used in the super resolution process. Therefore, in order to minimize unnecessary details, the proposed method analyzed the feature map from the generated super resolution result and applied it according to the amount of change. In addition, aimed to gradually improve the input image to be used in the super resolution process by using the super resolution generated in the previous step. By comparing and evaluating the proposed method with the conventional single image super resolution method with PSNR and SSIM, it is improved by about 3%.

Information is one of the foremost fact in the prompt world. Within that, text information plays an imperative role and can acquire diverse mold. The natural images that consist of such text information are called scene text images. Semantic information of the image is used for content-based image retrieval, indexing and classification purpose. First stage of text extraction is the text and non-text classification that determines the presence of the text in an image. Compared to English language, determining the presence of Malayalam text in a scene image is more difficult due to its agglutinative nature. In this paper, the proposed work classifies natural scene images into Malayalam text and non-text images using Multiple Instance Learning (MIL). Our own dataset that contains natural scene images with Malayalam text and non-text images are used for the performance evaluation. Analysis is done in terms of precision, recall, F1 score, accuracy rates and has a promising experimental result.

This paper proposes a robust single-image super resolution method for enlarging low quality camera captured text image. The contribution of this work is twofold. First, we point out the non-local reconstruction problem in neighbor embedding based super-resolution by statistical analysis on an empirical data set. Second, we introduce a local consistency constraint to explicitly regularize the linear reconstruction process, and adaptively generate the most possible candidates for the high resolution image patch. For the non-consistent candidates, we rely on its adjacent overlapping patches for capability verification. Experimental results demonstrate that our solution produces visually pleasing enlargements for various text images.

A new multi-frame image super resolution (SR) algorithm via Bayesian modeling with natural image prior modeled by fields of experts (FoE) is proposed. Multi-frame SR can be used to obtain a high resolution (HR) image from a set of degraded low resolution (LR) images without changing any hardware device. However, SR is well known to be an ill-posed problem. So state-of-the-art solutions usually formulate the problem with Bayesian modeling techniques, which infer the HR image based on not only the LR input images but also on prior information about the HR image. Current Bayesian SR approaches typically use simple prior models such as L1 norm, TV prior and Laplacian prior, which cannot exploit the statistics of natural scenes well. In this paper, a Bayesian multi-frame image SR approach using a FOE model as the prior for natural images is presented. The Maximum a Posteriori (MAP) framework is used for estimating the HR image. The proposed method cannot only capture the statistics of natural images well, but also require less computational power than the other Bayesian modelling methods such as Sampling methods and Approximate inference. The proposed method shows superior or comparable results to the state-of-art multi-frame SR methods.

From a single low resolution image, a real-time document image super-resolution algorithm is proposed to obtain high resolution document image with sharp text boundaries. First, a highly efficient document image matting algorithm based on local linear modeling is designed to decompose the input image into text, foreground and background layers, which contain the text edge information, the color information of the foreground and background respectively. Then the text layer is up-sampled with Teager filter to increase the sharpness of the text. For efficiency, the foreground and background layers are simply up-sampled through the bi-cubic interpolation. Finally, these three high resolution layers are composed to obtain the high-resolution image. Experiments on real scanned document images demonstrate the effectiveness of the proposed method in both visual perception and OCR performance

With the development of digital multimedia technologies, image matting has gained increasing interests from both academic and industrial communities. The purpose of image matting is to precisely extract the foreground objects with arbitrary shapes from an image or a video frame for further editing. It is generally known that image matting is inherently an ill-posed problem because we need to output three images out of only one input image. In this paper, we provide a comprehensive survey of the existing image matting algorithms and evaluate their performance. In addition to the blue screen matting, we systematically divide all existing natural image matting methods into four categories: 1) color sampling-based; 2) propagation-based; 3) combination of sampling-based and propagation-based; and 4) learning-based approaches. Sampling-based methods assume that the foreground and background colors of an unknown pixel can be explicitly estimated by examining nearby pixels. Propagation-based methods are instead based on the assumption that foreground and background colors are locally smooth. Learning-based methods treat the matting process as a supervised or semisupervised learning problem. Via the learning process, users can construct a linear or nonlinear model between the alpha mattes and the image colors using a training set to estimate the alpha matte of an unknown pixel without any assumption about the characteristics of the testing image. With three benchmark data sets, the various matting algorithms are evaluated and compared using several metrics to demonstrate the strengths and weaknesses of each method both quantitatively and qualitatively. Finally, we conclude this paper by outlining the research trends and suggesting a number of promising directions for future development.

Scene text image super-resolution (STISR) aims to improve the resolution and visual quality of low-resolution (LR) scene text images, while simultaneously boost the performance of text recognition. However, most of the existing STISR methods regard text images as natural scene images, ignoring the categorical information of text. In this paper, we make an inspiring attempt to embed text recognition prior into STISR model. Specifically, we adopt the predicted character recognition probability sequence as the text prior, which can be obtained conveniently from a text recognition model. The text prior provides categorical guidance to recover high-resolution (HR) text images. On the other hand, the reconstructed HR image can refine the text prior in return. Finally, we present a multi-stage text prior guided super-resolution (TPGSR) framework for STISR. Our experiments on the benchmark TextZoom dataset show that TPGSR can not only effectively improve the visual quality of scene text images, but also significantly improve the text recognition accuracy over existing STISR methods. Our model trained on TextZoom also demonstrates certain generalization capability to the LR images in other datasets.