Convolutional Feature Fusion for Multi-Language Text Detection in Natural Scene Images

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Text detection in natural scene images is a challenging problem in computer vision. To robust detect various texts in complex scenes, a hierarchical recursive text detection method is proposed in this paper. Usually, texts in natural scenes are not alone and arranged into lines for easy reading. To find all possible text lines in an image, candidate text lines are obtained using text edge box and conventional neural network at first. Then, to accurately find out the true text lines in the image, these candidate text lines are analyzed in a hierarchical recursive architecture. For each of them, connected components segmentation and hierarchical random field based analysis are recursively employed until the detected text line no more changes. Now the detected text lines are output as the text detection result. Experiments on ICDAR 2003 dataset, ICDAR 2013 dataset and Street View Dataset show that the hierarchical recursive architecture can improve text detection performance and the proposed method achieves the state-of-art in scene text detection.

Building robust recognizers for Arabic has always been challenging. We demonstrate the effectiveness of an end-to-end trainable CNN-RNN hybrid architecture in recognizing Arabic text in videos and natural scenes. We outperform previous state-of-the-art on two publicly available video text datasets - ALIF and ACTIV. For the scene text recognition task, we introduce a new Arabic scene text dataset and establish baseline results. For scripts like Arabic, a major challenge in developing robust recognizers is the lack of large quantity of annotated data. We overcome this by synthesising millions of Arabic text images from a large vocabulary of Arabic words and phrases. Our implementation is built on top of the model introduced here [37] which is proven quite effective for English scene text recognition. The model follows a segmentation-free, sequence to sequence transcription approach. The network transcribes a sequence of convolutional features from the input image to a sequence of target labels. This does away with the need for segmenting input image into constituent characters/glyphs, which is often difficult for Arabic script. Further, the ability of RNNs to model contextual dependencies yields superior recognition results.

Scene text detection and recognition are vital challenging tasks in computer vision, which are to detect and recognize sequences of texts in natural scenes. Recently, researchers have investigated a lot of state-of-the-art methods to improve the accuracy and efficiency of text detection and recognition. However, there has been little research on text detection and recognition in natural scenes in Vietnam. In this paper, a deep learning-based empirical investigation of Vietnamese scene text detection and recognition is presented. Firstly, four detection models including differentiable binarization network (DBN), pyramid mask text detector (PMTD), pixel aggregation network (PAN), and Fourier contour embedding network (FCEN), are employed to detect text regions from the images. Then, four text recognition models including convolutional recurrent neural network (CRNN), self-attention text recognition network (SATRN), no-recurrence sequence-to-sequence text recognizer (NRTR), and RobustScanner (RS) are also investigated to recognize the texts. Moreover, data augmentation methods are also applied to enrich data for improving the accuracy and enhancing the performance of scene text detection and recognition. To validate the effectiveness of scene text detection and recognition models, the VinText dataset is employed for evaluation. Empirical results show that PMTD and SATRN achieve the highest scores among the others for text detection and recognition, respectively. For knowledge-sharing, our implementation is publicly available at https://github.com/ThorPham/VN_scene_text_detection_recognition.

In recent years, growing attention has been paid to recognizing text in natural scenes images. Scene Character recognition (SCR) is an important step in automatizing the process of reading text in natural scenes.

Scene texts refer to arbitrary text found in images captured by cameras in real-world settings. The tasks of text detection and recognition are critical components of computer vision, with applications spanning scene understanding, information retrieval, robotics, and autonomous driving. Despite significant advancements in deep learning methods, achieving accurate text detection and recognition in complex images remains a formidable challenge for robust real-world applications. Several factors contribute to these challenges. First, the diversity of text shapes, fonts, colors, and styles complicates detection efforts. Second, the myriad combinations of characters, often with unstable attributes, make complete detection difficult, especially when background interruptions obscure character strokes and shapes. Finally, effective coordination of multiple sub-tasks in end-to-end learning is essential for success. This research aimed to tackle these challenges by enhancing text discriminative representation. This study focused on two interconnected problems: Scene Text Recognition (STR), which involves recognizing text from scene images, and Scene Text Detection (STD), which entails simultaneously detecting and recognizing multiple texts within those images. This research focuses on implementing and evaluating the Efficient and Accurate Scene Text Detector (EAST) algorithm for text detection and recognition in natural scene images. The study aims to compare the performance of three prominent Optical Character Recognition (OCR) techniques—TesseractOCR, PaddleOCR, and EasyOCR. The EAST model was applied to a series of sample test images, and the results were visually represented with bounding boxes highlighting the detected text regions. The inference times for each image were recorded, highlighting the algorithm's efficiency, with average times of 0.446, 0.439, and 0.440 seconds for the respective test images. These results indicate that the EAST algorithm is accurate and operates in real-time, making it suitable for applications requiring immediate text recognition.

In recent year's availability of economical image capturing devices in low cost products like mobile phones has led a significant attention of researchers to the problem of recognizing text in images. Recognition of scene text is a challenging problem compared to the recognition of printed documents. In this work a novel approach is proposed to recognize text in complex background natural scene, word formation from recognized text, spelling checking and word translation into user defined language and finally overlay translated word onto the image. The proposed approach is robust to different kinds of text appearances, including font size, font style, color, and background. Combining the respective strengths of different complementary techniques and overcoming their shortcomings, the proposed method uses efficient character detection and localization technique and multiclass classifier to recognize the text accurately. The proposed approach successfully recognizes text on natural scene images and does not depend on a particular alphabet, text background. It works with a wide variety in size of characters and can handle up to 20 degree skewness efficiently.

Text detection in natural and complex images play an important role in analysis of image. Text information appears everywhere like product labels, documents, scene images etc. It will be very difficult for visually impaired person to find text region in an image. This problem should be addressed because it cannot assume that the captured image contains only text. This paper presents camera-based text reading framework which helps visually impaired people to read texts in natural scenes, product labels etc. Here, task is divided into two steps i.e. text detection and text recognition. Text detection involves text localization phase. Firstly, we binarize the image. Geometric and stroke width filtering removes non text regions an image. Segmented text region in a cluttered scene is binarized and recognized by optical character recognition. Recognized texts are output to the visually impaired people in speech. Experimental result shows that proposed method offers better performance in text recognition.

Texts in natural scenes carry critical semantic clues for understanding images. When capturing natural scene images, especially by handheld cameras, a common artifact, i.e., blur, frequently happens. To improve the visual quality of such images, deblurring techniques are desired, which also play an important role in character recognition and image understanding. In this paper, we study the problem of recovering the clear scene text by exploiting the text field characteristics. A series of text-specific multiscale dictionaries (TMD) and a natural scene dictionary is learned for separately modeling the priors on the text and nontext fields. The TMD-based text field reconstruction helps to deal with the different scales of strings in a blurry image effectively. Furthermore, an adaptive version of nonuniform deblurring method is proposed to efficiently solve the real-world spatially varying problem. Dictionary learning allows more flexible modeling with respect to the text field property, and the combination with the nonuniform method is more appropriate in real situations where blur kernel sizes are depth dependent. Experimental results show that the proposed method achieves the deblurring results with better visual quality than the state-of-the-art methods.

Recognizing Urdu text in natural images is more challenging as compared to other languages, such as English, due to the cursive nature of Urdu script. However, Urdu scene text has not received enough attention from both industry and academia due to the lack of the dataset of Urdu text. We propose a large-scale Urdu Scene Text Dataset (USTD) to address this problem, which is designed for Urdu scene text detection and recognition. The proposed dataset contains 29674 text annotations (17877 Urdu and 11797 English), 749725 characters in 6389 images. It covers a wide variety of text images with both Nastaleeq and Naskh writing styles, taken from different streets and roads of Pakistan. The vast diversity of this dataset makes it a benchmark to work on and train robust neural networks for the detection and recognition of cursive text. Besides, baseline results are also provided with several state-of-the-art networks, including TextBoxes++, Seglink, DB(ResNet-50) and EAST for text localization and Convolutional Recurrent Neural Network (CRNN) for text recognition. To further evaluate the performance of these models, we have used the most popular evaluation matrices of precision, recall, and F-measure. Our experimental outputs reveal that an end-to-end combination of DB(ResNet-50) and CRNN provides the best results with precision, recall, and F-measure of 0.7526, 0.5974, and 0.6660, respectively.

To improve text detection and text recognition in natural scenes, it is important to improve legibility by removing the effects of illumination on the image. Therefore, in this study, we present a signal enhancement method for the dark regions in the image. In addition, we propose a procedure to reduce computational complexity of the bilateral filter that is used to estimate the illumination in our method. Since our main aim is to provide the preprocessing of text detection and recognition in natural scenes, we have proposed a simple processing in which only the signals of the dark regions in the image are enhanced. Experimental results show that our method is effective in conserving the naturalness of the image and improving the legibility of texts in natural scenes. Furthermore, by using our proposed procedure, bilateral filter computation loads can be reduced by about 30%.

Image text detection and recognition has always been one of the most important topics in the field of intelligent information processing. Given the diversity of text size, orientation and complex background problems while detecting text in natural scenes, this paper proposes a robust and adaptive text detection algorithm. We present a text box grouping strategy and an inclined text box fitting approach to make inclined text detection robust and accurate. We also fuse the Convolutional Recurrent Neural Network (CRNN) behind to eliminate the error detection results. Our algorithm achieves 0.84 accuracy, 0.76 recall rate and 0.8 F-Measure on the MSRA-TD500 dataset.

The enhancement of Optical Character Recognition (OCR) is a lively research domain for numerous well-known applications. These applications are automation of logistics, reading sign boards, helping visually impaired subjects, and retrieval of textual content from natural scene images. The foremost objective of OCR in these applications is to acquire significant performance in order to convert scene text into machine-readable text with adequate accuracy. Henceforth, an improved technique is proposed based on image processing for segmenting textual content (foreground) from unstructured and unconstrained backgrounds. Further, a multimodal feature fusion and selection is carried out for classification. These features are serially fused and out passed to Crow Search Optimization (CSO) in order to generate a salient feature vector using entropy as fitness function. After that, the optimal features are used by ensemble classifiers for classification. The presented work is comprehensively tested on three mainstream challenging datasets SVT, MSRA-TD500, and KAIST consisting several indoor/outdoor natural images with harsh scenic variabilities. The experimental results with existing benchmark techniques confirm that the presented technique performs better in retrieving textual content. Similarly, in the classification process, the proposed technique works fine on different experiments due to the novel optimization method. Finally, the presented optimization method is compared with its counterparts with the same setup and it is found that it also performs better.

Most of the text detection and script classification approaches from natural scenes only cater for a single script whereas text in natural scenes may come in various scripts. This research proposes a gestalt-based approach for multi-script text detection and classification based on human perception. Human perceptual organization is where humans are able to organize visual input into meaningful information. This approach is based on the figure-ground articulation where we perceive the figure or text as standing in front of the background. Features extracted from wavelet coefficients and MSER is used as input to SVM for text detection and script classification. Experimental results indicate that this approach is competitive with the state of the art text detection and script classification approaches.

Buffer-text: Detecting arbitrary shaped text in natural scene image