Text Detection Design Based on Deep Neural Network

Text detection in natural scenes is fundamental for text image analysis. In this paper, we propose a context-based approach for robust and fast text detection. Our main contribution is that we introduce a new concept of key region, which is described with context according to stroke properties, appearance consistency and specific spatial distribution of text line. With such context descriptors, we adopt SVM to learn a context-based classifier to find key regions in candidate regions. Therein, candidate regions are connected components generated by local binarization algorithm in the areas, which are detected by an offline learned text patch detector. Experimental results on two benchmark datasets demonstrate that our approach has achieved competitive performances compared with the state-of-the-art algorithms including the stroke width transform (SWT) [1] and the hybrid approach based on CRFs [2] with speedup rates of about 1.7x~4.4x.

Text detection in natural scenes is a current research hotspot. The Efficient and Accurate Scene Text (EAST) detector model has fast detection speed and good performance but is ineffective in detecting long text regions owing to its small receptive field. In this study, we built upon the EAST model by improving the bounding box’s shrinking algorithm to make the model more accurate in predicting short edges of text regions; altering the loss function from balanced cross-entropy to Focal loss; improving the model’s learning ability on hard, positive examples; and adding a feature enhancement module (FEM) to increase the receptive field of the EAST model and enhance its detection ability for long text regions. The improved EAST model achieved better detection results on both the ICDAR2015 dataset and the Street Sign Text Detection (SSTD) dataset proposed in this paper. The precision and F1 scores of the model also demonstrated advantages over other models on the ICDAR2015 dataset. A comparison of the text detection effects between the improved EAST model and the EAST model showed that the proposed FEM was more effective in increasing the EAST detector’s receptive field, which indicates that it can improve the detection of long text regions.

Scene Text Deblurring in Non-stationary Video Sequences

Previous approaches for scene text detection have already achieved promising performances across various benchmarks. However, they usually fall short when dealing with challenging scenarios, even when equipped with deep neural network models, because the overall performance is determined by the interplay of multiple stages and components in the pipelines. In this work, we propose a simple yet powerful pipeline that yields fast and accurate text detection in natural scenes. The pipeline directly predicts words or text lines of arbitrary orientations and quadrilateral shapes in full images, eliminating unnecessary intermediate steps (e.g., candidate aggregation and word partitioning), with a single neural network. The simplicity of our pipeline allows concentrating efforts on designing loss functions and neural network architecture. Experiments on standard datasets including ICDAR 2015, COCO-Text and MSRA-TD500 demonstrate that the proposed algorithm significantly outperforms state-of-the-art methods in terms of both accuracy and efficiency. On the ICDAR 2015 dataset, the proposed algorithm achieves an F-score of 0.7820 at 13.2fps at 720p resolution.

The non-maximum suppression (NMS) algorithm, which merges neighboring bounding boxes as the detection result, is widely utilized in object detection methods. However, the traditional NMS algorithm is not suitable for text detection in natural scenes, especially for long texts and dense texts. In this paper, we observe that the coordinates of candidate bounding boxes are presented in skew distribution. On this observation, an improved NMS algorithm called SD-NMS (Skew Distribution NMS) is designed. First, the mode and the median of the coordinate set are counted to filter out the redundant bounding boxes. Then the left bounding boxes are merged for the location of the text regions. The SD-NMS method improves the detection ability of the model without extra model training and can be easily embedded in text detection methods. The experimental results show that our method obtains F-measure more than other NMS methods in public data sets ICDAR2015 and MSRATD500.

With the increasing popularity of practical vision systems and smart phones, text detection in natural scenes becomes a critical yet challenging task. Most existing methods have focused on detecting horizontal or near-horizontal texts. In this paper, we propose a system which detects texts of arbitrary orientations in natural images. Our algorithm is equipped with a two-level classification scheme and two sets of features specially designed for capturing both the intrinsic characteristics of texts. To better evaluate our algorithm and compare it with other competing algorithms, we generate a new dataset, which includes various texts in diverse real-world scenarios; we also propose a protocol for performance evaluation. Experiments on benchmark datasets and the proposed dataset demonstrate that our algorithm compares favorably with the state-of-the-art algorithms when handling horizontal texts and achieves significantly enhanced performance on texts of arbitrary orientations in complex natural scenes.

Scene text detection and recognition are vital challenging tasks in computer vision, which are to detect and recognize sequences of texts in natural scenes. Recently, researchers have investigated a lot of state-of-the-art methods to improve the accuracy and efficiency of text detection and recognition. However, there has been little research on text detection and recognition in natural scenes in Vietnam. In this paper, a deep learning-based empirical investigation of Vietnamese scene text detection and recognition is presented. Firstly, four detection models including differentiable binarization network (DBN), pyramid mask text detector (PMTD), pixel aggregation network (PAN), and Fourier contour embedding network (FCEN), are employed to detect text regions from the images. Then, four text recognition models including convolutional recurrent neural network (CRNN), self-attention text recognition network (SATRN), no-recurrence sequence-to-sequence text recognizer (NRTR), and RobustScanner (RS) are also investigated to recognize the texts. Moreover, data augmentation methods are also applied to enrich data for improving the accuracy and enhancing the performance of scene text detection and recognition. To validate the effectiveness of scene text detection and recognition models, the VinText dataset is employed for evaluation. Empirical results show that PMTD and SATRN achieve the highest scores among the others for text detection and recognition, respectively. For knowledge-sharing, our implementation is publicly available at https://github.com/ThorPham/VN_scene_text_detection_recognition.

To improve text detection and text recognition in natural scenes, it is important to improve legibility by removing the effects of illumination on the image. Therefore, in this study, we present a signal enhancement method for the dark regions in the image. In addition, we propose a procedure to reduce computational complexity of the bilateral filter that is used to estimate the illumination in our method. Since our main aim is to provide the preprocessing of text detection and recognition in natural scenes, we have proposed a simple processing in which only the signals of the dark regions in the image are enhanced. Experimental results show that our method is effective in conserving the naturalness of the image and improving the legibility of texts in natural scenes. Furthermore, by using our proposed procedure, bilateral filter computation loads can be reduced by about 30%.

We propose an end-to-end system for text detection and recognition in natural scenes and consumer videos. Maximally Stable Extremal Regions which are robust to illumination and viewpoint variations are selected as text candidates. Rich shape descriptors such as Histogram of Oriented Gradients, Gabor filter, corners and geometrical features are used to represent the candidates and classified using a support vector machine. Positively labeled candidates serve as anchor regions for word formation. We then group candidate regions based on geometric and color properties to form word boundaries. To speed up the system for practical applications, we use Partial Least Squares approach for dimensionality reduction. The detected words are binarized, filtered and passed to a hidden Markov model based Optical Character Recognition (OCR) system for recognition. We show significant improvement in text detection and recognition tasks over previous approaches on a large consumer video dataset. Furthermore, the event detection system built upon the OCR output of this approach outperformed multiple other OCR-only based submissions in the recently concluded NIST TRECVID 2013 multimedia event detection evaluations.

Person detection is an important prerequisite of social interaction, but is not well understood. Following suggestions that people in the visual field can capture a viewer's attention, this study examines the role of the face and the body for person detection in natural scenes. We observed that viewers tend first to look at the center of a scene, and only then to fixate on a person. When a person's face was rendered invisible in scenes, bodies were detected as quickly as faces without bodies, indicating that both are equally useful for person detection. Detection was optimized when face and body could be seen, but observers preferentially fixated faces, reinforcing the notion of a prominent role for the face in social perception. These findings have implications for claims of attention capture by faces in that they demonstrate a mediating influence of body cues and general scanning principles in natural scenes.

The object contour detection in natural scenes is an important research problem in computer vision. The difficulty is that the integrity of contour extracted is interfered with a large number of texture edges in the background seriously. In recent years, some researchers have introduced image contour detection method with biological visual features and achieved preferable results. Such as, based on the fact of visual outer region suppression, they can suppress a certain amount of texture edge while extracting the contour of the object, and obtain the proper contour edge. However, when this method is used for contour detection in some image which the texture is similar to the contour, the texture edge still cannot be removed well, and the result is unsatisfactory. In consideration of the merit and demerit of the feature suppressed method, we propose an improved algorithm for object contour detection in natural scene based on image saliency. Firstly, the initial contour is obtained by using the suppression effect of non-classical receptive field, and the saliency map is obtained by using the hyper-complex Fourier transform image saliency detection method, and then the initial contour and the saliency map are integrated into the final contour. Compared with the traditional contour detection method, the contour extraction algorithm based on image saliency can remove more background information and improve the accuracy and integrity of object contour extracted in natural scenes.

As an important part in the field, the natural scene text detection has been widely applied in visual navigation system, content-based image and video retrieval, instant translation system and so on. In this paper, we introduce several object detection network based on deep learning, and apply the YOLO v2 into natural scene text detection, changing the multi objects detection problems into the two classification problems. The main works in the paper include the following: prepare the datasets; we train the YOLO v2 with the optimum parameters, carry out the regression analysis of the coordinate parameters and categories of bounding boxes, obtain the detection result; according to different detection models, the detection results of different datasets are compared and analyzed, YOLO V2 model detection speed 0.105s/image has certain advantages.

Recently, video text detection, tracking, and recognition in natural scenes are becoming very popular in the computer vision community. However, most existing algorithms and benchmarks focus on common text cases (e.g.,normal size, density) and single scenario, while ignoring extreme video text challenges, i.e., dense and small text in various scenarios. In this paper, we establish a video text reading benchmark, named DSText V2, which focuses on Dense and Small text reading challenges in the video with various scenarios. Compared with the previous datasets, the proposed dataset mainly include three new challenges: (1) Dense video texts, a new challenge for video text spotters to track and read. (2) High-proportioned small texts, coupled with the blurriness and distortion in the video, will bring further challenges. (3) Various new scenarios, e.g., ‘Game’, ‘Sports’, etc. The proposed DSText V2 includes 140 video clips from 7 open scenarios, supporting three tasks, i.e., video text detection (Task 1), video text tracking (Task 2), and end-to-end video text spotting (Task 3).In this article, we describe detailed statistical information of the dataset, tasks, evaluation protocols, and the results summaries. Most importantly, a thorough investigation and analysis targeting three unique challenges derived from our dataset are provided, aiming to provide new insights. Moreover, we hope the benchmark will promise video text research in the community. DSText v2 is built upon DSText v1, which was previously introduced to organize the ICDAR 2023 competition for dense and small video text. The dataset website can be found at RRC or Zenodo.

Deep Learning Based Text Detection and Recognition in Natural Scenes

As an important core in the intelligent traffic management system, the technology and application of license plate recognition have become research focus. Detecting the accurate location of a license plate from a vehicle image is considered to be the most crucial step of license plate recognition, which greatly affects the recognition rate and speed of the whole system. Nevertheless, due to the low accuracy of license plate detection in natural scenes, further investigations are still needed in this field in order to make the detection process very efficient. In this paper, We have studied and implemented a convolutional neural network license plate detection algorithm based on transfer learning. According to the invention, new energy license plates and ordinary license plates are adopted as the research objects. The text detection model AdvancedEAST is trained on the license plate images through the transfer learning method, and experiments are carried out on the self-built license plate dataset. The experimental results show that the algorithm can better adapt to light complexity, low resolution, target interference, license plate tilt and other complex conditions. The license plate positioning algorithm has high accuracy in natural scenes, and it is superior to the traditional license plate detection methods.