DSText V2: A comprehensive video text spotting dataset for dense and small text

We propose an end-to-end system for text detection and recognition in natural scenes and consumer videos. Maximally Stable Extremal Regions which are robust to illumination and viewpoint variations are selected as text candidates. Rich shape descriptors such as Histogram of Oriented Gradients, Gabor filter, corners and geometrical features are used to represent the candidates and classified using a support vector machine. Positively labeled candidates serve as anchor regions for word formation. We then group candidate regions based on geometric and color properties to form word boundaries. To speed up the system for practical applications, we use Partial Least Squares approach for dimensionality reduction. The detected words are binarized, filtered and passed to a hidden Markov model based Optical Character Recognition (OCR) system for recognition. We show significant improvement in text detection and recognition tasks over previous approaches on a large consumer video dataset. Furthermore, the event detection system built upon the OCR output of this approach outperformed multiple other OCR-only based submissions in the recently concluded NIST TRECVID 2013 multimedia event detection evaluations.

We present a two stage framework for automatic video text removal to detect and remove embedded video texts and fill-in their remaining regions by appropriate data. In the video text detection stage, text locations in each frame are found via an unsupervised clustering performed on the connected components produced by the stroke width transform (SWT). Since SWT needs an accurate edge map, we develop a novel edge detector which benefits from the geometric features revealed by the bandlet transform. Next, the motion patterns of the text objects of each frame are analyzed to localize video texts. The detected video text regions are removed, then the video is restored by an inpainting scheme. The proposed video inpainting approach applies spatio-temporal geometric flows extracted by bandlets to reconstruct the missing data. A 3D volume regularization algorithm, which takes advantage of bandlet bases in exploiting the anisotropic regularities, is introduced to carry out the inpainting task. The method does not need extra processes to satisfy visual consistency. The experimental results demonstrate the effectiveness of both our proposed video text detection approach and the video completion technique, and consequently the entire automatic video text removal and restoration process.

Text detection and recognition in scene images and videos attract much attention in computer vision recently. However, most existing text detection and recognition methods only focus on static images. In this paper an end-to-end scene text recognition method based on multi frame tracking is proposed for text in videos, in which temporal information is employed to improve performance. First, an end-to-end text recognition method based on a unified deep neural network is used to detect and recognize text in each frame of the input video. Then, multi frame text tracking is employed through associations of texts in current frame and several previous frames to obtain final results. Experiments on ICDAR datasets demonstrate that the proposed method outperforms the state-of-the-art methods in end-to-end video text recognition.

In video indexing and summarization, video text is the very compact and accurate information. Most video text detection and extraction methods deal with the static videotext on video frames. Few methods can handle motion videotext well since motion videotext may be hardly be extracted well. In this paper, we propose a low computation method to detect and localize the scrolling videotexts to provide this information. Besides, we propose an extraction method to extract the videotext. The extraction method consists of adaptive thresholding and our proposed modified seed-fill algorithm. The detection method is carried out by edge detection, and the projection profile method is used to localize the text region well. We have completed the whole system and the experimental results all of the horizontal and vertical scrolling videotexts can be extracted well.

Video scene text contains semantic information and thus can contribute significantly to video indexing and summarization. However, most of the previous approaches to detecting scene text from videos experience difficulties in handling texts with various character size and text alignments. In this paper, we propose a novel algorithm of scene text detection and localization in video. Based on our observation that text character strokes show intensive edge details in the fixed orientation no matter what text alignment and size are, a stroke map is first generated. In the scene text detection, we extract the texture feature of stroke map to locate text lines. The detected scene text lines are accurately located by using Harris' corners in the stroke map. Experimental results show that this approach is robust and can be effectively applied to scene text detection and localization in video.

Nowadays, a large number of video text detection systems have been developed for daily used video applications such as transportation surveillance, electronic payment, traffic safety detection, sport videos retrieval, and even commercial online advertisements, in which the existing closed-circuit television, road-rule enforcement cameras, or online videos can be the data sources. These systems have the same requirement of video content analysis, in which automatic video text detection is believed an essential task. The detected video texts can be recognized using OCR systems and further passed to a speech system, which potentially has other usages such as providing video services for blind people and automatic navigation for drivers. This chapter introduces several typical real-life video text detection applications, including license plate recognition, navigation assistance, sport video analysis, and online video advertising. The discussed techniques in these applications can be similarly adopted by or extended to many other real-life systems such as video content retrieval, person identification from videos, E-education or E-meeting, and even karaoke music entertainments.

Text detection in natural scenes is fundamental for text image analysis. In this paper, we propose a context-based approach for robust and fast text detection. Our main contribution is that we introduce a new concept of key region, which is described with context according to stroke properties, appearance consistency and specific spatial distribution of text line. With such context descriptors, we adopt SVM to learn a context-based classifier to find key regions in candidate regions. Therein, candidate regions are connected components generated by local binarization algorithm in the areas, which are detected by an offline learned text patch detector. Experimental results on two benchmark datasets demonstrate that our approach has achieved competitive performances compared with the state-of-the-art algorithms including the stroke width transform (SWT) [1] and the hybrid approach based on CRFs [2] with speedup rates of about 1.7x~4.4x.

Detecting and recognizing text in natural scene videos and images has brought more attention to computer vision researchers due to applications like robotic navigation and traffic sign detection. In addition, Optical Character Recognition (OCR) technology is applied to detect and recognize text on the license plate. It will be used in various commercial applications such as finding stolen cars, calculating parking fees, invoicing tolls, or controlling access to safety zones and aids in detecting fraud and secure data transactions in the banking industry. Much effort is required when scene text videos are in low contrast and motion blur with arbitrary orientations. Presently, text detection and recognition approaches are limited to static images like horizontal or approximately horizontal text. Detecting and recognizing text in videos with data dynamicity is more challenging because of the presence of multiple blurs caused by defocusing, motion, illumination changes, arbitrarily shaped, and occlusion. Thus, we proposed a combined DeepEAST (Deep Efficient and Accurate Scene Text Detector) and Keras OCR model to overcome these challenges in the proffered DEFUSE (Deep Fused) work. This two-combined technique detects the text regions and then deciphers the result into a machine-readable format. The proposed method has experimented with three different video datasets such as ICDAR 2015, Road Text 1K, and own video Datasets. Our results proved to be more effective with precision, recall, and F1-Score.

Text and captions in videos provide useful information for content analysis and understanding. In this paper, we present an approach to detecting video text in a coarse-to-fine strategy. In the coarse phase we propose an efficient method to detect multi-scale candidate text regions with high recall. Then the candidate text regions are segmented and sent to the fine phase where a convolutional neural network(CNN) is applied to generate a confidence map for each candidate text region. Finally, the candidate text regions are further refined and partitioned into text lines by projection analysis. The CNN classifier in the fine phase enables feature sharing and robustly identifies text regions. The coarse phase sharply reduce the number of windows needed to be scanned by the CNN. The combination endows the proposed method with both efficiency and robustness when detecting video text. It was verified by experiment results on two publicly testing datasets and a dataset created by us.

A means to naturally recognizing and fetching out the content of video description would possibly make them indexed in considerable and appropriate way for later reference, and would facilitate actions viz. automatic notification and dissemination, to be triggered in real time by the contents of streaming video. Video text recognition, or video OCR, is a constructive tool to characterize the contents of video containing overlay text (text captions superimposed over the video imagery, such as in broadcast news programs) and scene text (text that appears in the real scene of the video, such as text on street signs, nameplates, and billboards). In this paper exhaustive survey is done for text detecting, extraction and recognizing in complex images and video frames. Digital Videos are widely used both professionally and domestically because of the easy availability of camcorders to mobile phones. People are increasingly making videos may be for commercial use or personal use, this is leading to growing content of Video. While we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. This paper concentrates on extracting text out of the video frames, taken out of the video.

To improve text detection and text recognition in natural scenes, it is important to improve legibility by removing the effects of illumination on the image. Therefore, in this study, we present a signal enhancement method for the dark regions in the image. In addition, we propose a procedure to reduce computational complexity of the bilateral filter that is used to estimate the illumination in our method. Since our main aim is to provide the preprocessing of text detection and recognition in natural scenes, we have proposed a simple processing in which only the signals of the dark regions in the image are enhanced. Experimental results show that our method is effective in conserving the naturalness of the image and improving the legibility of texts in natural scenes. Furthermore, by using our proposed procedure, bilateral filter computation loads can be reduced by about 30%.

With the increasing popularity of practical vision systems and smart phones, text detection in natural scenes becomes a critical yet challenging task. Most existing methods have focused on detecting horizontal or near-horizontal texts. In this paper, we propose a system which detects texts of arbitrary orientations in natural images. Our algorithm is equipped with a two-level classification scheme and two sets of features specially designed for capturing both the intrinsic characteristics of texts. To better evaluate our algorithm and compare it with other competing algorithms, we generate a new dataset, which includes various texts in diverse real-world scenarios; we also propose a protocol for performance evaluation. Experiments on benchmark datasets and the proposed dataset demonstrate that our algorithm compares favorably with the state-of-the-art algorithms when handling horizontal texts and achieves significantly enhanced performance on texts of arbitrary orientations in complex natural scenes.

The video text detection is an expert system in which we study how the performance can be enhanced by adding neural network. The implementation of video text detection using algorithm based approach [9] is taken and compared with the neural networks based implementation [11]. A standard protocol [10] for evaluating the video text detection approach is taken and its metrics are used for the comparative study. With this comparison, the evaluation of both the systems for better performance can be done. The conclusions necessary for enhancing the usage of neural network is drawn based on the comparison study. The paper is about encouraging the use of neural network in an expert system (Video Text Detection Application).KeywordsNeural networksExpert systemOCR (Optical Character Recognition)video text detection

Video text is very important semantic information, which brings precise and meaningful clues for video indexing and retrieval. However, most previous approaches did video text extraction and recognition separately, while the main difficulty of extraction and recognition with complex background wasn’t handled very well. In this paper, these difficulty is investigated by combining text extraction and recognition together as well as using OCR feedback information. The following features are highlighted in our approach: (i) an efficient character image segmentation method is proposed in consideration of most prior knowledge. (ii) text extraction are implemented both on text-row and segmented single character images, since text-row based extraction maintains the color consistency of characters and backgrounds while single character has simpler background. After that, the best binary image is chosen for recognition with OCR feedback. (iii) The K-means algorithm is used for extraction which ensures that the best extraction result is involved, which is the binary image with clear classification of text strokes and background. Finally, extensive experiments and empirical evaluations on several video text images are conducted to demonstrate the satisfying performance of the proposed approach.

In this paper, we propose a new system for text information extraction from news videos. First of all, a method that integrates text detecting and text tracking is developed to locate text areas in the key-frames (images), together with a scheme to evaluate the performance of this approach. To get better recognition results, we then enhance the quality of the detected text blocks by multi-frame averaging. Finally, we use an adaptive thresholding method to binarize the text blocks and recognize the text using an off-the-shelf OCR module. The detection and recognition rate of the proposed system are 94.7% and 67.5% respectively.