A video text detection and recognition system

We propose an end-to-end system for text detection and recognition in natural scenes and consumer videos. Maximally Stable Extremal Regions which are robust to illumination and viewpoint variations are selected as text candidates. Rich shape descriptors such as Histogram of Oriented Gradients, Gabor filter, corners and geometrical features are used to represent the candidates and classified using a support vector machine. Positively labeled candidates serve as anchor regions for word formation. We then group candidate regions based on geometric and color properties to form word boundaries. To speed up the system for practical applications, we use Partial Least Squares approach for dimensionality reduction. The detected words are binarized, filtered and passed to a hidden Markov model based Optical Character Recognition (OCR) system for recognition. We show significant improvement in text detection and recognition tasks over previous approaches on a large consumer video dataset. Furthermore, the event detection system built upon the OCR output of this approach outperformed multiple other OCR-only based submissions in the recently concluded NIST TRECVID 2013 multimedia event detection evaluations.

Text detection and recognition in scene images and videos attract much attention in computer vision recently. However, most existing text detection and recognition methods only focus on static images. In this paper an end-to-end scene text recognition method based on multi frame tracking is proposed for text in videos, in which temporal information is employed to improve performance. First, an end-to-end text recognition method based on a unified deep neural network is used to detect and recognize text in each frame of the input video. Then, multi frame text tracking is employed through associations of texts in current frame and several previous frames to obtain final results. Experiments on ICDAR datasets demonstrate that the proposed method outperforms the state-of-the-art methods in end-to-end video text recognition.

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.

The successful analysis of video data is currently in great demand because a video is a major source of data in our lives. The

Text quality can significantly affect the results of text detection and recognition in digital video. In this paper we address the problem of estimating text quality. The quality of text that appears in video is often much lower than that in document images, and can be degraded by factors such as low resolution, background variation, uneven lighting, motion of the text and camera, and in the case of scene text, projection from 3D. Features based on text resolution, background noise, contrast, illumination and texture are selected to describe the text quality, normalized and fed into a trained RBF network to estimate the text quality. The performance using different training schemes are compared.

In this paper, we present a DAV-based system for text (mainly English and Chinese) detection and recognition. With the combination of unmanned aerial vehicle and scene text recognition, the system can realize text detection and recognition in long-range air-plane images, providing an underlay for unmanned navigation and fast text information understanding. Robust text detection and accurate text recognition can be achieved by two contributions. First, a scalable engine is proposed to synthesize text images by overlaying English or Chinese text into existing images in a natural way. Second, an framework which is trainable and end-to-end by combining Convolutional Neural Network and Recurrent Neural Network is adapted to recognize the variable-length text with a high accuracy. Field experiments are performed with different videos shot in various backgrounds and outdoors to show that the proposed system can detect and recognise text information in UAV imagery robustly and effectively.

Recently, video text detection, tracking, and recognition in natural scenes are becoming very popular in the computer vision community. However, most existing algorithms and benchmarks focus on common text cases (e.g.,normal size, density) and single scenario, while ignoring extreme video text challenges, i.e., dense and small text in various scenarios. In this paper, we establish a video text reading benchmark, named DSText V2, which focuses on Dense and Small text reading challenges in the video with various scenarios. Compared with the previous datasets, the proposed dataset mainly include three new challenges: (1) Dense video texts, a new challenge for video text spotters to track and read. (2) High-proportioned small texts, coupled with the blurriness and distortion in the video, will bring further challenges. (3) Various new scenarios, e.g., ‘Game’, ‘Sports’, etc. The proposed DSText V2 includes 140 video clips from 7 open scenarios, supporting three tasks, i.e., video text detection (Task 1), video text tracking (Task 2), and end-to-end video text spotting (Task 3).In this article, we describe detailed statistical information of the dataset, tasks, evaluation protocols, and the results summaries. Most importantly, a thorough investigation and analysis targeting three unique challenges derived from our dataset are provided, aiming to provide new insights. Moreover, we hope the benchmark will promise video text research in the community. DSText v2 is built upon DSText v1, which was previously introduced to organize the ICDAR 2023 competition for dense and small video text. The dataset website can be found at RRC or Zenodo.

Urdu text detection and extraction from videos especially from news feed is a challenging task. The main challenge is to design a robust algorithm that can accurately detect and localize Urdu text regions in videos. Urdu language is originated from oriental languages like Arabic, Persian and Turkish, so alphabets of these languages are very similar. Due to the similarities of basic alphabets and writing styles, this research is equally applicable to Arabic or Persian text. The development of an OCR that gives acceptable results on Urdu is still an ongoing research area. Thus need of a good text detection and extraction system is needed that can reduce the recognition errors as much as possible. It is because if the input image contains only text candidate regions, then possibilities of errors will be reduced for the same OCR. The proposed system is evaluated on a set of videos of news channels that vary along the dimensions of intensity variations, scale and orientation. Accuracy in terms of precision and recall rates is analyzed to determine the success and limitations of the approach. Problems, possible errors and suggestions for improvements are also discussed in the paper.

The video text detection is an expert system in which we study how the performance can be enhanced by adding neural network. The implementation of video text detection using algorithm based approach [9] is taken and compared with the neural networks based implementation [11]. A standard protocol [10] for evaluating the video text detection approach is taken and its metrics are used for the comparative study. With this comparison, the evaluation of both the systems for better performance can be done. The conclusions necessary for enhancing the usage of neural network is drawn based on the comparison study. The paper is about encouraging the use of neural network in an expert system (Video Text Detection Application).KeywordsNeural networksExpert systemOCR (Optical Character Recognition)video text detection

A means to naturally recognizing and fetching out the content of video description would possibly make them indexed in considerable and appropriate way for later reference, and would facilitate actions viz. automatic notification and dissemination, to be triggered in real time by the contents of streaming video. Video text recognition, or video OCR, is a constructive tool to characterize the contents of video containing overlay text (text captions superimposed over the video imagery, such as in broadcast news programs) and scene text (text that appears in the real scene of the video, such as text on street signs, nameplates, and billboards). In this paper exhaustive survey is done for text detecting, extraction and recognizing in complex images and video frames. Digital Videos are widely used both professionally and domestically because of the easy availability of camcorders to mobile phones. People are increasingly making videos may be for commercial use or personal use, this is leading to growing content of Video. While we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. This paper concentrates on extracting text out of the video frames, taken out of the video.

The lack of training datasets and computational complexity of 3 dimensions make text localization and recognition in point cloud environments challenging tasks, resulting in them being relatively undeveloped topics in the research community. In this paper, we introduce a method to adapt 2D text detection and recognition techniques in panoramic images with appropriate 3D mapping. This combined with heuristic methods such as a virtual camera creates a fast and efficient 3D text localization and recognition system. In real world applications, the objects of interest for a computer vision task may not be captured by the sensor from an ideal perspective; instead skewed imagery or partial occlusions are common. In a virtual 3D environment, we have full control of viewing angles and distances from the object. Therefore, by placing a virtual camera in certain positions, we can generate synthetic imagery of the object that is based on real imagery and avoids skewed views or occlusions. We use this synthetic imagery to improve the performance of text recognition when the object of interest is sufficiently close to the scanner, and hence the point density is high enough to generate quality imagery. The simplistic nature of this system is attractive and computationally inexpensive as it uses 2D data processes instead of natively 3D. The system shows promising results with over 85% accuracy for detection and localization tasks and 80% on the recognition task.

Information extraction (IE) systems are costly to build because they require development texts, parsing tools, and specialized dictionaries for each application domain and each natural language that needs to be processed. We present a novel method for rapidly creating IE systems for new languages by exploiting existing IE systems via cross-language projection. Given an IE system for a source language (e.g., English), we can transfer its annotations to corresponding texts in a target language (e.g., French) and learn information extraction rules for the new language automatically. In this paper, we explore several ways of realizing both the transfer and learning processes using off-the-shelf machine translation systems, induced word alignment, attribute projection, and transformation-based learning. We present a variety of experiments that show how an English IE system for a plane crash domain can be leveraged to automatically create a French IE system for the same domain.

The automatic extraction and recognition of news captions and annotations can be of great help locating topics of interest in digital news video libraries. To achieve this goal, we present a technique, called Video OCR (Optical Character Reader), which detects, extracts, and reads text areas in digital video data. In this paper, we address problems, describe the method by which Video OCR operates, and suggest applications for its use in digital news archives. To solve two problems of character recognition for videos, low-resolution characters and extremely complex backgrounds, we apply an interpolation filter, multiframe integration and character extraction filters. Character segmentation is performed by a recognition-based segmentation method, and intermediate character recognition results are used to improve the segmentation. We also include a method for locating text areas using text-like properties and the use of a language-based postprocessing technique to increase word recognition rates. The overall recognition results are satisfactory for use in news indexing. Performing Video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.

A light-weight natural scene text detection and recognition system

Video OCR is a technique that can greatly help to locate topics of interest in a large digital news video archive via the automatic extraction and reading of captions and annotations. News captions generally provide vital search information about the video being presented, the names of people and places or descriptions of objects. In this paper, two difficult problems of character recognition for videos are addressed: low resolution characters and extremely complex backgrounds. We apply an interpolation filter, multi-frame integration and a combination of four filters to solve these problems. Segmenting characters is done by a recognition-based segmentation method and intermediate character recognition results are used to improve the segmentation. The overall recognition results are good enough for use in news indexing. Performing video OCR on news video and combining its results with other video understanding techniques will improve the overall understanding of the news video content.