<title>Text quality estimation in video</title>

Text detection and recognition in scene images and videos attract much attention in computer vision recently. However, most existing text detection and recognition methods only focus on static images. In this paper an end-to-end scene text recognition method based on multi frame tracking is proposed for text in videos, in which temporal information is employed to improve performance. First, an end-to-end text recognition method based on a unified deep neural network is used to detect and recognize text in each frame of the input video. Then, multi frame text tracking is employed through associations of texts in current frame and several previous frames to obtain final results. Experiments on ICDAR datasets demonstrate that the proposed method outperforms the state-of-the-art methods in end-to-end video text recognition.

We propose an end-to-end system for text detection and recognition in natural scenes and consumer videos. Maximally Stable Extremal Regions which are robust to illumination and viewpoint variations are selected as text candidates. Rich shape descriptors such as Histogram of Oriented Gradients, Gabor filter, corners and geometrical features are used to represent the candidates and classified using a support vector machine. Positively labeled candidates serve as anchor regions for word formation. We then group candidate regions based on geometric and color properties to form word boundaries. To speed up the system for practical applications, we use Partial Least Squares approach for dimensionality reduction. The detected words are binarized, filtered and passed to a hidden Markov model based Optical Character Recognition (OCR) system for recognition. We show significant improvement in text detection and recognition tasks over previous approaches on a large consumer video dataset. Furthermore, the event detection system built upon the OCR output of this approach outperformed multiple other OCR-only based submissions in the recently concluded NIST TRECVID 2013 multimedia event detection evaluations.

A means to naturally recognizing and fetching out the content of video description would possibly make them indexed in considerable and appropriate way for later reference, and would facilitate actions viz. automatic notification and dissemination, to be triggered in real time by the contents of streaming video. Video text recognition, or video OCR, is a constructive tool to characterize the contents of video containing overlay text (text captions superimposed over the video imagery, such as in broadcast news programs) and scene text (text that appears in the real scene of the video, such as text on street signs, nameplates, and billboards). In this paper exhaustive survey is done for text detecting, extraction and recognizing in complex images and video frames. Digital Videos are widely used both professionally and domestically because of the easy availability of camcorders to mobile phones. People are increasingly making videos may be for commercial use or personal use, this is leading to growing content of Video. While we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. This paper concentrates on extracting text out of the video frames, taken out of the video.

The intelligent analysis of video data is currently in wide demand because a video is a major source of sensory data in our lives. Text is a prominent and direct source of information in video, while the recent surveys of text detection and recognition in imagery focus mainly on text extraction from scene images. Here, this paper presents a comprehensive survey of text detection, tracking, and recognition in video with three major contributions. First, a generic framework is proposed for video text extraction that uniformly describes detection, tracking, recognition, and their relations and interactions. Second, within this framework, a variety of methods, systems, and evaluation protocols of video text extraction are summarized, compared, and analyzed. Existing text tracking techniques, tracking-based detection and recognition techniques are specifically highlighted. Third, related applications, prominent challenges, and future directions for video text extraction (especially from scene videos and web videos) are also thoroughly discussed.

In this paper, we propose a new system for text information extraction from news videos. First of all, a method that integrates text detecting and text tracking is developed to locate text areas in the key-frames (images), together with a scheme to evaluate the performance of this approach. To get better recognition results, we then enhance the quality of the detected text blocks by multi-frame averaging. Finally, we use an adaptive thresholding method to binarize the text blocks and recognize the text using an off-the-shelf OCR module. The detection and recognition rate of the proposed system are 94.7% and 67.5% respectively.

The successful analysis of video data is currently in great demand because a video is a major source of data in our lives. The

Recently, video text detection, tracking, and recognition in natural scenes are becoming very popular in the computer vision community. However, most existing algorithms and benchmarks focus on common text cases (e.g.,normal size, density) and single scenario, while ignoring extreme video text challenges, i.e., dense and small text in various scenarios. In this paper, we establish a video text reading benchmark, named DSText V2, which focuses on Dense and Small text reading challenges in the video with various scenarios. Compared with the previous datasets, the proposed dataset mainly include three new challenges: (1) Dense video texts, a new challenge for video text spotters to track and read. (2) High-proportioned small texts, coupled with the blurriness and distortion in the video, will bring further challenges. (3) Various new scenarios, e.g., ‘Game’, ‘Sports’, etc. The proposed DSText V2 includes 140 video clips from 7 open scenarios, supporting three tasks, i.e., video text detection (Task 1), video text tracking (Task 2), and end-to-end video text spotting (Task 3).In this article, we describe detailed statistical information of the dataset, tasks, evaluation protocols, and the results summaries. Most importantly, a thorough investigation and analysis targeting three unique challenges derived from our dataset are provided, aiming to provide new insights. Moreover, we hope the benchmark will promise video text research in the community. DSText v2 is built upon DSText v1, which was previously introduced to organize the ICDAR 2023 competition for dense and small video text. The dataset website can be found at RRC or Zenodo.

Text detection in images or videos is an important step to achieve multimedia content retrieval. In this paper, an efficient algorithm which can automatically detect, localize and extract horizontally aligned text in images (and digital videos) with complex backgrounds is presented. The proposed approach is based on the application of a color reduction technique, a method for edge detection, and the localization of text regions using projection profile analyses and geometrical properties. The output of the algorithm are text boxes with a simplified background, ready to be fed into an OCR engine for subsequent character recognition. Our proposal is robust with respect to different font sizes, font colors, languages and background complexities. The performance of the approach is demonstrated by presenting promising experimental results for a set of images taken from different types of video sequences.

Video contains two types of texts. The first type pertains to caption texts which are edited texts or graphics texts artificially superimposed into video and are relevant to the content of the video. The second type belongs to scene texts, which are naturally existing texts, usually embedded in objects in the video. This chapter focuses on the state-of-the-art methods developed for caption text detection in video. According to the literature, current methods can be classified into two broad categories, namely, feature-based methods and machine learning-based methods. Feature-based methods described in this chapter make use of the following features for text detection, namely, image edges by means of gradient and filters, textures by combining a variety of image textures, connected components by analyzing skeletons obtained from the image, and frequency domain features by performing Fourier transform. On the other hand, machine learning methods presented in this chapter make use of classifiers such as support vector machines, neural networks, and Bayesian classifiers.

Video scene text contains semantic information and thus can contribute significantly to video indexing and summarization. However, most of the previous approaches to detecting scene text from videos experience difficulties in handling texts with various character size and text alignments. In this paper, we propose a novel algorithm of scene text detection and localization in video. Based on our observation that text character strokes show intensive edge details in the fixed orientation no matter what text alignment and size are, a stroke map is first generated. In the scene text detection, we extract the texture feature of stroke map to locate text lines. The detected scene text lines are accurately located by using Harris' corners in the stroke map. Experimental results show that this approach is robust and can be effectively applied to scene text detection and localization in video.

The video text detection is an expert system in which we study how the performance can be enhanced by adding neural network. The implementation of video text detection using algorithm based approach [9] is taken and compared with the neural networks based implementation [11]. A standard protocol [10] for evaluating the video text detection approach is taken and its metrics are used for the comparative study. With this comparison, the evaluation of both the systems for better performance can be done. The conclusions necessary for enhancing the usage of neural network is drawn based on the comparison study. The paper is about encouraging the use of neural network in an expert system (Video Text Detection Application).KeywordsNeural networksExpert systemOCR (Optical Character Recognition)video text detection

Support vector machine-based text detection in digital video

Nowadays, a large number of video text detection systems have been developed for daily used video applications such as transportation surveillance, electronic payment, traffic safety detection, sport videos retrieval, and even commercial online advertisements, in which the existing closed-circuit television, road-rule enforcement cameras, or online videos can be the data sources. These systems have the same requirement of video content analysis, in which automatic video text detection is believed an essential task. The detected video texts can be recognized using OCR systems and further passed to a speech system, which potentially has other usages such as providing video services for blind people and automatic navigation for drivers. This chapter introduces several typical real-life video text detection applications, including license plate recognition, navigation assistance, sport video analysis, and online video advertising. The discussed techniques in these applications can be similarly adopted by or extended to many other real-life systems such as video content retrieval, person identification from videos, E-education or E-meeting, and even karaoke music entertainments.

Facing the problems of low working efficiency, poor accuracy and insufficient stability of traditional motion detection algorithms and machine learning algorithms in pedestrian detection, this paper will take artificial intelligence technology as the core, adopt R-CNN series deep learning models, combine Support Vector Machine (SVM) classifier, and rely on TensorFlow deep learning framework to realize pedestrian detection and recognition in video and picture files with the support of OpengCV Open Source Visual Library. Combined with ASP.NET framework, C# language is used to complete the design and development of each functional module, improve the deployment of the corresponding API interface, and build a front-end interactive interface to form a pedestrian detection system based on Web. The overall design of the system chooses B/S architecture, which allows users to access the Web Server through simple request operation to complete the detection of video image data in the database and return the detection results to the front page for display. The system will greatly improve the precision and accuracy of pedestrian detection and make a positive attempt to further expand the application scenarios of pedestrian detection in video images.

Urdu text detection and extraction from videos especially from news feed is a challenging task. The main challenge is to design a robust algorithm that can accurately detect and localize Urdu text regions in videos. Urdu language is originated from oriental languages like Arabic, Persian and Turkish, so alphabets of these languages are very similar. Due to the similarities of basic alphabets and writing styles, this research is equally applicable to Arabic or Persian text. The development of an OCR that gives acceptable results on Urdu is still an ongoing research area. Thus need of a good text detection and extraction system is needed that can reduce the recognition errors as much as possible. It is because if the input image contains only text candidate regions, then possibilities of errors will be reduced for the same OCR. The proposed system is evaluated on a set of videos of news channels that vary along the dimensions of intensity variations, scale and orientation. Accuracy in terms of precision and recall rates is analyzed to determine the success and limitations of the approach. Problems, possible errors and suggestions for improvements are also discussed in the paper.