A number of projects are dedicated to creating digital libraries from scanned books, such as Google Books, UDL, Digital Library of India (DLI), etc. The ability to search in the content of document images is essential for the usability and popularity of these DLs. In this work, we aim toward building a retrieval system over 120K document images coming from 1000 scanned books of Telugu literature. This is a challenge because: i) OCRs are not robust enough for Indian languages, especially the Telugu script, ii) the document images contain large number of degradations and artifacts, iii) scalability to large collections is hard. Moreover, users expect that the search system accept text queries and retrieve relevant results in interactive times. We propose a Reverse Annotation framework [1], that labels word-images by their equivalent text label in the offline phase. Reverse Annotation applies a retrieval based approach to recognition. Unlike traditional annotation/recognition that identifies keywords for data, Reverse Annotation identifies data that corresponds to a given keyword. It first selects a set of keywords which are considered useful for labeling and retrieval, such as those that repeat often, and ignoring stopwords and rare-words. Exemplars are obtained for each word from a crude OCR or human annotations. The labels are then propagated across the rest of the collection by matching words in the image-feature space. Since such a matching is computationally expensive, scalability is achieved using a fast approximate nearest neighbor technique based on Hierarchical K-Means. Once text labels are assigned, each document image is considered a bag-of-words over the labeled keywords. A standard search engine is used to build a search index for quick online retrieval. An example query and the retrieved results are shown in Figure 1. We are unaware of any conventional OCRs which can retrieve such images for the given query. There are three major contributions of our work: i) recognizing the entire document collection together, instead of one-at-a-time; this means that the repetition of words in the test set is effectively used for improving accuracy, ii) speeding up recognition by clustering multiple instances of a given word, iii) recognising at the word-level, avoiding the pitfalls of character segmentation and recognition. Other OCR techniques that use word-level context still rely on inaccurate component-level classification. Using the techniques developed from this work, we were able to successfully build a retrieval system over our challenging dataset. To the best of our knowledge, this is the largest collection of document images that has been made searchable for any Indian language. Our algorithm is easily scalable to larger collections, and directly applicable to documents from other language scripts. The first issue to discuss, is the fraction of word-images that remain unrecognized at the end of the Reverse Annotation phase. Rare-words, nouns etc. are not labeled in the test set. It is important to estimate the cost of not being able to answer such queries. If this cost is indeed high, we need to explore methods to label such infrequently occurring words in the collection. Needless to say, such methods should be computationally efficient without compromising on accuracy. The other major issue to discuss is the evaluation of retrieval results. The true recall of the retrieval system cannot be computed, since it is impossible to identify every occurrence of the given query in such large data. Questions to be considered include: whether precision alone is a sufficient indicator of retrieval performance; whether there is some better document-level effectiveness assessment possible; and how best to estimate the relative satisfaction of the user's information need.
1
