Toward the optimized crowdsourcing strategy for OCR post-correction

<b>Natural Language Processing for Historical Texts</b> <b>Michael Piotrowski</b> (Leibniz Institute of European History) Morgan &amp; Claypool (Synthesis Lectures on Human Language Technologies, edited by Graeme Hirst, volume 17), 2012, ix+157 pp; paperbound, ISBN 978-1608459469

An internet of things (IoT) is an exclusive method, were its impact on the enactments of human life is very trendy. This research on library control system operates on the basis of IoT and optical character recognition (OCR) algorithm rules and its training. A closed-circuit television (CCTV) watched mechanism is created to control the book issuing and returning phenomenon via tag studying system in the library. In this proposed work text file is converted into an audio file. This audio file is being played and the contents of the book can be heard via the headset. This unique function of the OCR helps blind people. Now a days OCR widely focused in machine processes such as machine transformation, text to speech extraction and text data mining. It utilized in various area of research in artificial intelligence, computer vision and pattern recognition. Using OCR to scan the damaged book in the library converted into pdf format the book gets new life and sharing the contents to multiple readers. In this paper aims to implement IoT based library management system to maintaining books in digital format.

Lack of handwriting automatization in childhood can cause difficulties within and outside the school context. Therefore, the objective quantification of the handwriting performance is key. A Smart Ink Pen (SIP) used on paper demonstrated its validity in characterizing primary school children's handwriting process, although missing information on the handwriting product. To overcome this limitation, a trace reconstruction algorithm was developed, based on the force and IMU signals measured by the SIP. A total of 353 words "uno", written in cursive from the BVSCO-3 battery by 50 Italian students of the 5th grade of primary school, was reconstructed. The quality of the reconstructions was validated through an Optical Character Recognition (OCR) algorithm (Google Vision), using the scans of the actual traces as a reference. The character recognition rates were 81.02% and 59.49%, the character error rates 21.48% and 47.65%, for scans and reconstructions, respectively. A deeper analysis revealed that 15% of the reconstructions were read in the opposite direction by the OCR algorithm, likely due to a non sufficient sampling rate for the last portion of the words. A characterization of the differences between the good, bad and opposite reconstructions allowed to identify some directions of improvement. An increase of the SIP sampling rate, a better modeling of the thickness of the trace, a finer estimation of the relative distance between the IMU sensor and the tip and the reconstruction of the tip trajectory during in-air movements could improve the trace reconstruction algorithm. In addition, the possibility to leverage transfer learning approaches on the OCR algorithm using the reconstructed traces for additional training could further improve the performances in terms of character recognition rate. However, the preliminary results obtained are promising and highlight the possibility of reconstructing handwriting traces while maintaining the naturalness of the gesture.Clinical relevance- This contribution establishes the possibility of reconstructing handwritten traces from the kinematic and force signals recorded by an ecological sensorized ink pen.

Since the COVID-19 pandemic, numerous jobs have become necessary, including the storing and sharing of printed material across computers. One simple way to save data from printed papers to a computer system is to scan them first and then save them as images. However, it would be quite challenging to extract or query text or other information from these photo files to reuse this information. As a result, a method for automatically retrieving and storing information, particularly text, from picture files is required. Optical character recognition (OCR) is an ongoing research topic that aims to create a computer system capable of extracting and processing text from images. To accomplish successful automation, certain significant problems must be identified and addressed. The font properties of characters in paper documents, as well as image quality, are only a few of the latest problems. Characters may not be recognized correctly by the computer system because of many complexities. So, in this study, authors look into OCR in four different contexts and apply them to get our results. However, every OCR is further followed by these two steps. First, a comprehensive explanation of the challenges that may develop during the OCR phases is provided. The key phases of an OCR system are then executed, including pre-processing, segmentation, normalization, feature extraction, classification, and post-processing. It can be used with deep learning software to provide OCR data which is very useful for robotic and AI applications.

This paper describes a new expert system for automatically correcting errors made by optical character recognition (OCR) devices. The system, which we call the post-processing system, is designed to improve the quality of text produced by an OCR device in preparation for subsequent retrieval from an information system. The system is composed of numerous parts: an information retrieval system, an English dictionary, a domain-specific dictionary, and a collection of algorithms and heuristics designed to correct as many OCR errors as possible. For the remaining errors that cannot be corrected, the system passes them on to a user-level editing program. This post-processing system can be viewed as part of a larger system that would streamline the steps of taking a document from its hard copy form to its usable electronic form, or it can be considered a stand alone system for OCR error correction. An earlier version of this system has been used to process approximately 10,000 pages of OCR generated text. Among the OCR errors discovered by this version, about 87% were corrected. We implement numerous new parts of the system, test this new version, and present the results.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper describes the information properties of museum specimen labels and machine learning tools to automatically extract Darwin Core (DwC) and other metadata from these labels processed through Optical Character Recognition (OCR). The DwC is a metadata profile describing the core set of access points for search and retrieval of natural history collections and observation databases. Using the HERBIS Learning System (HLS) we extract 74 independent elements from these labels. The automated text extraction tools are provided as a web service so that users can reference digital images of specimens and receive back an extended Darwin Core XML representation of the content of the label. This automated extraction task is made more difficult by the high variability of museum label formats, OCR errors and the open class nature of some elements. In this paper we introduce our overall system architecture, and variability robust solutions including, the application of Hidden Markov and Naïve Bayes machine learning models, data cleaning, use of field element identifiers, and specialist learning models. The techniques developed here could be adapted to any metadata extraction situation with noisy text and weakly ordered elements.

By utilising web-based collaboration tools, institutions can engage users in the building of historical printed text resources created by mass digitisation projects. The paper presents the drivers for developing such tools and identifies the benefits that can be derived by both the user community and cultural heritage institutions. The perceived risks, such as errors introduced by the users or whether users will engage with resources in this way, will be set out. The paper will present the lessons that can be learnt from existing activities, such as the National Library of Australia’s newspaper website, which supports collaborative correction of Optical Character Recognition (OCR) output. The user collaboration tools being created by the IMPACT Project (Improving Access to Text, http://www.impact-project.eu), a large-scale integrating project funded by the European Commission as part of the Seventh Framework Programme (FP7), will be detailed. A primary aim of IMPACT is to develop tools that help improve OCR results for historical printed texts, specifically those works published before the industrial production of books in the middle of the 19th century. While technological improvements to image processing and OCR engine technology are key to improving access to historic text, engaging the user community also has an important role to play. Utilising the user community can aid in achieving the levels of accuracy currently found in born digital materials. Improving OCR results to this level is key to producing resources that support better resource discovery and enable greater performance when applying text mining and accessibility tools to the extracted text. The IMPACT project will specifically develop a tool that supports collaborative correction and validation of OCR results and to allow user involvement in building historical dictionaries that can be used to validate word recognition. The technologies use the characteristics of human perception as a basis for error detection.

Character recognition has been very popular and interested area for researches, and it continues to be a challenging and impressive research topic due to its diverse applicable environment. The optical character recognition has been introduced as a fast and accurate method to convert both existing text images as well as large archives of existing paper documents to editable digital text format. However, existing optical character recognition algorithms suffer from flawed tradeoffs between accuracy and speed, making them less effective and impractical for large and complex documents. This paper describes a suggested method for Assyrian optical character recognition using modified back propagation artificial neural network based on moments. The experimental results show that the proposed method achieves higher recognition accuracy rate in compared with the standard algorithm.

The trend of personalized medicine and the increasing flexibility of drug dosage relevant goals of the 21st century represent the foundation for the current research. To obtain doses smaller than the smallest available, physicians frequently write prescriptions for children and adults, without preserving the integrity of the pill. Moreover, patients purchase large amounts of medication for cost-saving reasons. To support the correct administration of the remedies and the partial alignment to the personalized treatment trend, this paper proposes a flexible and user-friendly solution for determining the medication quantity given to patients, using augmented reality and optical character recognition algorithm capabilities. Via the MATLAB development environment and a Logitech HD Pro C920 webcam, the results were 80% correct in identifying the cutting position of the pill, by means of the Hough transform, and 30% correct in weight recognition exploitation using an optical character recognition (OCR) algorithm. In future work, a higher resolution camera and a more powerful computer can be used to increase the percentages mentioned above. In addition, a 3D scan of the pill fragmentation, combined with the weight recognition functionality, could increase the accuracy of the splitting procedure, conducted by the patient or the patient caretaker.

Optical Character Recognition (OCR) deals with automated recognition of characters that are in the format of digital image. OCR refers to the process by which scanned images are electronically processed and converted to an editable document. Handwritten and printed texts are the primary research areas of an OCR. Many OCR systems are commercially available for English and Arabic characters but there is still no recognition system available which yields higher recognition rate even though the scanned images are of high quality. The general framework of a Tamil OCR in the literature involves: preprocessing, line segmentation, word segmentation, character segmentation, feature extraction and recognition of characters. OCR for printed Tamil documents poses challenge owing to: one line may have different font styles, presence of pictures, multi columns, touching of adjacent characters, presence of broken characters, low print quality and complex layout. Furthermore, when comparing 26 alphabets in English, Tamil language has 247 alphabets which makes the recognition more difficult. There are few OCRs for Tamil language that are freely available with a moderate recognition rate as the performance comparisons of such OCRs are not available on a benchmark dataset. In this paper we compare OCRs for printed Tamil texts on four different types of documents: books, magazines, newspapers and pamphlets. Furthermore we propose a post-processing error correction technique to the tested OCRs which reduces the overall mean error rate by nearly 10% on those four categories.

Many documents must be processed and scanned digitally, especially during this pandemic, physical documents often have to be reprocessed using word processing applications. This process takes a lot of time especially if the document is damaged, lost and does not have a copy. This can be overcome by using the Optical Character Recognition (OCR) algorithm. OCR converts letter images into computer-recognized ASCII characters. Image files are relatively larger than their ASCII text files. From the existing problems, the researchers tried to find a solution by creating an OCR application program to simplify the process. The created OCR program is successfully used to scan or scan a physical document if the document is lost or damaged, and can save time on document creation, processing and typing. Keywords: Algorithm, Optical Character Recognition, ASCII

In today’s digital world, we are committed to digitizing thousands of handwritten transcriptions to preserve their content. Historical Arabic Handwritten Text Recognition (HAHTR) remains a challenge for computer vision systems, due to the many difficulties inherently associated with document image quality and the complexity of Arabic script. In this work, we address the problem of recognizing historical Arabic documents that adapts to different writing styles and degrees of legibility. We developed a system that is able to recognize a whole page of a historical Arabic handwritten text in two consecutive steps comprising text line detection and recognition. The proposed approach performs detection using bounding boxes followed by a neural network-based model for character-level text recognition. However, the lack of data hinders the mass digitization of Arabic historical documents. Therefore, we provide a new and freely available dataset, focusing on diverse handwriting styles to facilitate a strong generalization of the trained model. This dataset will significantly benefits researchers and practitioners by accelerating progress in the field of HAHTR. Extensive experimental work demonstrates that the recognition models are effective when trained with different sources of data, and having different writing styles does not penalize the model’s ability to generalize but rather enhances it. Additionally, we define and develop a new metric to evaluate model robustness against character misclassification, particularly for characters with similar patterns. The experiments conducted demonstrated that the proposed HAHTR pipeline is accurate and highly generalizable, as well as the validity of bounding box methods for detecting text lines. The training approach with different data sources enabled us to surpass the state-of-the-art results with 5.7% of Character Error Rate (CER) on the KHATT database.

The technology of Optical Character Recognition (OCR) is used to generate texts in the process of digitizing print documents. Usually these texts need to be indexed and organized to simplify their access and retrieval. One of the powerful approaches in accomplishing this task is the use of Automated Text Classification. However, it is currently impossible for OCR technology to recognize all characters with an accuracy of 100%. We therefore propose the use of combined linguistic features in automated classification of OCR texts to formulate an informative feature set. The proposed method was experimentally evaluated using Japanese OCR texts. Empirical results indicate that the combination of linguistic features improved classification performance of OCR texts.

In the last decades, a huge number of documents has been digitised, before undergoing optical character recognition (OCR) to extract their textual content. This step is crucial for indexing the documents and to make the resulting collections accessible. However, the fact that documents are indexed through their OCRed content is posing a number of problems, due to the varying performance of OCR methods over time. Indeed, OCR quality has a considerable impact on the indexing and therefore the accessibility of digital documents. Named entities are among the most adequate information to index documents, in particular in the case of digital libraries, for which log analysis studies have shown that around 80% of user queries include a named entity. Taking full advantage of the computational power of modern natural language processing (NLP) systems, named entity recognition (NER) can be operated over enormous OCR corpora efficiently. Despite progress in OCR, resulting text files still have misrecognised words (or noise for short) which are harming NER performance. In this paper, to handle this challenge, we apply a spelling correction method to noisy versions of a corpus with variable OCR error rates in order to quantitatively estimate the contribution of post-OCR correction to NER. Our main finding is that we can indeed consistently improve the performance of NER when the OCR quality is reasonable (error rates respectively between 2% and 10% for characters (CER) and between 10% and 25% for words (WER)). The noise correction algorithm we propose is both language-independent and with low complexity.

A significant portion of the textual information of interest to an organization is stored in PDF files that should be converted into plain text before their contents can be processed by an information retrieval or text mining system. When the PDF documents consist of scanned documents, optical character recognition (OCR) is typically used to extract the textual contents. OCR errors can have a negative impact on the quality of information retrieval systems since the terms in the query will not match incorrectly extracted terms in the documents. This work introduces sOCRates, a post-OCR text correction method that relies on contextual word embeddings and on a classifier that uses format, semantic, and syntactic features. Our experimental evaluation on a test collection in Portuguese showed that sOCRates can accurately correct errors and improve retrieval results.