Handwritten text generation and strikethrough characters augmentation

Training Handwritten Text Recognition (HTR) models typically requires large amounts of labeled data which often are line or page images with corresponding line-level ground truth (GT) transcriptions. Many digital collections have page-level transcriptions for each image, but the transcription is unformatted, i.e., line breaks are not annotated. Can we train lined-based HTR models using such data? In this work, we present a novel alignment technique for segmenting page-level GT text into text lines during HTR model training. This text segmentation problem is formulated as an optimization problem to minimize the cost of aligning predicted lines with the GT text. Using both simulated and HTR model predictions, we show that the alignment method identifies line breaks accurately, even when the predicted lines have high character error rates (CER). We removed the GT line breaks from the ICDAR-2017 READ dataset and trained a HTR model using the proposed alignment method to predict line breaks on-the-fly. This model achieves comparable CER w.r.t. to the same model trained with the GT line breaks. Additionally, we downloaded an online digital collection of 50K English journal pages (not curated for HTR research) whose transcriptions do not contain line breaks, and achieve 11% CER.

Supporting programming on touchscreen devices requires effective text input and editing methods. Unfortunately, the virtual keyboard can be inefficient and uses valuable screen space on already small devices. Recent advances in stylus input make handwriting a potentially viable text input solution for programming on touchscreen devices. The primary barrier, however, is that handwriting recognition systems are built to take advantage of the rules of natural language, not those of a programming language. In this paper, we explore this particular problem of handwriting recognition for source code. We collect and make publicly available a dataset of handwritten Python code samples from 15 participants and we characterize the typical recognition errors for this handwritten Python source code when using a state-of-the-art handwriting recognition tool. We present an approach to improve the recognition accuracy by augmenting a handwriting recognizer with the programming language grammar rules. Our experiment on the collected dataset shows an 8.6% word error rate and a 3.6% character error rate which outperforms standard handwriting recognition systems and compares favorably to typing source code on virtual keyboards.

Handwritten Text Recognition (HTR) is a challenging task due to the complex structures and variations present in handwritten text. In recent years, the application of gated mechanisms, such as Long Short-Term Memory (LSTM) networks, has brought significant advancements to HTR systems. This paper presents an overview of HTR using a gated mechanism and highlights its novelty and advantages. The gated mechanism enables the model to capture long-term dependencies, retain relevant context, handle variable length sequences, mitigate error propagation, and adapt to contextual variations. The pipeline involves preprocessing the handwritten text images, extracting features, modeling the sequential dependencies using the gated mechanism, and decoding the output into readable text. The training process utilizes annotated datasets and optimization techniques to minimize transcription discrepancies. HTR using a gated mechanism has found applications in digitizing historical documents, automatic form processing, and real-time transcription. The results show improved accuracy and robustness compared to traditional HTR approaches. The advancements in HTR using a gated mechanism open up new possibilities for effectively recognizing and transcribing handwritten text in various domains. This research does a better job than the most recent iteration of the HTR system when compared to five different handwritten datasets (Washington, Saint Gall, RIMES, Bentham and IAM). Smartphones and robots are examples of low-cost computing devices that can benefit from this research.

Optical Character Recognition (OCR) and Handwritten Text Recognition (HTR) are technologies that enable text recognition. The difference between OCR and HTR is one designed specifically for digital text and one designed for handwritten text. There are already various implementations of OCR and HTR online. However, such systems do not guarantee the systems are in premises. To solve this problem, the OCR and HTR system must be built from the scratch. The purpose of this research is to improve the recognition by separating the text whether it is a handwritten or a printed text, which will later be forwarded into the appropriate recognition system. An application program interface (API) was also created in order to finalize the classification system into real world usage. In this research, the classification system being developed using convolutional neural network (CNN) method. To be able to reach the highest accuracy of the classification system, the experimentation and improvement about hyperparameters, dataset format, data augmentation and analysis on 3 CNN architectures were conducted. In the end of this research, there are 2 architectures in a tight competition, one is VGG-16 with 90.63% accuracy and one is AlexNet with 90.17% accuracy on ideal data testing. However, AlexNet is chosen as the winner after the testing with real data.

Most Facial Expression Recognition (FER) systems rely on machine learning approaches that require large databases for an effective training. As these are not easily available, a good solution is to augment the databases with appropriate data augmentation (DA) techniques, which are typically based on either geometric transformation or oversampling augmentations (e.g., generative adversarial networks (GANs)). However, it is not always easy to understand which DA technique may be more convenient for FER systems because most state-of-the-art experiments use different settings which makes the impact of DA techniques not comparable. To advance in this respect, in this paper, we evaluate and compare the impact of using well-established DA techniques on the emotion recognition accuracy of a FER system based on the well-known VGG16 convolutional neural network (CNN). In particular, we consider both geometric transformations and GAN to increase the amount of training images. We performed cross-database evaluations: training with the "augmented" KDEF database and testing with two different databases (CK+ and ExpW). The best results were obtained combining horizontal reflection, translation and GAN, bringing an accuracy increase of approximately 30%. This outperforms alternative approaches, except for the one technique that could however rely on a quite bigger database.

Handwriting recognition as challenging task for a machine with lot of variations in different aspects like tilt, contrast, strokes being different for every person even being different for the same person. This paper is assessing state of the art models for comparison and provides a simple deep learning model based on convolutional network and Connectionist Temporal Classification (CTC). The accuracy accomplished after implementation is not high up as state-of-the-art models but gives a general architecture to start from for handwritten text recognition (HTR). This work also provides a comparison between two stated decoding schemes based on CTC technique. The model in this paper has been trained on IAM dataset with "lines" images. The model gives a word error rate (WER) of 37% and character error rate of 14.8%. The model aims to predict the text of the image line at once instead of segmenting the lines into words

In recent years, since deep learning technology have been applied to handwritten text recognition, the need for handwritten document image Datasets has been growing more and more. In particular, the development of the dataset is of great significance for improving performance of handwritten Korean text recognition because no dataset for handwritten Korean text recognition has been published. In this paper, we present the “RanPil”, a new training and performance evaluation dataset for handwritten Korean text recognition, which consists of a total of 8,600 pages of images (182,000 text lines and 4,300,000 characters) written by 1,804 people. We evaluate writing- diversity of handwritten document images, such as text line spacing, text line slope, character size, word spacing, and character compactness. In addition, we propose an MOS (Mean Opinion Score) evaluation method for the scrawl-level. Finally, we evaluate the performance of TrOCR based on vision encoder and decoder with a test dataset classified by the scrawl-levels.

Recently, an effective segmentation-free approach via deep neural network based hidden Markov model (DNN-HMM) was proposed and successfully applied to offline handwritten Chinese text recognition. In this study, to further improve the modeling capability, we adopt deep convolutional neural networks (DCNN) to calculate the HMM state posteriors. First, on the frame basis, the DCNN-HMM can automatically learn the features from the raw image of the handwritten text line via the convolutional architecture rather than the handcrafted gradient features using in the DNN-HMM. Second, we examine several important factors of DCNN to the recognition performance, namely the kernel size, the number of blocks and convolutional layers. We also improve the language modeling by using more text data and high-order N-gram. Tested on ICDAR 2013 competition task of CASIA-HWDB database, the proposed DCNN-HMM could achieve a character error rate (CER) of 4.07%, yielding a relative CER reduction of 30.8\% over the DNN-HMM approach. To the best of our knowledge, this is the best published result of the segmentation-free approaches. Furthermore, we explain why DCNN-HMM is more effective than DNN-HMM via the visualization of feature learning and the error pattern analysis.

In AEC (Architecture, Engineering & Construction) industry, drawing documents are used as a blueprint to facilitate the construction process. It is also represented as a graphical language that communicates ideas and information from one mind to another. In AEC documents, text is present in Machine-printed and hand-written format. Since the algorithms for recognition of machine-printed and hand-written texts are different, it is important to distinguish between these two types of texts before sending the document to respective recognition system. In this paper we proposed a novel approach for the classification machine-printed and hand-written text from AEC Documents. Before Classification Hand-Written and Machine-Printed text from the documents our system used some preprocessing which includes binarization, text graphics separation and word segmentation. The Words are segmented based on certain structural properties of Isothetic Covers (IC) tightly enclosing the words in a document. The grid size properties of IC are selected by some statistical analysis of connected component of the document. Then Word level Gabor Filter based features are extracted with spooling information for classification. A standard classifier based on SVM is used to classify the text. This task is performed at word level of AEC documents and we achieved an overall accuracy of 98.45%.

Improving the prediction of extreme wind speed events with generative data augmentation techniques

The Khmer script presents a unique challenge due to its distinct characters, which make recognizing handwriting especially challenging. This study tackles this challenge by creating a specialized computer program that recognizes handwritten Khmer text. Our goal is to enhance Khmer language education by offering a valuable tool that supports educators and students in improving their Khmer language proficiency. This user-friendly tool improves handwriting skills and the program's adaptability and accuracy across handwriting styles, making learning more stimulating and effective. To accomplish this objective, we are beginning an extensive data collection initiative, creating a diverse dataset of handwritten Khmer samples. This dataset forms the foundation of our program's training, facilitating precise recognition. Despite the size of the dataset and the computational resource requirements, we are optimistic about this tool's potential impact on Khmer language education. Beyond improving handwriting, our program aims to improve Khmer educational material accessibility and efficiency. Utilizing a sequence-to-sequence-based model, particularly the LSTM-based encoder-decoder architecture, demonstrates our commitment to achieving the highest accuracy. Our model's performance is evaluated using the character error rate (CER), a metric that represents its precision in recognizing individual characters. With a dataset comprising 50,720 Khmer words, our model currently achieves an error rate of 3.36%, highlighting its effectiveness. By preserving and developing the Khmer language with advanced neural networks, our Khmer handwriting recognition tool preserves culture and promotes education in Cambodia. This tool has practical implications for enhancing teaching and learning experiences in educational settings by making educational materials more accessible and providing instant feedback on handwriting and text recognition.

Recently, hidden Markov models (HMMs) have achieved promising results for offline handwritten Chinese text recognition. However, due to the large vocabulary of Chinese characters with each modeled by a uniform and fixed number of hidden states, a high demand of memory and computation is required. In this study, to address this issue, we present parsimonious HMMs via the state tying which can fully utilize the similarities among different Chinese characters. Two-step algorithm with the data-driven question-set is adopted to generate the tied-state pool using the likelihood measure. The proposed parsimonious HMMs with both Gaussian mixture models (GMMs) and deep neural networks (DNNs) as the emission distributions not only lead to a compact model but also improve the recognition accuracy via the data sharing for the tied states and the confusion decreasing among state classes. Tested on ICDAR-2013 competition database, in the best configured case, the new parsimonious DNN-HMM can yield a relative character error rate (CER) reduction of 6.2%, 25% reduction of model size and 60% reduction of decoding time over the conventional DNN-HMM. In the compact setting case of average 1-state HMM, our parsimonious DNN-HMM significantly outperforms the conventional DNN-HMM with a relative CER reduction of 35.5%.

The study aims to comprehensively analyse existing methods and technologies for handwritten text recognition to justify the optimal approach to developing an effective recognition system for mobile applications.Methodology.The work uses a systematic literature review method to analyse scientific publications and technical documentation in handwritten text recognition.A comparative analysis was used to evaluate the advantages and disadvantages of different approaches, from traditional methods (pattern recognition, feature extraction) to modern solutions based on machine learning (k-NN, SVM) and deep neural networks (CNN, RNN).An analytical method was used to justify the optimal solution's choice theoretically, and a synthetic method was used to form a hybrid approach.Results.A detailed analysis of eight categories of handwritten text recognition methods was conducted.It was found that traditional methods have limited effectiveness due to their low adaptability to variations in handwriting.Machine learning algorithms show better results but depend on the quality of pre-processing.Deep learning methods demonstrate the highest efficiency due to automatic feature extraction and modelling of complex dependencies.A combined CNN + RNN + CTC approach is justified as the optimal solution for handwritten text recognition.Scientific novelty.A comprehensive systematisation of handwritten text recognition methods has been carried out, considering the specifics of mobile platforms.The advantages of a hybrid architecture combining convolutional and recurrent neural networks with the CTC function for recognition tasks without prior segmentation have been scientifically substantiated.A methodological approach to selecting optimal algorithms based on criteria of accuracy, speed, and adaptability to mobile constraints has been developed.Practical significance.The research results provide a theoretical basis for developing practical mobile applications for handwritten text recognition.The proposed approach ensures high recognition accuracy, adaptability to different handwriting styles, and optimisation for mobile devices.Developers can use the systematisation of methods to make informed decisions about technology selection.The research opens up prospects for creating Ukrainian-language recognition systems and further developing the field of digital processing of handwritten documents.

This work presents a dialect-aware and noise-robust Kannada automatic speech recognition (ASR) system that bridges the gap between low-resource linguistic contexts and state-of-the-art deep learning models. We design a two-stage approach: (i) a scratch-built convolutional neural network (CNN)–Transformer hybrid trained on curated Kannada speech data with fast Fourier transform-based noise reduction and (ii) fine-tuning OpenAI’s Whisper-small model on a dialect-diverse corpus. The proposed pipeline integrates adaptive noise suppression, subword tokenization, and beam-search decoding to handle agglutinative morphology, speaker variation, and environmental noise. Extensive experiments were conducted on two datasets: the Few-shot Learning Evaluation of Universal Representations of Speech (FLEURS) multilingual Kannada subset and a curated 150-sample, Custom-collected Kannada speech dataset covering both formal and conversational speech. On the FLEURS test set (≈ 2.5 hours), our fine-tuned Whisper model achieves a word error rate (WER) of 0.15, a character error rate (CER) of 0.255, and a BLEU score of 0.912, representing a 53% relative reduction in WER and 36% reduction in CER compared to the scratch CNN–Transformer baseline. For the customized dataset, the fine-tuned Whisper model achieves a WER of 0.2311 and a CER of 0.0453, outperforming Google Speech-to-Text Application Programming Interface (API) by 16.8% (relative WER reduction) and surpassing the scratch transformer by over 70% in WER. We further evaluate robustness under dialectal variation and noisy recordings, providing detailed error analysis and computational efficiency metrics. To our knowledge, this is the first comprehensive evaluation of Whisper fine-tuning for Kannada, demonstrating its viability for real-time, edge-deployable applications in education, accessibility, and public administration.

This work explores the use of Large Language Models (LLMs) for post-OCR spelling correction in full sentences across Portuguese, French, and English. Using outputs from a state-of-the-art recognition model on the BRESSAY, RIMES, and IAM datasets, we evaluated two different zero-shot prompts. Closed LLMs, such as Gemini and GPT series, consistently outperform open-source models in reducing Character Error Rate (CER) and Word Error Rate (WER), while also offering faster inference. Despite the good accuracy of open models, their high computational demands hinder their practical use. Code is available at https://github.com/savi8sant8s/zero-shot-spelling-corrector.