A Novel System for Image Text Recognition and Classification using Deep Learning

Optical Character Recognition (OCR) and Handwritten Text Recognition (HTR) are technologies that enable text recognition. The difference between OCR and HTR is one designed specifically for digital text and one designed for handwritten text. There are already various implementations of OCR and HTR online. However, such systems do not guarantee the systems are in premises. To solve this problem, the OCR and HTR system must be built from the scratch. The purpose of this research is to improve the recognition by separating the text whether it is a handwritten or a printed text, which will later be forwarded into the appropriate recognition system. An application program interface (API) was also created in order to finalize the classification system into real world usage. In this research, the classification system being developed using convolutional neural network (CNN) method. To be able to reach the highest accuracy of the classification system, the experimentation and improvement about hyperparameters, dataset format, data augmentation and analysis on 3 CNN architectures were conducted. In the end of this research, there are 2 architectures in a tight competition, one is VGG-16 with 90.63% accuracy and one is AlexNet with 90.17% accuracy on ideal data testing. However, AlexNet is chosen as the winner after the testing with real data.

Abstract: A computer vision program called Handwritten Text Recognition (HTR) attempts to recognize and translate handwritten text from scanned or photographed images. In this project, we suggest implementing an HTR system using Tesseract and OpenCV. English, Chinese, and Arabic are all supported by the popular open-source optical character recognition (OCR) engine known as Tesseract. It is employed to find and identify printed text within photographs. On the other hand, OpenCV is a well-liked computer vision library that offers several tools for processing and analyzing images. The pre-processing step of the proposed system uses OpenCV to increase the input image's quality and OCR accuracy. After that, Tesseract receives the pre-processed image for text recognition. The extracted text is then saved in a text file after being identified. To enhance the quality of the input image, the project will use several pre-processing techniques, including deskewing, noise removal, and binarization. With the help of a sizable dataset of handwritten photographs, the Tesseract OCR engine is taught to recognize handwritten text more accurately. The HTR system can be used in a variety of fields, including document analysis, historical manuscript digitalization, and postal automation. It can also be applied in academic settings to help students translate their notes and assignments. Therefore, it is anticipated that the proposed HTR system employing Tesseract and OpenCV will offer a reliable and effective method for identifying and transcribing handwritten text from photographs.

In today’s digital environment, the pattern is everywhere. It is present in many aspects of our daily life. Algorithms can be used to detect or mathematically observe a pattern physically. Vector feature values represent a pattern in the digital world. With the advent of artificial intelligence (AI) techniques in the recent era, there are so many machine learning (ML) and deep learning (DL) models that have been developed. ML is the branch of AI, which can perform tasks like data analysis, analytical model building, and classification without being explicitly programmed. DL is the subset of ML in AI. In DL, mostly artificial neural networks (ANNs) are utilized. It has the capability of work based upon unsupervised learning from the data that is unlabeled and unstructured. Using these DL and ML models, extract the meaningful features from the given image or video is known as pattern recognition (PR). PR is used in many engineering applications such as computer vision, natural language processing, character recognition, robotics, speech recognition, and so on. It is also used in many medical image processing applications and telemedicine. The present work discussed the pattern recognition problem and its various stages in detail. In addition to that, the application of deep learning and machine learning in pattern recognition is also explained briefly.

Handwritten Text Recognition (HTR) is a challenging task due to the complex structures and variations present in handwritten text. In recent years, the application of gated mechanisms, such as Long Short-Term Memory (LSTM) networks, has brought significant advancements to HTR systems. This paper presents an overview of HTR using a gated mechanism and highlights its novelty and advantages. The gated mechanism enables the model to capture long-term dependencies, retain relevant context, handle variable length sequences, mitigate error propagation, and adapt to contextual variations. The pipeline involves preprocessing the handwritten text images, extracting features, modeling the sequential dependencies using the gated mechanism, and decoding the output into readable text. The training process utilizes annotated datasets and optimization techniques to minimize transcription discrepancies. HTR using a gated mechanism has found applications in digitizing historical documents, automatic form processing, and real-time transcription. The results show improved accuracy and robustness compared to traditional HTR approaches. The advancements in HTR using a gated mechanism open up new possibilities for effectively recognizing and transcribing handwritten text in various domains. This research does a better job than the most recent iteration of the HTR system when compared to five different handwritten datasets (Washington, Saint Gall, RIMES, Bentham and IAM). Smartphones and robots are examples of low-cost computing devices that can benefit from this research.

The optical character recognition technique is used to convert information, mainly printed or handwritten text in paper materials, into an electronic format that the computers can edit. According to the literature, there are few competent OCR systems for recognizing multilingual characters in the form of Sinhala and English characters together. The lack of an appropriate technology to recognize multilingual text still remains as a problem that the current research community must address, and it has been designated as the key problem for this study. The main goal of this research is to develop a multilingual character recognition system that uses character image geometry features and Artificial Neural Networks to recognize printed Sinhala and English scripts together. It is intended that the solution would be improved to cover three Sri Lanka’s most commonly spoken languages, with the addition of Tamil as a later upgrade. The primary technologies for this study were character geometry features and Artificial Neural Networks. At the moment almost an 85% of success rate has been achieved with a database containing around 800 images, which are divided into 46 characters (20 Sinhala and 26 English), and each character is represented in 20 different forms of character images. Recognition of text from printed bi-lingual documents is experimented by extracting individual character data from such printed text documents and feeding them to the system.

With continuous improvements in performance of microprocessors over the years, they now possess capabilities of supercomputers of earlier decade. Further with the continuous increase in the packaging density on the silicon and General Purpose Graphics Processing Unit (GPGPU) enhancements, has led to utilization the deep learning (DL) techniques, which had lost steam during the last decade. A GPGPU is a parallel programming setup using a combination of GPUs and CPUs that can manipulate large matrices. Interestingly, GPUs were created for faster graphic processing, but found its way into relevant scientific computing. DL is a subset of the artificial intelligence (AI) domain and falls specifically under the set of machine learning (ML) techniques which are based on learning data representations rather than task‐specific algorithms. It has been observed that the accuracy and the pragmatism of deploying DL at massive level was restricted by technological issues of executing DL based AI models, with extremely large training sessions running into weeks. DL applications can solve problems of very large order and areas like computer vision/image processing is one of the early successes and becoming quite a sensation in many areas such as natural language processing (NLP) with state of the art real‐time translation capabilities, automatic game playing, optical character recognition especially handwritten text, and so on. This overview traverses the evolution and successful adoption in the various industry verticals.This article is categorized under: Application Areas > Industry Specific Applications Application Areas > Business and Industry Technologies > Machine Learning

A study of several methods for combining information from two classifiers in a system for the recognition of handwritten text is presented. The system uses two hidden Markov models (HMMs) per character to model columns and rows of pixels in the character image. We show that the best method of combining the results from the vertical and horizontal classifiers is simply to multiply the probabilities produced by the two methods. This approach outperforms more complicated classifier combination strategies such as the behaviour-knowledge space (BKS) method.

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurones) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurones. The ANN has recently been applied in process control, identification, diagnostics, character recognition, sensory prediction, robot vision, and forecasting. In Textiles and Clothing industries, it involves the interaction of a large number of variables. Because of the high degree of variability in raw materials, multistage processing and a lack of precise control on process parameters, the relation between such variables and the product properties is relied on the human knowledge but it is not possible for human being to remember all the details of the process-related data over the years. As the computing power has substantially improved over last decade, the ANN is able to learn such datasets to reveal the unknown relation between various variables effectively. Therefore, the application of ANN is more widespread in textiles and clothing industries over last decade. In this chapter, it aims to review current application of ANN in textiles and clothing industries over last decade. Based on literature reviews, the challenges encountered by ANN used in the industries will be discussed and the potential future application of ANN in the industries will also be addressed. The structure of this chapter comprises of seven sections. The first section includes background of ANN, importance of ANN in textiles and clothing and the arrangement of this chapter. In forthcoming three sections, they include review of applications of ANN in fibres and yarns, in chemical processing, and in clothing over last decade. Afterwards, challenges encountered by ANN used in textiles and clothing industries will be discussed and potential future application of ANN in textiles and clothing industries will be addressed in last section.

Deep learning is one machine learning method based on the layers used in artificial neural networks. The breakthrough of deep learning in classification led to its rapid application in speech recognition, natural language understanding, and image processing and recognition. In the field of biodiversity informatics, deep learning efforts are being applied in rapid species identification and counts of individuals identified based on image, audio, video, and other data types. However, deep learning methods hold great potential for application in all aspects of biodiversity informatics. We present a case study illustrating how to increase data collection quality and efficiency using well-established technology such as optical character recognition (OCR) and some image classification. Our goal is to image data from the scanned documents of various butterfly atlases, add species, specimens, collections, photographs and other relevant information, and build a database of butterfly images. Information collection involves image annotation and text-based descritpion input. Although the work of image annotation is simple, this process can be accelerated by deep learning-based target segmentation to make the selection process easier, such as changing box select to a double click. The process of information collection is complicated, involving input of species names, specimen collection, specimen description, and other information. Generally, there are many images in atlases, the text layout is rather messy, and overall OCR success is poor. Therefore, the measures we take are as follows: Step A: select the screenshot of the text and then call the OCR interface to generate the text material; Step B: proceed with NLP- (natural language processing) related processing; Step C: perform manual operations on the results, and introduce the NLP function again to this process; Step D: submit the result. The deep learning applications we integrated in our client tool include: target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. Some simple machine learning methods such as k-nearest neighbor can be used to automatically determine gender, pose, and so on. While complex information such as collection place, time, and collector can be analyzed by deep learning-based NLP methods in the future. In our infomation collection process, ten fields are required to submit one single record. Of those, about 4-5 input fields can be dealt with the AI-assistant. It can thus be seen from the above process that deep learning has reduced the workload of manual information annotation by at least 30%. With improvements in accuracy, the era of using automatic information extraction robots to replace manual information annotation and collection is just around the corner.

With the wide application of artificial intelligence technology in intelligent education, intelligent office and other fields, handwritten optical character recognition (OCR) as an important content of intelligent education research has been widely concerned. Traditional character recognition technology is mainly for high-quality printed images, but there are some problems in English handwriting, such as irregular handwriting, overlapping letters, word smearing, etc., which can not achieve the recognition accuracy of printed images. In particular, the lack of database construction for handwriting images with scrawl and daub increases the difficulty of model training, so handwriting recognition is a very challenging task. In view of the existing problems mentioned above and the limitations of traditional handwriting recognition methods, this paper studies the English handwriting recognition method based on deep learning technology, and realizes the English handwriting image detection and recognition system on this basis. The main tasks of this paper are as follows: 1. The database of English handwriting image is established, including neat, scrawl, neat and smeared and scrawl and smeared. Label information is assigned to each type of data, and SVM classifier is trained to classify new samples using this database. In order to reduce the interference of smear area and background information on the character recognition model, the image is preprocessed with smear block removal, gray scale and tilt correction, and the processing results are used as the input of the subsequent detection and recognition model. 2. A convolutional cyclic neural network based on cross-channel and cross-space attention mechanism is proposed. The convolutional neural network is used to extract local feature information and cross-channel and cross-space attention mechanisms are introduced to reduce the loss of key features. At the same time, bidirectional recurrent neural network is used to capture the context dependence of characters and obtain the word order information of English script. 3. An English handwritten recognition method based on Semantic Reasoning Networks (SRN) and CCSACRNN (SNCN) is proposed.

In AEC (Architecture, Engineering & Construction) industry, drawing documents are used as a blueprint to facilitate the construction process. It is also represented as a graphical language that communicates ideas and information from one mind to another. In AEC documents, text is present in Machine-printed and hand-written format. Since the algorithms for recognition of machine-printed and hand-written texts are different, it is important to distinguish between these two types of texts before sending the document to respective recognition system. In this paper we proposed a novel approach for the classification machine-printed and hand-written text from AEC Documents. Before Classification Hand-Written and Machine-Printed text from the documents our system used some preprocessing which includes binarization, text graphics separation and word segmentation. The Words are segmented based on certain structural properties of Isothetic Covers (IC) tightly enclosing the words in a document. The grid size properties of IC are selected by some statistical analysis of connected component of the document. Then Word level Gabor Filter based features are extracted with spooling information for classification. A standard classifier based on SVM is used to classify the text. This task is performed at word level of AEC documents and we achieved an overall accuracy of 98.45%.

The advancement of computing and communication technologies has fueled the growth of Information Technology (IT), with Artificial Intelligence (AI) emerging as a transformative force in modernizing project management practices. This study explores the application of two prominent AI techniques—Decision Tree Analysis (DTA) and Artificial Neural Networks (ANN)—in improving project planning and control. A review of empirical studies highlights the limitations of conventional tools such as Gantt charts and the Critical Path Method (CPM) in managing complex project variables, often resulting in cost overruns and schedule delays. In contrast, DTA and ANN demonstrate superior predictive accuracy, decision support, and adaptability capabilities. DTA offers transparent and structured decision-making models, while ANN excels in pattern recognition and outcome forecasting. The findings underscore that integrating these AI tools enhances project efficiency, cost estimation, and time management, establishing AI as a critical asset for future project success.Keywords: Artificial Intelligence, Decision Tree Analysis, Artificial Neural Networks, Project Planning, Project Control

With the use of fuzzy logic techniques, neural computing can be integrated in symbolic reasoning to solve complex real world problems. In fact, artificial neural networks, expert systems, and fuzzy logic systems, in the context of approximate reasoning, share common features and techniques. A model of Fuzzy Connectionist Expert System is introduced, in which an artificial neural network is designed to construct the knowledge base of an expert system from, training examples (this model can also be used for specifications of rules in fuzzy logic control). Two types of weights are associated with the synaptic connections in an AND-OR structure: primary linguistic weights, interpreted as labels of fuzzy sets, and secondary numerical weights. Cell activation is computed through min-max fuzzy equations of the weights. Learning consists in finding the (numerical) weights and the network topology. This feedforward network is described and first illustrated in a biomedical application (medical diagnosis assistance from inflammatory-syndromes/proteins profiles). Then, it is shown how this methodology can be utilized for handwritten pattern recognition (characters play the role of diagnoses): in a fuzzy neuron describing a number for example, the linguistic weights represent fuzzy sets on cross-detecting lines and the numerical weights reflect the importance (or weakness) of connections between cross-detecting lines and characters.

evelopment of OCRs for Indian script is an active area of activity today. Optical character recognition (OCR) is the mechanical or electronic translation of images of handwritten, typewritten or printed text (usually captured by a scanner) into machine-editable text. In simple words OCR is a visual recognition process that turns printed or written text into an electronic character-based file. OCR is a field of research in pattern recognition, artificial intelligence and machine vision. Though academic research in the field continues, the focus on OCR has shifted to implementation of proven techniques. A lot of work had been carried out for OCR at international scenario but in Indian context a concrete approach for character recognition is still required as scripts of Indian languages are from the group of most complex scripts and it is very hard to recognize them. Indian scripts present great challenges to an OCR designer due to the large number of letters in the alphabet, the sophisticated ways in which they combine, and the complicated graphemes they result in. The problem is compounded by the unstructured manner in which popular fonts are designed. There is a lot of common structure in the different Indian scripts. All existing OCR systems developed for various Indian scripts do not provide sufficient efficiency due to various factors. The objective of this paper is to discuss a more efficient character recognition technique. This paper introduces a new technical approach to recognize Indian script characters which are unpredictable due to different problems in other OCR‟s.

The conversion of handwritten text into machine-readable format is termed as Handwritten Character Recognition (HCR). The differences in size, design, and alignment angle of the Telugu and Kannada alphabets have difficulty in recognizing handwritten documents in these languages across various real-world applications. Newly developed machine learning and deep learning models provide a significant improvements in the handwritten text recognition. These innovative methods offer promising enhancements in the accuracy and efficiency of character recognition within handwritten documents. However, effective recognition of digits is not an easy task due to people’s varying writing styles in the input sample. To overcome such limitations, we explore a novel approach specifically designed to boost the performance of HCR in South Indian languages such as Kannada and Telugu. Initially, handwritten images are gathered using traditional data sources. These collected images are then given into the recognition phase. Here, an Adaptive Dilated convolution-based Deep Network (ADC-DeepNet) is developed for character identification purposes. In ADC-DeepNet, the ShuffleNetV2 blends with the Bidirectional Long Short-Term Memory (Bi-LSTM) to produce accurate results. This fusion provides effective character recognition. Here, the Iterative Concept of Lyrebird Optimization (ICLO) is newly proposed to optimize the variables from ADC-DeepNet to improve the character recognition efficacy. The efficiency of the HCR system is evaluated among several recent techniques with some performance measures. Finally, the outcome showed that the accuracy of the proposed approach is 95.6, and other models like CNN, ResNet, Convolutional Autoencoder, and DeepNet gave the accuracy of 88.8, 91.5, 90.6, and 93.3, respectively. Thus, the findings of the experiment show that the developed ADC-DeepNet model can effectively identify the handwritten characters in south Indian languages.