Library Automation System: Book cover recognition using deep learning

Scene text detection and recognition are vital challenging tasks in computer vision, which are to detect and recognize sequences of texts in natural scenes. Recently, researchers have investigated a lot of state-of-the-art methods to improve the accuracy and efficiency of text detection and recognition. However, there has been little research on text detection and recognition in natural scenes in Vietnam. In this paper, a deep learning-based empirical investigation of Vietnamese scene text detection and recognition is presented. Firstly, four detection models including differentiable binarization network (DBN), pyramid mask text detector (PMTD), pixel aggregation network (PAN), and Fourier contour embedding network (FCEN), are employed to detect text regions from the images. Then, four text recognition models including convolutional recurrent neural network (CRNN), self-attention text recognition network (SATRN), no-recurrence sequence-to-sequence text recognizer (NRTR), and RobustScanner (RS) are also investigated to recognize the texts. Moreover, data augmentation methods are also applied to enrich data for improving the accuracy and enhancing the performance of scene text detection and recognition. To validate the effectiveness of scene text detection and recognition models, the VinText dataset is employed for evaluation. Empirical results show that PMTD and SATRN achieve the highest scores among the others for text detection and recognition, respectively. For knowledge-sharing, our implementation is publicly available at https://github.com/ThorPham/VN_scene_text_detection_recognition.

An internet of things (IoT) is an exclusive method, were its impact on the enactments of human life is very trendy. This research on library control system operates on the basis of IoT and optical character recognition (OCR) algorithm rules and its training. A closed-circuit television (CCTV) watched mechanism is created to control the book issuing and returning phenomenon via tag studying system in the library. In this proposed work text file is converted into an audio file. This audio file is being played and the contents of the book can be heard via the headset. This unique function of the OCR helps blind people. Now a days OCR widely focused in machine processes such as machine transformation, text to speech extraction and text data mining. It utilized in various area of research in artificial intelligence, computer vision and pattern recognition. Using OCR to scan the damaged book in the library converted into pdf format the book gets new life and sharing the contents to multiple readers. In this paper aims to implement IoT based library management system to maintaining books in digital format.

Lack of handwriting automatization in childhood can cause difficulties within and outside the school context. Therefore, the objective quantification of the handwriting performance is key. A Smart Ink Pen (SIP) used on paper demonstrated its validity in characterizing primary school children's handwriting process, although missing information on the handwriting product. To overcome this limitation, a trace reconstruction algorithm was developed, based on the force and IMU signals measured by the SIP. A total of 353 words "uno", written in cursive from the BVSCO-3 battery by 50 Italian students of the 5th grade of primary school, was reconstructed. The quality of the reconstructions was validated through an Optical Character Recognition (OCR) algorithm (Google Vision), using the scans of the actual traces as a reference. The character recognition rates were 81.02% and 59.49%, the character error rates 21.48% and 47.65%, for scans and reconstructions, respectively. A deeper analysis revealed that 15% of the reconstructions were read in the opposite direction by the OCR algorithm, likely due to a non sufficient sampling rate for the last portion of the words. A characterization of the differences between the good, bad and opposite reconstructions allowed to identify some directions of improvement. An increase of the SIP sampling rate, a better modeling of the thickness of the trace, a finer estimation of the relative distance between the IMU sensor and the tip and the reconstruction of the tip trajectory during in-air movements could improve the trace reconstruction algorithm. In addition, the possibility to leverage transfer learning approaches on the OCR algorithm using the reconstructed traces for additional training could further improve the performances in terms of character recognition rate. However, the preliminary results obtained are promising and highlight the possibility of reconstructing handwriting traces while maintaining the naturalness of the gesture.Clinical relevance- This contribution establishes the possibility of reconstructing handwritten traces from the kinematic and force signals recorded by an ecological sensorized ink pen.

In recent years, with the development of Neural Network, it has made a significant breakthrough in the field of text detection and recognition. However, large-scale deep Neural Network needs a large amount of storage space and computing resources, which makes it difficult to be effectively applied in various hardware platforms. In order to solve this problem, we quantize the text detection and recognition model from floating-point to low bit-width. Utilizing the low bit-width quantization method in DoReFa-Net, we quantize the text detection network, Connectionist Text Proposal Network (CTPN) and the text recognition network, Convolutional Recurrent Neural Network(CRNN). Comparing the results in different quantized bit-width, we find that after quantizing the weights and activations of CTPN to 2-bit, the F-score only drops by 0.09 on the MTWI 2018 Dataset. When the weights and activations of CRNN is quantized to 4-bits, the accuracy decrease of training with Synthetic Chinese String Dataset is merely 0.1%. After the compression of the text detection and recognition, the size of model is reduced to only 32% of the original, which means that storage space can be saved by 68%.

Since the COVID-19 pandemic, numerous jobs have become necessary, including the storing and sharing of printed material across computers. One simple way to save data from printed papers to a computer system is to scan them first and then save them as images. However, it would be quite challenging to extract or query text or other information from these photo files to reuse this information. As a result, a method for automatically retrieving and storing information, particularly text, from picture files is required. Optical character recognition (OCR) is an ongoing research topic that aims to create a computer system capable of extracting and processing text from images. To accomplish successful automation, certain significant problems must be identified and addressed. The font properties of characters in paper documents, as well as image quality, are only a few of the latest problems. Characters may not be recognized correctly by the computer system because of many complexities. So, in this study, authors look into OCR in four different contexts and apply them to get our results. However, every OCR is further followed by these two steps. First, a comprehensive explanation of the challenges that may develop during the OCR phases is provided. The key phases of an OCR system are then executed, including pre-processing, segmentation, normalization, feature extraction, classification, and post-processing. It can be used with deep learning software to provide OCR data which is very useful for robotic and AI applications.

At present, the accuracy of text detection from natural scene images based on deep learning is much higher than that of traditional text detection algorithm. However, most of the text detectors based on deep learning should use large-scale neural network models (such as VGG and ResNet) for improving the accuracy. Because the weight files of these models are very large (VGG is about 500MB and ResNet is about 100MB), they are not suitable for the computationally limited platforms in daily life. Based on the small-scale neural network model named MobileNet V2, and U-Net that is used in semantic segmentation commonly, a text detector for natural scene by using neural network with low complexity is proposed in this paper. The weight file of the designed neural network model is about 16MB, which is suitable for running on mobile platforms. The experiment result based on the ICDAR 2013 dataset proves that this model has good performance in text detection.

Text recognition in the wild is a challenging task in the field of computer vision and machine learning. Existing optical character recognition engines cannot perform well in the natural scene. In this context, deep learning models have emerged as a powerful state-of-the-art technique in the classification and recognition process. This study proposes a new Convolutional Neural Network based system for scene text reading. We investigate how to combine the character recognition module followed by the word recognition module to achieve the overall system goal. The first module analyzes characters within multi-scale images by relaying on the power of the convolutional network and the fully connected network for character recognition. The second module relies on the Viterbi search to find the closest word to a given characters sequence. For the sake of more precision, a bigram based linguistic module is applied. The proposed system achieves the state-of-the-art performance on three standard scene text recognition benchmarks: chars74k, ICDAR 2003 and ICDAR 2013. In particular, this performance is proven on both of character and word recognition accuracy as well as speed aspects via a comparative study between different deep learning architectures.

Visual attention models have been used to recognize the most prominent region in a natural scene. These regions are going to pull the human attention. The state-of-art models keep on under-predicting the significant image regions having text. These are specifically the regions with most noteworthy semantic significance in a natural scene and turn out to be useful for saliency-based applications like image classification and captioning. The text or character detection as a salient region in image remains a challenging research problem. Text contents within the scene convey vital information about the image. For example, signboard content conveys the important information for visually impaired person. In this paper, we have proposed a new model for salient text detection in a natural scene. In the proposed model, we integrate saliency model with the segmentation and text detection approach in a natural scene to generate the text saliency. The experimental outcomes in ROC curve and DET curves illustrate that the proposed model outperformed the state-of-art methods for detection of salient text content from a natural scene.

Character recognition has been very popular and interested area for researches, and it continues to be a challenging and impressive research topic due to its diverse applicable environment. The optical character recognition has been introduced as a fast and accurate method to convert both existing text images as well as large archives of existing paper documents to editable digital text format. However, existing optical character recognition algorithms suffer from flawed tradeoffs between accuracy and speed, making them less effective and impractical for large and complex documents. This paper describes a suggested method for Assyrian optical character recognition using modified back propagation artificial neural network based on moments. The experimental results show that the proposed method achieves higher recognition accuracy rate in compared with the standard algorithm.

The trend of personalized medicine and the increasing flexibility of drug dosage relevant goals of the 21st century represent the foundation for the current research. To obtain doses smaller than the smallest available, physicians frequently write prescriptions for children and adults, without preserving the integrity of the pill. Moreover, patients purchase large amounts of medication for cost-saving reasons. To support the correct administration of the remedies and the partial alignment to the personalized treatment trend, this paper proposes a flexible and user-friendly solution for determining the medication quantity given to patients, using augmented reality and optical character recognition algorithm capabilities. Via the MATLAB development environment and a Logitech HD Pro C920 webcam, the results were 80% correct in identifying the cutting position of the pill, by means of the Hough transform, and 30% correct in weight recognition exploitation using an optical character recognition (OCR) algorithm. In future work, a higher resolution camera and a more powerful computer can be used to increase the percentages mentioned above. In addition, a 3D scan of the pill fragmentation, combined with the weight recognition functionality, could increase the accuracy of the splitting procedure, conducted by the patient or the patient caretaker.

PurposeDigitization of historical documents is a challenging task in many digital humanities projects. A popular approach for digitization is to scan the documents into images, and then convert images into text using optical character recognition (OCR) algorithms. However, the outcome of OCR processing of historical documents is usually inaccurate and requires post-processing error correction. The purpose of this paper is to investigate how crowdsourcing can be utilized to correct OCR errors in historical text collections, and which crowdsourcing methodology is the most effective in different scenarios and for various research objectives.Design/methodology/approachA series of experiments with different micro-task’s structures and text lengths were conducted with 753 workers on the Amazon’s Mechanical Turk platform. The workers had to fix OCR errors in a selected historical text. To analyze the results, new accuracy and efficiency measures were devised.FindingsThe analysis suggests that in terms of accuracy, the optimal text length is medium (paragraph-size) and the optimal structure of the experiment is two phase with a scanned image. In terms of efficiency, the best results were obtained when using longer text in the single-stage structure with no image.Practical implicationsThe study provides practical recommendations to researchers on how to build the optimal crowdsourcing task for OCR post-correction. The developed methodology can also be utilized to create golden standard historical texts for automatic OCR post-correction.Originality/valueThis is the first attempt to systematically investigate the influence of various factors on crowdsourcing-based OCR post-correction and propose an optimal strategy for this process.

In today’s digital world, we are committed to digitizing thousands of handwritten transcriptions to preserve their content. Historical Arabic Handwritten Text Recognition (HAHTR) remains a challenge for computer vision systems, due to the many difficulties inherently associated with document image quality and the complexity of Arabic script. In this work, we address the problem of recognizing historical Arabic documents that adapts to different writing styles and degrees of legibility. We developed a system that is able to recognize a whole page of a historical Arabic handwritten text in two consecutive steps comprising text line detection and recognition. The proposed approach performs detection using bounding boxes followed by a neural network-based model for character-level text recognition. However, the lack of data hinders the mass digitization of Arabic historical documents. Therefore, we provide a new and freely available dataset, focusing on diverse handwriting styles to facilitate a strong generalization of the trained model. This dataset will significantly benefits researchers and practitioners by accelerating progress in the field of HAHTR. Extensive experimental work demonstrates that the recognition models are effective when trained with different sources of data, and having different writing styles does not penalize the model’s ability to generalize but rather enhances it. Additionally, we define and develop a new metric to evaluate model robustness against character misclassification, particularly for characters with similar patterns. The experiments conducted demonstrated that the proposed HAHTR pipeline is accurate and highly generalizable, as well as the validity of bounding box methods for detecting text lines. The training approach with different data sources enabled us to surpass the state-of-the-art results with 5.7% of Character Error Rate (CER) on the KHATT database.

With the wide application of artificial intelligence technology in intelligent education, intelligent office and other fields, handwritten optical character recognition (OCR) as an important content of intelligent education research has been widely concerned. Traditional character recognition technology is mainly for high-quality printed images, but there are some problems in English handwriting, such as irregular handwriting, overlapping letters, word smearing, etc., which can not achieve the recognition accuracy of printed images. In particular, the lack of database construction for handwriting images with scrawl and daub increases the difficulty of model training, so handwriting recognition is a very challenging task. In view of the existing problems mentioned above and the limitations of traditional handwriting recognition methods, this paper studies the English handwriting recognition method based on deep learning technology, and realizes the English handwriting image detection and recognition system on this basis. The main tasks of this paper are as follows: 1. The database of English handwriting image is established, including neat, scrawl, neat and smeared and scrawl and smeared. Label information is assigned to each type of data, and SVM classifier is trained to classify new samples using this database. In order to reduce the interference of smear area and background information on the character recognition model, the image is preprocessed with smear block removal, gray scale and tilt correction, and the processing results are used as the input of the subsequent detection and recognition model. 2. A convolutional cyclic neural network based on cross-channel and cross-space attention mechanism is proposed. The convolutional neural network is used to extract local feature information and cross-channel and cross-space attention mechanisms are introduced to reduce the loss of key features. At the same time, bidirectional recurrent neural network is used to capture the context dependence of characters and obtain the word order information of English script. 3. An English handwritten recognition method based on Semantic Reasoning Networks (SRN) and CCSACRNN (SNCN) is proposed.

With the rapid development of artificial intelligence (AI), the application of this technology in the medical field is becoming increasingly extensive, along with a gradual increase in the amount of intelligent equipment in hospitals. Service robots can save human resources and replace nursing staff to achieve some work. In view of the phenomenon of mobile service robots' grabbing and distribution of patients' drugs in hospitals, a real‐time object detection and positioning system based on image and text information is proposed, which realizes the precise positioning and tracking of the grabbing objects and completes the grasping of a specific object (medicine bottle). The lightweight object detection model NanoDet is used to learn the features of the grasping objects and the object category, and bounding boxes are regressed. Then, the images in the bounding boxes are enhanced to overcome unfavourable factors, such as a small object region. The text detection and recognition model PP‐OCR is used to detect and recognise the enhanced images and extract the text information. The object information provided by the two models is fused, and the text recognition result is matched with the object detection box to achieve the precise positioning of the grasping object. The kernel correlation filter (KCF) tracking algorithm is introduced to achieve real‐time tracking of specific objects to precisely control the robot's grasping. Both deep learning models adopt lightweight networks to facilitate direct deployment. The experiments show that the proposed robot grasping detection system has high reliability, accuracy and real‐time performance.

A Metaverse text recognition model based on character-level contrastive learning