Convolutional Neural Network Based Intelligent Handwritten Document Recognition

In today's world automatic license plate recognition play an important role in monitoring and organizing vehicles. In this paper, we propose a method of detecting and recognizing the license plates of vehicles in an automatic way in our country. The system can be used to collect toll, in car parking and find stolen vehicles. We have used different image processing techniques like resizing image, image binarization, connected component analysis, image enhancement and noise filtering. Our system is composed of four main modules, such as detection of the license plate area, extraction of license plate. Then, segmentation of characters and words and finally recognition of the characters and words. As Bangladesh Road Transport Authority (BRTA) imposed a common standard for vehicle license plate, the size and aspect ratio of all license plates are same. We have used a threshold value to detect and extract the license plate based on our analysis. Afterward, for character segmentation, we used connected components. Later, we used deep learning tool ‘the Convolutional Neural Network’ for character recognition. Due to the lack of a standard data set, we have developed a customized dataset of Bangladeshi number plate for the implementation of our method. The accuracy of our proposed system is 95.42%.

Font recognition is one of the essential issues in document recognition and analysis, and is frequently a complex and time-consuming process. Many techniques of optical character recognition (OCR) have been suggested and some of them have been marketed, however, a few of these techniques considered font recognition. The issue of OCR is that it saves copies of documents to make them searchable, but the documents stop having the original appearance. To solve this problem, this paper presents a system for recognizing three and six English fonts from character images using Convolution Neural Network (CNN), and then compare the results of proposed system with the two studies. The first study used NCM features and SVM as a classification method, and the second study used DP features and SVM as classification method. The data of this study were taken from Al-Khaffaf dataset [21]. The two types of datasets have been used: the first type is about 27,620 sample for the three fonts classification and the second type is about 72,983 sample for the six fonts classification and both datasets are English character images in gray scale format with 8 bits. The results showed that CNN achieved the highest recognition rate in the proposed system compared with the two studies reached 99.75% and 98.329 % for the three and six fonts recognition, respectively. In addition, CNN got the least time required for creating model about 6 minutes and 23- 24 minutes for three and six fonts recognition, respectively. Based on the results, we can conclude that CNN technique is the best and most accurate model for recognizing fonts.

In this study, a method of predicting the seismic responses of building structures based on a convolutional neural network (CNN) is proposed. In the method, the time histories of acceleration responses previously measured in a building during earthquakes are used in the CNN input layer, with the corresponding time histories of the displacement responses being used in the CNN output layer. The correlations between the features automatically extracted from the acceleration responses by the convolution and pooling operations in the various CNN layers and the displacement responses in the CNN output layer are established through CNN training. The trained CNN predicts the displacement responses for a future earthquake event using only the measured acceleration responses. To verify the validity of the proposed method, a numerical study on the ASCE benchmark model and an experimental study on a reinforced concrete frame structure are conducted. In the numerical study, the structural responses of the ASCE model subject to artificial earthquakes are utilized for CNN training, and the performance of the trained CNN is verified through seismic response prediction. In the experimental study, shaking table tests are performed for various earthquakes to measure the seismic responses of the reinforced concrete test structure. The seismic response prediction performance of the proposed method is examined using CNN trained with the measured seismic responses. Furthermore, the performances of CNNs trained with different numbers of datasets generated by the proposed data generation method based on data overlapping with the same data pool are discussed with related to the number of CNN training iteration number.

Accurate and reliable detection of white matter hyperintensities and their volume quantification can provide valuable clinical information to assess neurologic disease progression. In this work, a stacked generalization ensemble of orthogonal 3D convolutional neural networks, StackGen-Net, is explored for improving automated detection of white matter hyperintensities in 3D T2-FLAIR images. Individual convolutional neural networks in StackGen-Net were trained on 2.5D patches from orthogonal reformatting of 3D-FLAIR (n = 21) to yield white matter hyperintensity posteriors. A meta convolutional neural network was trained to learn the functional mapping from orthogonal white matter hyperintensity posteriors to the final white matter hyperintensity prediction. The impact of training data and architecture choices on white matter hyperintensity segmentation performance was systematically evaluated on a test cohort (n = 9). The segmentation performance of StackGen-Net was compared with state-of-the-art convolutional neural network techniques on an independent test cohort from the Alzheimer's Disease Neuroimaging Initiative-3 (n = 20). StackGen-Net outperformed individual convolutional neural networks in the ensemble and their combination using averaging or majority voting. In a comparison with state-of-the-art white matter hyperintensity segmentation techniques, StackGen-Net achieved a significantly higher Dice score (0.76 [SD, 0.08], F1-lesion (0.74 [SD, 0.13]), and area under precision-recall curve (0.84 [SD, 0.09]), and the lowest absolute volume difference (13.3% [SD, 9.1%]). StackGen-Net performance in Dice scores (median = 0.74) did not significantly differ (P = .22) from interobserver (median = 0.73) variability between 2 experienced neuroradiologists. We found no significant difference (P = .15) in white matter hyperintensity lesion volumes from StackGen-Net predictions and ground truth annotations. A stacked generalization of convolutional neural networks, utilizing multiplanar lesion information using 2.5D spatial context, greatly improved the segmentation performance of StackGen-Net compared with traditional ensemble techniques and some state-of-the-art deep learning models for 3D-FLAIR.

Information extraction and coding of free-text pathology reports is an important activity for cancer registries to support national cancer surveillance. Cancer registrars must process high volumes of pathology reports on an annual basis. In this study, we investigated an automated approach using a coarse-to-fine training of convolutional neural networks (CNNs) for extracting the primary site, histological grade and laterality from unstructured cancer pathology text reports. Our proposed training scheme consists of two stages. In the first stage, the multi-task learning (MTL) with hard parameter sharing approach is used to train a multi-task MT-CNN model for all the tasks. Then, the TM-CNN model parameters are used to initialize a CNN model for each task to be fine trained individually using its corresponding dataset. The performance of our proposed approach was compared against a state-of-the-art CNN and the commonly used SVM classifier. We observed that the proposed model consistently outperformed the base line models, especially for the less prevalent classes. Specifically, the proposed training approach achieved a micro-F score of 0.7749 over 12 ICD-O-3 topography codes which is a significant improvement as compared with state-of-the-art CNN (0.7101) and the SVM (0.6019) classifiers. Also, the results demonstrate the potential of the proposed method for handling class imbalance within each task. It significantly improves macro-F score by 24% and 12% of the primary site and histology grade tasks, respectively.

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.

Functional imaging has successfully been applied to capture functional changes in the pathological tissues of a body in recent years. Nuclear medicine functional imaging has been used to acquire information about areas of concerns (e.g. lesions and organs) in a non‐invasive manner, enabling semi‐automated or automated decision‐making for disease diagnosis, treatment, evaluation, and prediction. Focusing on functional nuclear medicine images, in this study, the authors review existing work on the classification of single‐photon emission computed tomography, positron emission tomography, and their hybrid modalities with computed tomography and magnetic resonance imaging images by using convolutional neural network (CNN) techniques. Specifically, they first present an overview of nuclear imaging and the CNN technique, such as nuclear imaging modalities, nuclear image data format, CNN architecture, and the main CNN classification models. According to the diseases of concern, they then classify the existing CNN‐based work on the classification of functional nuclear images into three different categories. For the typical work in each of these categories, they present details about their research objectives, adopted CNN models, and achieved main results. Finally, they discuss research challenges and directions for developing technological solutions to classify nuclear medicine images based on the CNN technique.

Millions of clinicians rely daily on automated preliminary electrocardiogram (ECG) interpretation. Critical comparisons of machine learning-based automated analysis against clinically accepted standards of care are lacking. To use readily available 12-lead ECG data to train and apply an explainability technique to a convolutional neural network (CNN) that achieves high performance against clinical standards of care. This cross-sectional study was conducted using data from January 1, 2003, to December 31, 2018. Data were obtained in a commonly available 12-lead ECG format from a single-center tertiary care institution. All patients aged 18 years or older who received ECGs at the University of California, San Francisco, were included, yielding a total of 365 009 patients. Data were analyzed from January 1, 2019, to March 2, 2021. A CNN was trained to predict the presence of 38 diagnostic classes in 5 categories from 12-lead ECG data. A CNN explainability technique called LIME (Linear Interpretable Model-Agnostic Explanations) was used to visualize ECG segments contributing to CNN diagnoses. Area under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated for the CNN in the holdout test data set against cardiologist clinical diagnoses. For a second validation, 3 electrophysiologists provided consensus committee diagnoses against which the CNN, cardiologist clinical diagnosis, and MUSE (GE Healthcare) automated analysis performance was compared using the F1 score; AUC, sensitivity, and specificity were also calculated for the CNN against the consensus committee. A total of 992 748 ECGs from 365 009 adult patients (mean [SD] age, 56.2 [17.6] years; 183 600 women [50.3%]; and 175 277 White patients [48.0%]) were included in the analysis. In 91 440 test data set ECGs, the CNN demonstrated an AUC of at least 0.960 for 32 of 38 classes (84.2%). Against the consensus committee diagnoses, the CNN had higher frequency-weighted mean F1 scores than both cardiologists and MUSE in all 5 categories (CNN frequency-weighted F1 score for rhythm, 0.812; conduction, 0.729; chamber diagnosis, 0.598; infarct, 0.674; and other diagnosis, 0.875). For 32 of 38 classes (84.2%), the CNN had AUCs of at least 0.910 and demonstrated comparable F1 scores and higher sensitivity than cardiologists, except for atrial fibrillation (CNN F1 score, 0.847 vs cardiologist F1 score, 0.881), junctional rhythm (0.526 vs 0.727), premature ventricular complex (0.786 vs 0.800), and Wolff-Parkinson-White (0.800 vs 0.842). Compared with MUSE, the CNN had higher F1 scores for all classes except supraventricular tachycardia (CNN F1 score, 0.696 vs MUSE F1 score, 0.714). The LIME technique highlighted physiologically relevant ECG segments. The results of this cross-sectional study suggest that readily available ECG data can be used to train a CNN algorithm to achieve comparable performance to clinical cardiologists and exceed the performance of MUSE automated analysis for most diagnoses, with some exceptions. The LIME explainability technique applied to CNNs highlights physiologically relevant ECG segments that contribute to the CNN's diagnoses.

Image denoising faces significant challenges, arising from the sources of noise. Specifically, Gaussian, impulse, salt, pepper, and speckle noise are complicated sources of noise in imaging. Convolutional neural network (CNN) has increasingly received attention in image denoising task. Several CNN methods for denoising images have been studied. These methods used different datasets for evaluation. In this paper, we offer an elaborate study on different CNN techniques used in image denoising. Different CNN methods for image denoising were categorized and analyzed. Popular datasets used for evaluating CNN image denoising methods were investigated. Several CNN image denoising papers were selected for review and analysis. Motivations and principles of CNN methods were outlined. Some state-of-the-arts CNN image denoising methods were depicted in graphical forms, while other methods were elaborately explained. We proposed a review of image denoising with CNN. Previous and recent papers on image denoising with CNN were selected. Potential challenges and directions for future research were equally fully explicated.

Convolutional Neural Networks (CNNs) have achieved excellent performance on various artificial intelligence (AI) applications, while a higher demand on energy efficiency is required for future AI. Resistive Random-Access Memory (RRAM)-based computing system provides a promising solution to energy-efficient neural network training. However, it's difficult to support high-precision CNN in RRAM-based hardware systems. Firstly, multi-bit digital-analog interfaces will take up most energy overhead of the whole system. Secondly, it's difficult to write the RRAM to expected resistance states accurately; only low-precision numbers can be represented. To enable CNN training based on RRAM, we propose a low-bitwidth CNN training method, using low-bitwidth convolution outputs (CO), activations (A), weights (W) and gradients (G) to train CNN models based on RRAM. Furthermore, we design a system to implement the training algorithms. We explore the accuracy under different bitwidth combinations of (A, CO, W, G), and propose a practical tradeoff between accuracy and energy overhead. Our experiments demonstrate that the proposed system perform well on low-bitwidth CNN training tasks. For example, training LeNet-5 with 4-bit convolution outputs, 4-bit weights, 4-bit activations and 4-bit gradients on MNIST can still achieve 97.67% accuracy. Moreover, the proposed system can achieve 23.0X higher energy efficiency than GPU when processing the training task of LeNet-5, and 4.4X higher energy efficiency when processing the training task of ResNet-20.

Facial recognition systems, which are a type of biometric systems, use the person’s facial features to identify people, and determine the information in the position between the eyes, nose, cheekbones, jawline, chin and the like and reveal a personal numerical model. The main purpose of this study is to determine the gender and age group of a selected image. By making necessary studies on the image, it is aimed to separate the gender of the person and to extract child or adult information. This study is proposed as research that performs noise reduction using the fuzzy logic fire filter algorithm and classifies the result by gender using a convolutional neural network (CNN), matrix completion and deep learning techniques. CNN algorithm, which is one of the deep facial recognition algorithms that can be used in the recognition of facial images, is the algorithm used in the study. In this study, the data obtained after the cleaning of the noises in the photographs were classified. The fuzzy logic fire filter algorithm is used to remove the noise in the image, and the image is classified according to gender and age range with the help of CNN and TensorFlow algorithms. The system has been trained using certain learning and test data sets and these algorithms. The image entered into the system is converted into a matrix expression and compared with the matrix expressions on the trained system. According to the comparison result, the image is classified as male, female, boy or girl, which matrix is closer to the value group. The data we obtained as a result of our work and the success rates show the applicability of the study. The main purpose of the study is to make gender and age group estimations according to the image with CNN classifications.

Image segmentation of brain low-grade glioma (LGG) magnetic resonance imaging (MRI) contributes tremendously to diagnosis, classification and treatment of the disease. A tangible, accurate, reliable and fast image segmentation technique is demanded in clinical diagnosis and research. The emerging machine learning technique has been demonstrated its unique capability in the field of medical image processing, including medical image segmentation. Support vector machine (SVM) and convolutional neural network (CNN) are two widely used machine learning methods. In this work, image segmentation tools based on SVM and CNN are developed and evaluated for brain LGG MR image segmentation studies. The segmentation performance in terms of accuracy and cost is quantitatively analyzed and compared between the SVM and CNN techniques developed. Computed on the Google CoLab, each of the 109 SVM models represents an individual patient, is trained using a single image of that patient and takes a few seconds to complete. The CNN model is trained on a drastically larger dataset of 19,760 data augmented images and takes approximately 2 hours to obtain the most optimal result. The SVM models achieved an average and median accuracy of 0.937 and 0.976 respectively, precision of 0.456 and 0.535 respectively, recall of 0.878 and 0.906 respectively, and F1 score of 0.546 and 0.662 respectively. Although the CNN model required a significantly longer calculation time, it surpassed the SVM models in performance in LGG MR image segmentation, achieving an accuracy of 0.998, a precision of 0.999, a recall of 0.999 and an F1 score of 0.999. This study shows that SVM with appropriate filtering techniques is capable of obtaining reliable and fast segmentation of brain LGG MR images with sufficient accuracy and limited image data. CNN technique outperforms SVM in the accuracy of segmentation with requirements of significantly enlarged data set, long computation time and high-performance computer.

This study presents a convolutional neural network (CNN)‐based response estimation model for structural health monitoring (SHM) of tall buildings subject to wind loads. In this model, the wind‐induced responses are estimated by CNN trained with previously measured sensor signals; this enables the SHM system to operate stably even when a sensor fault or data loss occurs. In the presented model, top‐level wind‐induced displacement in the time and frequency domains, and wind data in the frequency domain are configured into the input map of the CNN to reflect the resisting capacity of a tall building, the change in the dynamic characteristics of the building due to wind loads, and the relationship between wind load and the building. To evaluate stress, which is used as a safety indicator for structural members in the building, the maximum and minimum strains of columns are set as the output layer of the CNN. The CNN is trained using measured wind and wind response data to predict the column strains during a future wind load. The presented model is validated using data from a wind tunnel test of a building model. The performance of the presented model is verified through strain estimation with data that were not used in the CNN training. To assess the validity of the presented input map configuration, the estimation performance is compared with a CNN that considered only the time domain responses as input. Furthermore, the effects of the variations in the configuration of the CNN on the wind response estimation performance are examined.

This paper investigates two-types of shape representations for individual Maya codical glyphs: traditional bag-of-words built on knowledge-driven local shape descriptors (HOOSC), and Convolutional Neural Networks (CNN) based representations, learned from data. For CNN representations, first, we evaluate the activations of typical CNNs that are pretrained on large-scale image datasets; second, we train a CNN from scratch with all the available individual segments. One of the main challenges while training CNNs is the limited amount of available data (and handling data imbalance issue). Here, we attempt to solve this imbalance issue by introducing class-weights into the loss computation during training. Another possibility is oversampling the minority class samples during batch selection. We show that deep representations outperform the other, but CNN training requires special care for small-scale unbalanced data, that is usually the case in the cultural heritage domain.

Road detection aims at detecting the (drivable) road surface ahead vehicle and plays a crucial role in driver assistance system. To improve the accuracy and robustness of road detection approaches in complex environments, a new road detection method based on CNN (convolutional neural network) and MRF (markov random field) is proposed. The original road image is segmented into super-pixels of uniform size using simple linear iterative clustering (SLIC) algorithm. On this basis, we train the CNN which can automatically learn the features that are most beneficial to classification. Then, the trained CNN is applied to classify road region and non-road region. Finally, based on the relationship between the super-pixels neighborhood, we utilize MRF to optimize the classification results of CNN. Quantitative and qualitative experiments on the publicly datasets demonstrate that the proposed method is robust in complex environments. Furthermore, compared with state-of-the-art algorithms, the approach provides the better performance.