HMMs with Explicit State Duration Applied to Handwritten Arabic Word Recognition

Methods for text segmentation from scene images

A computationally efficient algorithm is proposed for modeling and coding the time-varying spectra of musical sounds. The aim is to encode individual data sets and not the statistical properties of the sounds. A given sequence of acoustic feature vectors is modeled by finding such a set of “states” (anchor points in the feature space) that the input data can be efficiently represented by interpolating between them. The achieved modeling accuracy for a database of musical sounds was approximately two times better than that of a conventional “vector quantization” model where the input data was k-means clustered and the input data vectors were then replaced by their corresponding cluster centroids. The computational complexity of the proposed algorithm as a function of the input sequence length T is O(TlogT).

We have conducted speaker-independent isolated digit recognition experiments using vector quantized cochleagrams. Without the use of time order information, we were able to achieve a recognition rate of 97.24%. With a modified Viterbi algorithm we achieved a rate of 98.38%. Since we achieved a 98.05% recognition rate with a scheme that did static pattern matching on the first and second time-halves of our utterances, we must call into question the effectiveness with which the Viterbi algorithm uses time order information. Our results also lead us to conclude that future progress may depend on our ability to construct more sophisticated vector quantizers.

Color-based indexing is an important tool in image database retrieval. Compared with other features of the image, color features are less sensitive to noise and background complication. Based on the human visual system's perception of color information, this paper presents a dependent scalar quantization approach to extract the characteristic colors of an image as color features. The characteristic colors are suitably arranged in order to obtain a sequence of feature vectors. Using this sequence of feature vectors, a dynamic matching method is then employed to match the query image with database images for a nonstationary identification environment. The empirical results show that the characteristic colors are reliable color features for image database retrieval. In addition, the proposed matching method has acceptable accuracy of image retrieval compared with existing methods.

This paper investigates the development of language identification based on Mel-Frequency Cepstral Coefficients (MFCC) and Vector Quantization (VQ) algorithm. In this study, a total of ten speakers were chosen randomly with different languages from online language database. A total of six males and four females were selected as subjects for this research and each of them spoke different languages, including Arabic, Chinese, English, Korean and Malay. The MFCC will be extracted to derive the related feature vector. Vector Quantization (VQ) algorithm is then used as classifier. The recognition rate is then calculated for each language. Several experiments were conducted to find the optimum parameters, in which we found that sampling frequency of 16000 Hz and codebook size of 75 provided good results. On average, the recognition rate for all five languages evaluated was 78%. The experimental results show that our proposed system provides a good recognition rate.

In this work, we investigate the combination of PGM (Propabilistic Graphical Models) classifiers, either independent or coupled, for the recognition of Arabic handwritten words. The independent classifiers are vertical and horizontal HMMs (Hidden Markov Models) whose observable outputs are features extracted from the image columns and the image rows respectively. The coupled classifiers associate the vertical and horizontal observation streams into a single DBN (Dynamic Bayesian Network). A novel method to extract word baseline and a simple and easily extractable features to construct feature vectors for words in the vocabulary are proposed. Some of these features are statistical, based on pixel distributions and local pixel configurations. Others are structural, based on the presence of ascenders, descenders, loops and diacritic points. Experiments on handwritten Arabic words from IFN/ENIT strongly support the feasibility of the proposed approach. The recognition rates achieve 90.42% with vertical and horizontal HMM, 85.03% and 85.21% with respectively a first and a second DBN which outperform results of some works based on PGMs.

Offline handwriting recognition is the ability to decode an intelligible handwritten input from paper documents into digitized format readable by machines. This field remains an on-going research problem especially for Arabic Script due to its cursive appearance, the variety of writers and the diversity of styles. In this paper we focus on the Intelligent Words Recognition system based on MDLSTM, on which a dropout technique is applied during training stage. This technique prevents our system against overfitting and improves the recognition rate. To evaluate our system we use IFN/ENIT database.

<span lang="EN-US">Arabic handwritten text recognition has long been a difficult subject, owing to the similarity of its characters and the wide range of writing styles. However, due to the intricacy of Arabic handwriting morphology, solving the challenge of cursive handwriting recognition remains difficult. In this paper, we propose a new efficient based image processing approach that combines three image descriptors for the feature extraction phase. To prepare the training and testing datasets, we applied a series of preprocessing techniques to 100 classes selected from the handwritten Arabic database of the </span><em><span lang="EN-US">Institut Für Nachrichtentechnik/Ecole Nationale d'Ingénieurs de Tunis</span></em><span lang="EN-US"> (IFN/ENIT). Then, we trained the k-nearest neighbor’s algorithm (k-NN) algorithm to generate the best model for each feature extraction descriptor. The best k-NN model, according to common performance evaluation metrics, is used to classify Arabic handwritten images according to their classes. Based on the performance evaluation results of the three k-NN generated models, the majority-voting algorithm is used to combine the prediction results. A high recognition rate of up to 99.88% is achieved, far exceeding the state-of-the-art results using the IFN/ENIT dataset. The obtained results highlight the reliability of the proposed system for the recognition of handwritten Arabic words.</span>

Automatic lipreading is automatic speech recognition that uses only visual information. The relevant data in a video signal is isolated and features are extracted from it. From a sequence of feature vectors, where every vector represents one video image, a sequence of higher level semantic elements is formed. These semantic elements are “visemes” the visual equivalent of “phonemes” The developed prototype uses a Time Delayed Neural Network to classify the visemes.

Following recent advancements in deep learning and artificial intelligence, spoken language identification applications are playing an increasingly significant role in our day-to-day lives, especially in the domain of multi-lingual speech recognition. In this article, we propose a spoken language identification system that depends on the sequence of feature vectors. The proposed system uses a hybrid Convolutional Recurrent Neural Network (CRNN), which combines a Convolutional Neural Network (CNN) with a Recurrent Neural Network (RNN) network, for spoken language identification on seven languages, including Arabic, chosen from subsets of the Mozilla Common Voice (MCV) corpus. The proposed system exploits the advantages of both CNN and RNN architectures to construct the CRNN architecture. At the feature extraction stage, it compares the Gammatone Cepstral Coefficient (GTCC) feature and Mel Frequency Cepstral Coefficient (MFCC) feature, as well as a combination of both. Finally, the speech signals were represented as frames and used as the input for the CRNN architecture. After conducting experiments, the results of the proposed system indicate higher performance with combined GTCC and MFCC features compared to GTCC or MFCC features used individually. The average accuracy of the proposed system was 92.81% in the best experiment for spoken language identification. Furthermore, the system can learn language-specific patterns in various filter size representations of speech files.

We present a Histogram of Oriented Gradient (HoG) based two-directional Dynamic Time Warping (DTW) matching method for handwritten word spotting. Firstly, we extract HoG descriptors from each cell in the normalized images. Then we connect the HoG descriptors in the same column and get a sequence of feature vectors. We do the same operation for the HoG descriptors in the same row. We then apply the two-directional DTW method to calculate the distance between the feature vectors sequences extracted from the query word and the candidate one. The experimental results show that the two-directional DTW is more robust to word deformation than the traditional DTW. And the local features such as HoG, LBP and SIFT combined with the two-directional DTW method outperform the method using the local feature descriptors directly. The HoG based two-directional DTW get the highest mean average precision on both the George Washington dataset and the CASIA-HWDB 2.1 dataset.

We propose an approach to the task of automatic pose initialization of swimmers in videos.Thus, our goal is to detect a swimmer inside a target video and assign an estimated position to her/his body parts.We first apply a non-skin-color filter to reduce the search space inside each target frame.We then match previously devised template sequences of Gaussian feature descriptors against sequences of feature vectors which are computed within the remaining image regions.Finally, relative average joint positions from annotated images featuring the key pose are assigned to the detection result and three-dimensional joint positions are estimated.We present detection results for test videos of three different swim strokes and examine the performance of four types of feature descriptors.

Signature, a form of handwritten depiction, has been and is still widely used as a proof of the writer's identity/intent in human society. Online signatures represents the dynamic process of handwriting as a sequence of feature vectors along time. Dynamic time warping (DTW) has been popularly adopted to compare sequence data. A basic problem in using DTW for signature verification is how to estimate the difference between the feature vectors. Most previous researches made use of Euclidean distance (ED) for this problem. However, ED treats each feature equally and cannot take account of the correlations between features. To overcome this problem, this paper proposed Mahalanobis distance (MD) for signature verification. One key question is how to estimate covariance matrix in MD calculation. We formulate this problem in a learning framework and introduce two criterion for estimating the matrix. The first criteria aims at minimizing the signature difference for the same writer, while the second criteria try to maximize the signature difference between different writers while minimize the within-writer signature difference. We carried out experiments on the MCYT biometric database. The experimental results exhibit that the proposed MD based method achieved better results than ED based method.

Recognition-directed recovering of temporal information from handwriting images

In speaker recognition, most of the computation originates from the likelihood computations between feature vectors of the unknown speaker and the models in the database. In this paper, we concentrate on optimizing Mel Frequency Cepstral Coefficient (MFCC) for feature extraction and Vector Quantization (VQ) for feature modeling. We reduce the number of feature vectors by pre-quantizing the test sequence prior to matching, and number of speakers by ruling out unlikely speakers during recognition process. The two important parameters, Recognition rate and minimized Average Distance between the samples, depends on the codebook size and the number of cepstral coefficients. We find, that this approach yields significant performance when the changes are made in the number of mfcc's and the codebook size. Recognition rate is found to reach upto 89% and the distortion reduced upto 69%.