A Modern Approach for Speech to Text and Text to Speech Conversion Application Using Machine Learning Techniques

Growth in Digital signal processing technology gives positives influences in human life. One of the branch of science that gives significant influence is digital speech processing. It can be expand into some applications that make human life easier. Digital speech processing is appropriate to speech recognition. Speech recognition is used to arrange the movement of remote control car. The remote control car will move according to our speech. This final project is closely related with speech recognition. The LPC (Linear Predictive Coding) method will extract the speech signal features and HMM (Hidden Markov Model) to modelling the speech signal are used. It is done by comparing model from extracting feature that is available in HMM modelling. Models will be used in the speech recognition process, if the models have highest level of conformity. The experiment has been done in two conditions, i.e ideal condition in room wih low noise level and unideal condition in room with noise. Result of the experiment from the whole sistem performace at ideal condition is 97,71% for people that have been inputed in the database, and 95,42% for people which have not been inputed in the database. Result of the experiment of database say it regulary is 97,14%. Result of the experiment of database at unideal condition is 54,28% for inputing word at noisy area, result of the experiment of database for inputing word at high frequency from siren voice is 98,57% and result of the experiment of inputing similar word with database is 97,6% Keywords: speech recognition, LPC, HMM

This paper describes about speech recognition implemented on ATmega162 microcontroller. The word (voice command) in a speech signal is applied for controlling movement of a mobile robot. The mobile robot will move according to the voice command in the speech signal. There are five commands, which are Indonesian language, used to control the movement of mobile robot. They are “maju”, “mundur”, “kiri”, “kanan”, and “stop” used to command the mobile robot to move forward, move backward, turn left, turn right and stop respectively.The methods used in this research are Linear Predictive Coding (LPC) and Hidden Markov Model (HMM). LPC is used to extract word data from a speech signal. HMM is used to recognize the word pattern data, which are extracted from a speech signal. Sampling rate of the speech signal is 8 kHz and the speech signal is sampled for 0.5 seconds. Experiments were done in several variations of observation symbol number and number of sample. The experimental results show that the highest recognition rate that can be achieved by the system is 87%. The mobile robot can move in accordance with the voice command that is given to the robot.

Much of the progress in automatic speech recognition is attributable to the use of hidden Markov models (HMMs) to characterize acoustic speech patterns. Despite their success, HMMs make little use of knowledge about the speech signal, and variation that may be explicable in terms of the physical properties of the human speech production system is treated as random. There is, therefore, a need to develop a speech modeling paradigm which reflects human speech processes more closely. Segmental hidden Markov models (HMMs) are extended versions of conventional HMMs in which states are associated with sequences of observation vectors rather than individual vectors. By treating a segment as a homogeneous unit, dependencies between vectors within a segment can be modeled explicitly. This letter describes a segmental HMM in which a segment is modeled as a noisy function of a linear trajectory. The basic theory of the model is presented, together with formulae for model parameter optimization.

A model-based framework of classification error rate estimation is proposed for speech and speaker recognition. It aims at predicting the run-time performance of a hidden Markov model (HMM) based recognition system for a given task vocabulary and grammar without the need of running recognition experiments using a separate set of testing samples. This is highly desirable both in theory and in practice. However, the error rate expression in HMM-based speech recognition systems has no closed form solution due to the complexity of the multi-class comparison process and the need for dynamic time warping to handle various speech patterns. To alleviate the difficulty, we propose a one-dimensional model-based misclassification measure to evaluate the distance between a particular model of interest and a combination of many of its competing models. The error rate for a class characterized by the HMM is then the value of a smoothed zero-one error function given the misclassification measure. The overall error rate of the task vocabulary could then be computed as a function of all the available class error rates. The key here is to evaluate the misclassification measure in terms of the parameters of environmental-matched models without running recognition experiments, where the models are adapted by very limited data that could be just the testing utterance itself. In this paper, we show how the misclassification measure could be approximated by first computing the distance between two mixture Gaussian densities, then between two HMMs with mixture Gaussian state observation densities and finally between two sequences of HMMs. The misclassification measure is then converted into classification error rate. When comparing the error rate obtained in running actual experiments and that of the new framework, the proposed algorithm accurately estimates the classification error rate for many types of speech and speaker recognition problems. Based on the same framework, it is also demonstrated that the error rate of a recognition system in a noisy environment could also be predicted.

A hidden Markov model based classifier is proposed in this paper to perform automatic speech recognition using myoelectric signals from the muscles of vocal articulation. The classifier's resilience to temporal variance is compared to a linear discriminant analysis classifier that was used in a pervious study. Speech recognition was performed, using five channels of myoelectric signals, on isolated words from a 10-word vocabulary. Temporal variance was induced by temporally misaligning data from the test set, with respect to the training set. When compared to the LDA classifier, the hidden Markov model classifier demonstrated a markedly lower variation in classification error due to the temporal misalignment. Characteristics of the hidden Markov model MES classifier suggest that it would effectively complement a conventional acoustic speech recognizer, in a multi-modal speech recognition system.

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

Massive machine-type communication (mMTC) is expected to play a pivotal role in emerging 5G networks. Considering the dense deployment of small cells and the existence of heterogeneous cells, an MTC device can discover multiple cells for association. Under traditional cell association mechanisms, MTC devices are typically associated with an eNodeB with highest signal strength. However, the selected eNodeB may not be able to handle mMTC requests due to network congestion and overload. Therefore, reliable cell association would provide a smarter solution to facilitate mMTC connections. To enable such a solution, a hidden Markov model (HMM) based machine learning (ML) technique is proposed in this paper to perform optimal cell association. As such, we consider MTC devices with network-assisted decision-making capabilities for selecting the most appropriate eNodeB for data transmission. The proposed HMM based ML technique focuses mainly on the reliability and availability of network resources. Correspondingly, two schemes are developed based on the classical reliability function and the next probable state of the HMM. Based on simulations under various configurations, we demonstrate the advantage of the proposed schemes over a random cell selection scheme.

A low-power and real-time hidden Markov model (HMM) accelerator is proposed for gesture user interfaces on wearable smart devices. HMM algorithm is widely used for sequence recognitions such as speech recognition and gesture recognition due to its best-in-class recognition accuracy. However, the HMM algorithm incorporates high computational complexity and requires massive memory bandwidth for sequence matches. There have been studies on hardware acceleration of the HMM algorithm to resolve these issues, but they were focused on the speech recognition and did not incorporate the motion orientation capability required for the gesture recognition case. In this paper, we propose an HMM accelerator incorporating the motion orientation block for gesture recognitions on wearable devices. Binary search is exploited in the motion orientation to avoid the division and arctangent associated with the orientation and reduce its arithmetic complexity. In addition, gesture models are clustered in the gesture database to save the memory bandwidth by reducing memory transactions. Moreover, logarithmic arithmetic is used in Viterbi decoding in the HMM for more reduction in its complexity. Thanks to these schemes, this work achieves 25.6% power reduction compared with a plain hardware implementation of the gesture recognizing HMM.

Great strides have been made in the development of automatic speech recognition (ASR) technology over the past thirty years. Most of this effort has been centered around the extension and improvement of Hidden Markov Model (HMM) approaches to ASR. Current commercially-available and industry systems based on HMMs can perform well for certain situational tasks that restrict variability such as phone dialing or limited voice commands. However, the holy grail of ASR systems is performance comparable to humans—in other words, the ability to automatically transcribe unrestricted conversational speech spoken by an infinite number of speakers under varying acoustic environments. This goal is far from being reached. Key to the success of ASR is effective modeling of variability in the speech signal. This tutorial will review the basics of ASR and the various ways in which our current knowledge of speech production, speech perception and prosody can be exploited to improve robustness at every level of the system.

The Relative Distance Vector Neural Network (RDVNN) Model: A Hybrid Approach to Speech Recognition

The movements of talkers' face, nose, mouth and throat are known to convey visual cues and represent several different kinds of informationl, and that can improve speech recognition rate, especially for persons with speech-impairments. We proposed a new speech recognition method using these visual features and hidden Markov model (HMM). Based on global optimisation, a new genetic algorithm (GA) for training HMM was proposed. Six chinese vowels were taken as the experimental data, ten handicapped speakers were taken as the testee. Recognition experiments show that the method is effective and high speed and accuracy for speech recognition. At present, the average recognition rate is 91.47% using improved HMM and 88.96% using the classic training HMM algorithm, So the features has very good robustness and the improved HMM is very good.

In recent years, with the rapid development of Deep Learning (DL) and the widespread uses of Deep Neural Networks (DNN), speech recognition technology has attracted great attention. This technology enables machines to convert human speech into text and plays an important role in some areas such as virtual assistants, conference transcription and smart home devices. Traditional models such as Hidden Markov Models (HMM) and Gaussian Mixture Models (GMM) used to play a key role in modeling and recognizing speech signals. Even though these methods still have challenges in dealing with high‐dimensional feature vectors as they struggle to efficiently learn large amounts of model parameters while maintaining good generalization performance. The emergence of DNNs has revolutionized the field of speech recognition, making it possible to automatically extract features from raw speech signals. Thereby, modern speech recognition systems can exhibit higher adaptability and generalization capabilities, achieving higher accuracy and stability under various conditions. This review explores the application of DL techniques to speech recognition, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformer‐based architectures, etc. Advances in these techniques have dramatically improved recognition accuracy, noise immunity, and adaptability to multiple accents, thereby improving the total efficiency of automatic speech systems. By leveraging advanced feature extraction and modeling capabilities, deep learning‐based systems support real‐ time processing and minimize the dependency on traditional manually designed features. However, there are still limitations in terms of interpretability, computational efficiency, and fairness to different accents and languages, which impede the widespread use of these techniques. Future research should focus on solving these issues in order to achieve fairer, efficient, and scalable deep learning‐driven speech recognition schemes.

In fact, millions of people in the world speak many languages. In order to communicate with each other, it is necessary to know the language we use to perform this operation. The Arabic language has many different dialects and it must be recognized before using the automatic speech recognition (ASR). On the other hand, it is observed in all Arab countries that the standard Arabic language is widely written and used in an official speech, newspapers, public administration, and schools but it is not used in the daily conversations instead the dialect is widely spoken in daily life and rarely written. In this paper, we examine the difficult task of properly identifying various Arabic dialects and propose a system developed to identify a set of four regional and modern standard Arabic speeches, based on speech recognition using Hidden Markov Models (HMMs) algorithms. HMMs have become a very popular way to build a speech recognition system. It is set as hidden states and possibilities of transition from one state to another. Due to the similarities and differences between the Arabic dialects, speeches collected from the ADI5 datasets were retrieved from the MGB-3 challenge source. We proposed an Arabic Dialect Identification System called "Building a System for Arabic Dialects Identification based on Speech Recognition using Hidden Markov Models (HMMs)" that takes Input as speech utterances and produces output as dialect being spoken. During the training phase, speech utterances from one or more dialects were analyzed to capture the important properties of audio signals in terms of time and frequency. During the testing phase, previously unseen test utterances were utilized to the system, and the system outputs the dialect associated with the model of dialect that most closely matches the test utterance. The proposed model of the system shows promising results of the model for each dialects match.

In this paper we investigated Artificial Neural Networks (ANN) based Automatic Speech Recognition (ASR) by using limited Arabic vocabulary corpora. These limited Arabic vocabulary subsets are digits and vowels carried by specific carrier words. In addition to this, Hidden Markov Model (HMM) based ASR systems are designed and compared to two ANN based systems, namely Multilayer Perceptron (MLP) and recurrent architectures, by using the same corpora. All systems are isolated word speech recognizers. The ANN based recognition system achieved 99.5% correct digit recognition. On the other hand, the HMM based recognition system achieved 98.1% correct digit recognition. With vowels carrier words, the MLP and recurrent ANN based recognition systems achieved 92.13% and 98.06, respectively, correct vowel recognition; but the HMM based recognition system achieved 91.6% correct vowel recognition.

Arabic language can be used by native and non- native speakers; due to Arabic is the language of the holy book of Muslims. In this paper, Arabic phoneme recognition system is proposed based on Malay speakers. This system consists of three main stages. The first stage is noise reduction and it aims to enhance the phoneme signals by excluding the unvoiced signals and keep only the voiced signal. Wiener filter is adapted to accomplish this task. The second stage is based on Mel-Frequency Cepstral Coefficients method to extract a vector of features to represent each phoneme signal. Eventually, pattern recognition neural network is designed as recognizer. The proposed system produces sufficient outcomes with 20 hidden neurons. Keywords—Arabic; Malay; pattern recognition; Wiener; Mel- Frequency Cepstral Coefficients I. INTRODUCTION It is well known the wide range of Artificial Neural Networks (ANN) application in speech recognition, financial, telecommunications, electronics and medical. As for speech recognition system for Arabic language, (1) & (2) have addressed the implementation of recognition system based on Arabic native speakers. In general, neural networks (NNs) for phonemes recognition are divided into several categories for instance Very Large Scale Integration (VLSI) NN that can be divided into digital and analogue with digital NNs being more compatible with feed-forward neural networks while analogue NNs are found to be more successful with recurrent NNs (3). From Automatic Speech System (ASR) point of view, NNs for recognition purpose can be divided into two main category namely conventional neural networks such as Multilayer Perceptron (MLP) and Radial Basis Function (RBF) and secondly Recurrent Neural Networks (RNN). The first category of NNs was implemented as pattern classifiers and proven capable for recognition purpose but not at par as compared to Hidden Markov Model (HMM) performance (2). At present, HMM has been proven to be the most successful approach in ASR research area until recently the possibility to hybrid both HMM and ANN (4) & (5). In general, speech recognition systems consist of three stages specifically noise reduction, feature extraction and classification. The aim of noise reduction is to reduce the level of noise in the speech signal and make it more compatible for the ASR. As for feature extraction, this process attempted to extract the most relevant features related to the phoneme signal. The extracted features acted as input features of the NN during training and testing. Finally, the purpose of classification process is to arrange the most similar and related features in categories based on the training phase. Basically, networks training database are being observed and updated during training session of the networks, involving the weights and biases arguments. Networks with lower errors can be considered as better networks (6). Hence, this paper deems to explore further the ability of NN for Arabic phonemes recognition spoken by Malay individuals. This paper is arranged as following; related work is discussed in section II followed by section III that discussed method to be implemented based on Wiener filter, Mel-Frequency Cepstral Coefficients and pattern recognition neural network. Section IV elaborated experimental analysis and results and followed by Conclusion section.