Large vocabulary speech recognition using subword units

In this paper we present an experimental study exploiting structural Bayesian adaptation for handling potential mismatches between training and test conditions for real-world applications to be realized in our multilingual very large vocabulary speech recognition (VLVSR) system project sponsored by MOTIE (The Ministry of Trade, Industry and Energy), Republic of Korea. The goal of the project is to construct a national-wide VLVSR cloud service platform for mobile applications. Besides system architecture design issues, at such a large scale, performance robustness problems, caused by mismatches in speakers, tasks, environments, and domains, etc., need to be taken into account very carefully as well. We decide to adopt adaptation, especially the structural MAP, techniques to reduce system accuracy degradation caused by these mismatches. Being part of an ongoing project, we describe how structural MAP approaches can be used for adaptation of both acoustic and language models for our VLVSR systems, and provide convincing experimental results to demonstrate how adaptation can be utilized to bridge the performance gap between the current state-of-the-art and deployable VLVSR systems.

This paper presents the results of a study on modeling the highly inflective Slovenian language. We focus on creating a language model for a large vocabulary speech recognition system. A new data-driven method is proposed for the induction of inflectional morphology into language modeling. The research focus is on data sparsity, which results from the complex morphology of the language. The idea of using subword units is examined. An attempt is made to figure out the segmentation of words into two subword units: stems and endings. No prior knowledge of the language is used. The subword units should fit into the frameworks of the probabilistic language models. A morphologically correct decomposition of words is not being sought, but searching for a decomposition which yields the minimum entropy of the training corpus. This entropy is approximated by using N-gram models. Despite some seemingly over-simplified assumption, the subword models improve the applicability of the language models for a sparse training corpus. The experiments were performed using the VEČER newswire text corpus as a training corpus. The test set was taken from the SNABI speech database, because the final models were evaluated in speech recognition experiments on SNABI speech database. Two different subword-based models are proposed and examined experimentally. The experiments demonstrate that subword-based models, which considerably reduce OOV rate, improve speech recognition WER when compared with standard word-based models, even though they increase test set perplexity. Subword-based models with improved perplexity, but which reduce the OOV rate much less than the previous ones, do not improve speech recognition results.

This paper describes, how to perform speaker adaptation for a hybrid large vocabulary speech recognition system. The hybrid system is based on a Maximum Mutual Information Neural Network (MMINN), which is used as a Vector Quantizer (VQ) for a discrete HMM speech recognizer. The combination of MMINNs and HMMs has shown good performance on several large vocabulary speech recognition tasks like RM and WSJ. This paper now presents two approaches to perform speaker adaptation with this hybrid system. The first approach is a transformation of the feature space, which is performed by a neural network with maximum likelihood (ML) as objective function for the complete system, which means, that the parameters of the NN are estimated in order to match the HMM-parameters of the pre-trained speaker independent system. The second approach is to adapt the HMM parameters depending on the amount of training data available per HMM, using a regularization approach. Both approaches can be applied jointly, which further improves the recognition accuracy.

Implicit modelling of pronunciation variation in automatic speech recognition

In this paper we present two new techniques that have been used to build a large-vocabulary continuous Hindi speech recognition system. We present a technique for fast bootstrapping of initial phon...

The word graphs produced by a large vocabulary speech recognition system usually contain a path labelled with the correct utterance, but this is not always the highest scoring path. Boosting increases the probability of words which occur often in the word graph, which are in some sense robust. Adding syntactic information allows rescoring of arc probabilities with the possibility that more grammarical word sequences will also be the correct ones. A theory is developed which allows general probabilistic syntactic models to be used to rescore word lattices. Experiments conducted on the Wall Street Journal (WSJ) corpus with a version of the AT&T 1995 FST LVSR system with part of speech (POS) trigram sequences show that using only POS leads to a loss in performance. Boosting alone provides an improvement in performance which is not statistically significant. Cascading the two methods, boosting first and then using syntactic information improves performance 4.5 % relative on a large portion of the 1995 DARPA test set.

Acoustic modeling for large vocabulary speech recognition

In addressing the problem of achieving high-accuracy real-time speech recognition systems, we focus on recognizing speech from ARPA's 20,000-word Wall Street Journal (WSJ) task, using current UNIX workstations. We have found that our standard approach---using a narrow beam width in a Viterbi search for simple discrete-density hidden Markov models (HMMs)---works in real time with only very low accuracy. Our most accurate algorithms recognize speech many times slower than real time. Our (yet unattained) goal is to recognize speech in real time at or near full accuracy.We describe the speed/accuracy trade-offs associated with several techniques used in a one-pass speech recognition framework:• Trade-offs associated with reducing the acoustic modeling resolution of the HMMs (e.g., output-distribution type, number of parameters, cross-word modeling)• Trade-offs associated with using lexicon trees, and techniques for implementing full and partial bigram grammars with those trees• Computation of Gaussian probabilities are the most time-consuming aspect of our highest accuracy system, and techniques allowing us to reduce the number of Gaussian probabilities computed with little or no impact on speech recognition accuracy.Our results show that tree-based modeling techniques used with appropriate acoustic modeling approaches achieve real-time performance on current UNIX workstations at about a 30% error rate for the WSJ task. The results also show that we can dramatically reduce the computational complexity of our more accurate but slower modeling alternatives so that they are near the speed necessary for real-time performance in a multipass search. Our near-future goal is to combine these two technologies so that real-time, high-accuracy large-vocabulary speech recognition can be achieved.

Virtual assistants are improving and providing consumers with greater advantages. The comprehension and fulfilment of requests by virtual assistants will increase as voice recognition and natural language processing continue to grow. Virtual assistants are projected to be employed in more commercial activities as speech recognition technology advances. The main goal of developing personal assistant software (virtual assistant) is to use web-based semantic data sources, user-generated content, and knowledge from knowledge libraries. Basically, main objective of making this Voice-Based Virtual Assistant is to make life easier and having a personal assistant to everyone which can perform many tasks. As the end user interacts with a virtual assistant, the AI programming learns from the data provided and improves its ability to forecast the end user's needs. Virtual assistants are often used to do things like add tasks to a calendar, provide information that would normally be found in a website, and operate and monitor Smart Home devices like lighting and cameras and thermostats. Massive volumes of data are required to fuel virtual assistant technologies, which feed Artificial Intelligence (AI) platforms such as machine learning, natural language processing, and speech recognition. Speech recognition has a lengthy history and has seen several key advancements. On smartphones and wearable devices, speech recognition for dictation, search, and voice commands has become a standard feature. Design of a small, large vocabulary speech recognition system that can run quickly, accurately, and with minimum latency on mobile devices.

This paper describes an attempt to design a knowledge-based large vocabulary speech recognition system. Our motivation is to replace features based on the short-term spectra, such as Mel-frequency cepstral coefficients (MFCC), by features that explicitly represent some of the distinctive features of the speech signal. However, rather than attempting to compute acoustic correlates of these distinctive features, we have engineered an approach where neural networks are trained to map short-term spectral features to the posterior probability of some distinctive features. These probabilities are then used as features in a large vocabulary tied-state HMM-based recognizer. Experimental results on the Wall Street Journal Task show that such a system, while not outperforming a MFCC-based system, generates very different error patterns. After combining the results of a base-line MFCC system with the results of several systems based on the proposed approach, we were able to obtain reductions in word error rates of 19% and 10 % on the 5K and 20K tasks respectively over our best MFCC-based systems.

Hidden Markov models (HMM) with Gaussian mixture observation densities are the dominant approach in speech recognition. These systems typically use a representational model for acoustic modeling which can often be prone to overfitting and does not translate to improved discrimination. We propose a new paradigm centered on principles of structural risk minimization using a discriminative framework for speech recognition based on support vector machines (SVMs). SVMs have the ability to simultaneously optimize the representational and discriminative ability of the acoustic classifiers. We have developed the first SVM-based large vocabulary speech recognition system that improves performance over traditional HMM-based systems. This hybrid system achieves a state-of-the-art word error rate of 10.6% on a continuous alphadigit task—a 10% improvement relative to an HMM system. On SWITCHBOARD, a large vocabulary task, the system improves performance over a traditional HMM system from 41.6% word error rate to 40.6%. This dissertation discusses several practical issues that arise when SVMs are incorporated into the hybrid system.

Integration of language constraints into a large vocabulary speech recognition system often leads to prohibitive complexity. We propose to factor the constraints into two components. The first is characterized by a covering grammar which is small and easily integrated into existing speech recognizers. The recognized string is then decoded by means of an efficient language post-processor in which the full set of constraints is imposed to correct possible errors introduced by the speech recognizer.

In previous work, a class of transforms were proposed which achieve a remapping of the frequency axis much like conventional vocal tract length normalization. These mappings, known collectively as all-pass transforms (APT), were shown to produce substantial improvements in the performance of a large vocabulary speech recognition system when used to normalize incoming speech prior to recognition. In this application, the most advantageous characteristic of the APT was its cepstral-domain linearity; this linearity makes speaker normalization simple to implement, and provides for the robust estimation of the parameters characterizing individual speakers. In the current work, we exploit the APT to develop a speaker adaptation scheme in which the cepstral means of a speech recognition model are transformed to better match the speech of a given speaker. In a set of speech recognition experiments conducted on the Switchboard corpus, we report reductions in word error rate of 3.7% absolute.

A language model-based design of reduced phoneme set for acoustic model is proposed. In the case where the amount of training data is too small to train each phoneme model, the reduction of the phoneme set can lead to a reduced discriminative model of phonemes, which can increase homophones that yield degradation of speech recognition. The proposed approach enables us to reduce phonemes preventing the degradation, regarding pronunciation/word sequence confusion rate calculated from n-grams in a language model. In an experiment, the phoneme set designed with proposed approach was applied to Japanese large vocabulary speech recognition system. The word error rate with full 39 phonemes set was 9.5%, while the error rate with the 10 phonemes set designed with the proposed approach was 11.1%. The degradation was able to be prevented within 2%.

The authors present a context-dependent, phoneme and function work based, hidden control neutral network (HCNN-CDF) architecture for continuous speech recognition. The system can be seen as a large vocabulary extension of the word-based HCNN system proposed by E. Levin (1990). Two main extensions towards a large vocabulary speech recognition system are presented and discussed, i.e., the context-dependent HCNN phoneme model and the context-dependent HCNN function word model. When compared to the linked predictive neural network (LPNN) system of, J. Tebelskis (1990) significant savings in resource requirements and computational load for the HCNN-CDF implementation can be achieved. In speaker-dependent recognition experiments with perplexity 111, the current versions of the LPNN and HCNN-CDF systems achieve 60% and 75% word recognition accuracies, respectively.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>