IPA: improved phone modelling with recurrent neural networks

In addressing the problem of achieving high-accuracy real-time speech recognition systems, we focus on recognizing speech from ARPA's 20,000-word Wall Street Journal (WSJ) task, using current UNIX workstations. We have found that our standard approach---using a narrow beam width in a Viterbi search for simple discrete-density hidden Markov models (HMMs)---works in real time with only very low accuracy. Our most accurate algorithms recognize speech many times slower than real time. Our (yet unattained) goal is to recognize speech in real time at or near full accuracy.We describe the speed/accuracy trade-offs associated with several techniques used in a one-pass speech recognition framework:• Trade-offs associated with reducing the acoustic modeling resolution of the HMMs (e.g., output-distribution type, number of parameters, cross-word modeling)• Trade-offs associated with using lexicon trees, and techniques for implementing full and partial bigram grammars with those trees• Computation of Gaussian probabilities are the most time-consuming aspect of our highest accuracy system, and techniques allowing us to reduce the number of Gaussian probabilities computed with little or no impact on speech recognition accuracy.Our results show that tree-based modeling techniques used with appropriate acoustic modeling approaches achieve real-time performance on current UNIX workstations at about a 30% error rate for the WSJ task. The results also show that we can dramatically reduce the computational complexity of our more accurate but slower modeling alternatives so that they are near the speed necessary for real-time performance in a multipass search. Our near-future goal is to combine these two technologies so that real-time, high-accuracy large-vocabulary speech recognition can be achieved.

This paper focuses on the problem of environmental noises in human-human communication and in automatic speech recognition. To deal with this problem, the use of alternative acoustic sensors -which are attached to the talker and receive the uttered speech through skin or bones- is investigated. In the current study, throat microphones and ear bone microphones are integrated with standard microphones using several fusion methods. The results obtained show that the recognition rates in noisy environments are drastically increased when these sensors are integrated with standard microphones. Moreover, the system does not show any recognition degradations in clean environments. In fact, recognition rates also increase slightly in clean environments. Using late fusion to integrate a throat microphone, an ear bone microphone, and a standard microphone, we achieved a 44% relative improvement in recognition rate in a noisy environment and a 24% relative improvement in recognition rate in a clean environment.

Connectionist gender adaptation in a hybrid neural network / hidden Markov model speech recognition system

A new recurrent neural network based language model (RNN LM) with applications to speech recognition is presented. Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model. Speech recognition experiments show around 18% reduction of word error rate on the Wall Street Journal task when comparing models trained on the same amount of data, and around 5% on the much harder NIST RT05 task, even when the backoff model is trained on much more data than the RNN LM. We provide ample empirical evidence to suggest that connectionist language models are superior to standard n-gram techniques, except their high computational (training) complexity. Index Terms: language modeling, recurrent neural networks, speech recognition

Recurrent neural networks (RNN) are at the core of modern automatic speech recognition (ASR) systems. In particular, long short-term memory (LSTM) recurrent neural networks have achieved state-of-the-art results in many speech recognition tasks, due to their efficient representation of long and short term dependencies in sequences of inter-dependent features. Nonetheless, internal dependencies within the element composing multidimensional features are weakly considered by traditional real-valued representations. We propose a novel quaternion long short-term memory (QL-STM) recurrent neural network that takes into account both the external relations between the features composing a sequence, and these internal latent structural dependencies with the quaternion algebra. QLSTMs are compared to LSTMs during a memory copy-task and a realistic application of speech recognition on the Wall Street Journal (WSJ) dataset. QLSTM reaches better performances during the two experiments with up to 2.8 times less learning parameters, leading to a more expressive representation of the information.

The authors describe the latest development by the speech research group at CRIM (Centre de Recherche Informatique de Montreal) in speaker-independent connected digit recognition, using hidden Markov Models (HMMs) trained with maximum mutual information estimation, in conjunction with connectionist models. The experiments described were all done on the complete adult portion of the 10 kHz speaker-independent TI/NIST connected digit database. The baseline system, using discrete HMMs and maximum likelihood estimation, has a 98.6% word recognition rate and a 96.1% string recognition rate. The authors describe techniques that made it possible to improve greatly the baseline system recognition rate. The 99.3% recognition rate and 98.0% string recognition rate were obtained with a single model per unit using discrete HMMs and recurrent neural networks. Using semi-continuous HMMs with two models per digit (one for male and one for female speakers), a 99.5% word recognition rate and a 98.4% string recognition rate were achieved.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

In this study, we try to find ways to recognize face recognition more stably and to improve the effectiveness and reliability of face recognition. In order to improve the face recognition rate, a lot of data must be used, but that does not necessarily mean that the recognition rate is improved. Another criterion for improving the recognition rate can be seen that the top/bottom of the recognition rate is determined depending on how accurately or precisely the degree of classification of the data to be used is made. There are various methods for classification analysis, but in this study, classification analysis is performed using a support vector machine (SVM). In this study, feature information is extracted using a normalized image with rotation information, and then projected onto the eigenspace to investigate the relationship between the feature values through the classification analysis of SVM. Verification through classification analysis can improve the effectiveness and reliability of various recognition fields such as object recognition as well as face recognition, and will be of great help in improving recognition rates.

Recognition of car horns based on principal component analysis of MEL frequency Cepstral coefficients and support vector machine

In this research, we selected the speech recognition to implement the electric wheelchair system as a method to control it by only using the speech and used DTW (Dynamic Time Warping), which is speaker-dependent and has a relatively high recognition rate among the speech recognitions. However, it has to have small memory and fast process speed performance under consideration of real-time. Thus, we introduced VQ (Vector Quantization) which is widely used as a compression algorithm of speaker-independent recognition, to secure fast recognition and small memory. However, we found that the recognition rate decreased after using VQ. To improve the recognition rate, we applied ART2 (Adaptive Reason Theory 2) algorithm as a post-process algorithm to obtain about 5% recognition rate improvement. To utilize ART2, we have to apply an error range. In case that the subtraction of the first distance from the second distance for each distance obtained to apply DTW is 20 or more, the error range is applied. Likewise, ART2 was applied and we could obtain fast process and high recognition rate. Moreover, since this system is a moving object, the system should be implemented as an embedded one. Thus, we selected TMS320C32 chip, which can process significantly many calculations relatively fast, to implement the embedded system. Considering that the memory is speech, we used 128kbyte-RAM and 64kbyte ROM to save large amount of data. In case of speech input, we used 16-bit stereo audio codec, securing relatively accurate data through high resolution capacity.

Methods for text segmentation from scene images

A hidden Markov model (HMM) is a natural and highly robust statistical method for automatic speech recognition. It has been tested and proved effective in a wide range of applications. The HMM model parameters are used to describe the utterance of the speech segment presented by the HMM. Many successful heuristic algorithms are developed to optimize the model parameters to best describe the training observation sequences. However, all these methods are exploring for only one local maximum in practice. No single method can be recovered from the local maximum and to obtain the global maximum or other more optimized local maxima. In this paper, a stochastic search method called the genetic algorithm (GA) is presented for HMM training. GA mimics natural evolution and performs searching within the defined searching space. Experimental results showed that using GA for HMM training (GA-HMM training) can obtain better solutions than using heuristic algorithms. One of the major drawbacks is that GAs require a lot of computation power for global searching before it can converge. Therefore, in order to outperform heuristic algorithms, a parallel version of GA called the parallel genetic algorithm (PGA) is introduced. Experimental results showed that using PGA in speech recognition systems provides 18% improvement in recognition rate with the same amount of computational time.

Copyright © 2015 ISCA. This paper describes the recently developed artificial neural network (ANN) modules in HTK hidden Markov model toolkit, which enables ANN models with very general feed-forward architectures to be used for either acoustic modelling or feature extraction. The HTK ANN extension includes many recent ANN-based speech processing techniques, such as sequence training, model stacking, speaker adaptation, and parameterised activation functions. The implementation allows efficient training by supporting GPUs and various types of data cache. The ANN modules are fully integrated into the rest of the HTK toolkit, which allows existing GMM-HMM methods to be easily used in the ANN-HMM framework. Speech recognition results on a 300 hours DARPA BOLT conversational Mandarin task show that HTK can produce tandem and hybrid systems with state-of-the-art performance on this very challenging task. Furthermore, the flexibility of the implementation is illustrated using demo systems for a Wall Street Journal (WSJ) task. The HTK ANN extension is planned for release in HTK version 3.5.

Individuals with cochlear implants (CIs) show reduced word and auditory emotion recognition abilities relative to their peers with normal hearing. Modern CI processing strategies are designed to preserve acoustic cues requisite for word recognition rather than those cues required for accessing other signal information (e.g., talker gender or emotional state). While word recognition is undoubtedly important for communication, the inaccessibility of this additional signal information in speech may lead to negative social experiences and outcomes for individuals with hearing loss. This study aimed to evaluate whether the emphasis on word recognition preservation in CI processing has unintended consequences on the perception of other talker information, such as emotional state. Twenty-four young adult listeners with normal hearing listened to sentences and either reported a target word in each sentence (word recognition task) or selected the emotion of the talker (emotion recognition task) from a list of options (Angry, Calm, Happy, and Sad). Sentences were blocked by task type (emotion recognition versus word recognition) and processing condition (unprocessed versus 8-channel noise vocoder) and presented randomly within the block at three signal-to-noise ratios (SNRs) in a background of speech-shaped noise. Confusion matrices showed the number of errors in emotion recognition by listeners. Listeners demonstrated better emotion recognition performance than word recognition performance at the same SNR. Unprocessed speech resulted in higher recognition rates than vocoded stimuli. Recognition performance (for both words and emotions) decreased with worsening SNR. Vocoding speech resulted in a greater negative impact on emotion recognition than it did for word recognition. These data confirm prior work that suggests that in background noise, emotional prosodic information in speech is easier to recognize than word information, even after simulated CI processing. However, emotion recognition may be more negatively impacted by background noise and CI processing than word recognition. Future work could explore CI processing strategies that better encode prosodic information and investigate this effect in individuals with CIs as opposed to vocoded simulation. This study emphasized the need for clinicians to consider not only word recognition but also other aspects of speech that are critical to successful social communication.

ASR for emotional speech: Clarifying the issues and enhancing performance

In this paper, we propose a communication scheme using automatic speech recognition and speech synthesis in order to assist dysarthric speakers. This system aims at improving recognition rate and intelligibility of dysarthric speech. An HMM-based recognizer using variable duration of Hamming window permits to raise the recognition rate of dysarthric speech up to 80 %. In order to improve the intelligibility of synthetic speech while keeping the naturalness close to the voice of dysarthric speaker, we synthesize the recognized text using new basic unit segmenter, a new concatenating algorithm and a grafting technique to correct the bad pronounced phonemes. The NEMOURS dysarthric database is used to evaluate the proposed assistive communication system. Results show that a rate of 65 % to 80% of correct word recognition and a Mean Opinion Score (MOS) of 4 were obtained.