Techniques to achieve an accurate real-time large-vocabulary speech recognition system

In this study, a real-time automatic speech recognition (ASR) system based on the Kaldi ASR toolkit, with low-latency and customizable models, without any internet connection, was developed. The proposed ASR system includes a voice activity detection (VAD) module and an audio transmitter as a front-end speech processing and a decoder for the received audio signals. The ASR system was evaluated in terms of ASR accuracy and speech processing speed. Consequently, the ASR system achieved high ASR accuracy on the CSJ (Corpus of Spontaneous Japanese) test set with super low-latency.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

This paper proposes an architecture for real-time large vocabulary speech recognition on a mobile embedded device. The speech recognition system is based on Hidden Markov Model (HMM), which involves complex mathematical operations such as probability estimation and Viterbi decoding. This computational nature makes it power hungry and realtime recognition is not achieved by porting software solutions on embedded device. Our system architecture has a low power embedded processor and dedicated ASIC units for complex computations. These units operate at a low frequency of 50 MHz thus consuming low power. The system uses RAM for the intermediate values and flash memory to store acoustic and language models for speech recognition.

It has further shaped the domain of real-time voice-controlled speech recognition systems. Things like language bias, computational expense and background noise have made strides more difficult in the past. This paper provides a novel view on these tasks, allowing for broader accessibility and real-world applicability of state-of-the-art models. We advocate a multi-dimensional methodology, including ad hoc model contextualization, tailored neural designs, and personalized learning strategies, to achieve voice and chip-height optimization. Existing speech recognition systems have a major limitation of being limited to few languages, dialects and accents. This study introduces a multilingual and multicultural model to address this issue and also provides access to the benefit of no-bias technologies to all parts of society. Out of these regional filters, it can work with more and work on them more precisely. In addition to this, the efficient structure also improves computational efficiency, meaning that the model is able to gain speed in real-time processing on low-power devices, fitting to rising demand for speech recognition in mobile and edge computing settings. Performance in contextual understanding remains a problem, errors in pronunciation or deviations from the dialect tend to lead to mistakes. As a result, the current study utilise semantic analysis and natural language processing (NLP) methods to assist understanding across different languages globally. These services are used in applications like medical/legal transcription or customer support, where correct transcription is critical. Additionally, the architecture enhances real-time processing by reducing latency and increasing responsiveness, which is critical in emergency response systems and autonomous vehicles where timely decision-making is crucial. By enhancing the efficiency and accuracy of ANN-based speech recognition, this research drives advancements in increasingly more accessible, effective, and reliable voice-activated technologies.

Despite the maturity and availability of speech recognition systems, there are few available spiking speech recognition tasks that can be implemented with current neuromorphic systems. The methods used previously to generate spiking speech data are not capable of encoding speech in real-time or encoding very large modern speech datasets efficiently for input to neuromorphic processors. The ability to efficiently encode audio data to spikes will enable a wider variety of spiking audio datasets to be available and can also enable algorithmic development of real-time neuromorphic automatic speech recognition systems. Therefore, we developed speech2spikes, a simple and efficient audio processing pipeline that encodes recorded audio into spikes and is suitable for real-time operation with low-power neuromorphic processors. To demonstrate the efficacy of our method for audio to spike encoding we show that a small feed-forward spiking neural network trained on data generated with the pipeline achieves accuracy on the Google Speech Commands recognition task, exceeding the state-of-the art set by Spiking Speech Commands, a prior spiking encoding of the Google Speech Commands dataset, by over 10%. We also demonstrate a proof-of-concept real-time neuromorphic automatic speech recognition system using audio encoded with speech2spikes streamed to an Intel Loihi neuromorphic research processor.

We present an end-of-utterance detector for real-time automatic speech recognition in far-field scenarios. The proposed system consists of three components: a long short-term memory (LSTM) neural network trained on acoustic features, an LSTM trained on l-best recognition hypotheses of the automatic speech recognition (ASR) decoder, and a feedforward deep neural network (DNN) combining embeddings derived from both LSTMs with pause duration features from the ASR decoder. At inference time, lower and upper latency (pause duration) bounds act as safeguards. Within the latency bounds, the utterance end-point is triggered as soon as the DNN posterior reaches a tuned threshold. Our experimental evaluation is carried out on real recordings of natural human interactions with voice-controlled far-field devices. We show that the acoustic embeddings are the single most powerful feature and particularly suitable for cross-lingual applications. We furthermore show the benefit of ASR decoder features, especially as a low cost alternative to ASR hypothesis em-beddings.

Real-time audio and speech processing are now a necessary component of modern mobile applications, enabling users to engage comfortably with voice-based inputs and speech recognition capabilities. Mobile environments do come with inherent challenges such as limited computation, battery constraints, and the need for low-latency noise-immune solutions. The present paper addresses the design and implementation of real-time speech recognition and command-detection systems optimized for mobile platforms. We focus on low-latency methods and algorithms for wake word spotting and command understanding in order to provide quick reaction to user feedback. We also address noise-robust modeling techniques to battle the various acoustic scenes of mobile scenarios, making the most out of deep neural networks architectures, noise-irrelevant feature learning, and data augmentation. For optimizing model performance on mobile, we discuss model compression, pruning, and the benefits of on-device inference for real-time processing. Next, we discuss case studies and identify future directions and challenges in deploying real-time speech systems on mobile, in terms of accuracy vs. latency vs. power consumption. The results discussed herein present a comprehensive framework for building mobile-based speech and audio processing technology.

This paper presents a parallel phoneme recognition algorithm based on the continuous Hidden Markov Model (HMM). The parallel phoneme recognition algorithm distributes 3-state HMMs of context dependent phonemes to the multiprocessors, computes output probabilities in parallel, and enhances the Viterbi beam search with a message passing mechanism. The algorithm is implemented in a multi-transputer system using distributed-memory MIMD multiprocessors. Experimental results show the feasibility of the parallel phoneme recognition algorithm in constructing a real-time parallel speech recognition system based on time-consuming continuous HMM.

A new recurrent neural network based language model (RNN LM) with applications to speech recognition is presented. Results indicate that it is possible to obtain around 50% reduction of perplexity by using mixture of several RNN LMs, compared to a state of the art backoff language model. Speech recognition experiments show around 18% reduction of word error rate on the Wall Street Journal task when comparing models trained on the same amount of data, and around 5% on the much harder NIST RT05 task, even when the backoff model is trained on much more data than the RNN LM. We provide ample empirical evidence to suggest that connectionist language models are superior to standard n-gram techniques, except their high computational (training) complexity. Index Terms: language modeling, recurrent neural networks, speech recognition

Combined speech enhancement and auditory modelling for robust distributed speech recognition

This paper presents a set of 'strategies' that enabled the development of a real-time continuous speech recognition system for Czech language. The optimization strategies include efficient computation of HMM probability densities, pruning schemes applied to HMM states, words and word hypotheses, a bigram compression technique as well as parallel implementation of the real recognition system. In a series of off-line speaker-independent tests done with 1,600 Czech sentences based on 7,033-word lexicon we got 65% recognition rate. Several on-line tests proved that similar rates can be achieved under real conditions and with response time that is shorter than 1 second.

In this paper, we propose an application of methods for fast speaker based on kernelizing the eigenspace-based maximum-likelihood linear regression method. We call our new method kernel eigenspace-based maximum-likelihood linear regression adaptation (KEMLLR). In KEMLLR, speaker-dependent (SD) models are estimated from a common speaker-independent (SI) model using MLLR adaptation, and the MLLR transformation matrices are mapped to a kernel-induced high-dimensional feature space, wherein principal component analysis is used to derive a set of eigenmatrices. In addition, a composite is used to preserve row information in the transformation matrices. A new speaker's MLLR transformation matrix is then represented as a linear combination of the leading eigenmatrices, which, though exists only in the feature space, still allows the speaker's mean vectors to be found explicitly. As a result, at the end of KEMLLR adaptation, a regular hidden Markov model (HMM) is obtained for the new speaker and subsequent speech recognition is as fast as normal HMM decoding. KEMLLR was tested and compared with other methods on the Resource Management and Wall Street Journal tasks using 5 or 10 s of speech. In both cases, KEMLLR gives the greatest improvement over the SI model with 11%-20% word error rate reduction

Neural network has the advantages of high tolerance of error and ability of parallelism calculation . When applying to the real time speech recognition system, through one time calculation and can get the recognition result immediately that is different from other methods like VQ, DTW, HMM. Those methods need to build models one by one and the same as to recognize, and it is inconvenient on the situation needing the real time response. But using neural network as the identifier, the dimension of input vector will large, it will occupy more memory storage, then will affect the efficiency of calculation. Therefore, in this paper we raise the concept to combine HMM and BPNN, it can reduce the dimension of input vector to decrease the burden of memory storage; on the other hand, it can also raise the calculating efficiency. For resolving a general BP network problem of slow convergence while training, in this paper we raise the concept of using the recognition rate as a factor to judge whether to stop the training procedure or not, which can save more training time and can also get the required recognition rate.

We have developed a low-power VLSI chip for 60-kWord real-time continuous speech recognition based on a Hidden Markov Model (HMM). Our implementation includes a cache architecture using locality of speech recognition, beam pruning using a dynamic threshold, two-stage language model searching, highly parallel Gaussian Mixture Model (GMM) computation based on the mixture level, a variable 50-frame look-ahead scheme and elastic pipeline operation between the Viterbi transition and GMM processing. Results show that our implementation achieves 95% bandwidth reduction (70.86 MB/s) and 78% required frequency reduction (126.5 MHz) for 60-kWord real-time continuous speech recognition. The test chip, fabricated using 40 nm CMOS technology and containing 1.9 M transistors for logic and 7.8 Mbit on-chip memory, occupies 2.2 mm × 2.5 mm area. Measured data show 144 mW power consumption at 126.5 MHz and 1.1 V.

We propose a novel one-way ring optical fiber Sagnac system for voice sensing. The system can realize real-time speech-to-text speech recognition using an open-source neural network model.