Speech2Spikes: Efficient Audio Encoding Pipeline for Real-time Neuromorphic Systems

In this study, a real-time automatic speech recognition (ASR) system based on the Kaldi ASR toolkit, with low-latency and customizable models, without any internet connection, was developed. The proposed ASR system includes a voice activity detection (VAD) module and an audio transmitter as a front-end speech processing and a decoder for the received audio signals. The ASR system was evaluated in terms of ASR accuracy and speech processing speed. Consequently, the ASR system achieved high ASR accuracy on the CSJ (Corpus of Spontaneous Japanese) test set with super low-latency.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

In addressing the problem of achieving high-accuracy real-time speech recognition systems, we focus on recognizing speech from ARPA's 20,000-word Wall Street Journal (WSJ) task, using current UNIX workstations. We have found that our standard approach---using a narrow beam width in a Viterbi search for simple discrete-density hidden Markov models (HMMs)---works in real time with only very low accuracy. Our most accurate algorithms recognize speech many times slower than real time. Our (yet unattained) goal is to recognize speech in real time at or near full accuracy.We describe the speed/accuracy trade-offs associated with several techniques used in a one-pass speech recognition framework:• Trade-offs associated with reducing the acoustic modeling resolution of the HMMs (e.g., output-distribution type, number of parameters, cross-word modeling)• Trade-offs associated with using lexicon trees, and techniques for implementing full and partial bigram grammars with those trees• Computation of Gaussian probabilities are the most time-consuming aspect of our highest accuracy system, and techniques allowing us to reduce the number of Gaussian probabilities computed with little or no impact on speech recognition accuracy.Our results show that tree-based modeling techniques used with appropriate acoustic modeling approaches achieve real-time performance on current UNIX workstations at about a 30% error rate for the WSJ task. The results also show that we can dramatically reduce the computational complexity of our more accurate but slower modeling alternatives so that they are near the speed necessary for real-time performance in a multipass search. Our near-future goal is to combine these two technologies so that real-time, high-accuracy large-vocabulary speech recognition can be achieved.

It has further shaped the domain of real-time voice-controlled speech recognition systems. Things like language bias, computational expense and background noise have made strides more difficult in the past. This paper provides a novel view on these tasks, allowing for broader accessibility and real-world applicability of state-of-the-art models. We advocate a multi-dimensional methodology, including ad hoc model contextualization, tailored neural designs, and personalized learning strategies, to achieve voice and chip-height optimization. Existing speech recognition systems have a major limitation of being limited to few languages, dialects and accents. This study introduces a multilingual and multicultural model to address this issue and also provides access to the benefit of no-bias technologies to all parts of society. Out of these regional filters, it can work with more and work on them more precisely. In addition to this, the efficient structure also improves computational efficiency, meaning that the model is able to gain speed in real-time processing on low-power devices, fitting to rising demand for speech recognition in mobile and edge computing settings. Performance in contextual understanding remains a problem, errors in pronunciation or deviations from the dialect tend to lead to mistakes. As a result, the current study utilise semantic analysis and natural language processing (NLP) methods to assist understanding across different languages globally. These services are used in applications like medical/legal transcription or customer support, where correct transcription is critical. Additionally, the architecture enhances real-time processing by reducing latency and increasing responsiveness, which is critical in emergency response systems and autonomous vehicles where timely decision-making is crucial. By enhancing the efficiency and accuracy of ANN-based speech recognition, this research drives advancements in increasingly more accessible, effective, and reliable voice-activated technologies.

We present an end-of-utterance detector for real-time automatic speech recognition in far-field scenarios. The proposed system consists of three components: a long short-term memory (LSTM) neural network trained on acoustic features, an LSTM trained on l-best recognition hypotheses of the automatic speech recognition (ASR) decoder, and a feedforward deep neural network (DNN) combining embeddings derived from both LSTMs with pause duration features from the ASR decoder. At inference time, lower and upper latency (pause duration) bounds act as safeguards. Within the latency bounds, the utterance end-point is triggered as soon as the DNN posterior reaches a tuned threshold. Our experimental evaluation is carried out on real recordings of natural human interactions with voice-controlled far-field devices. We show that the acoustic embeddings are the single most powerful feature and particularly suitable for cross-lingual applications. We furthermore show the benefit of ASR decoder features, especially as a low cost alternative to ASR hypothesis em-beddings.

Real-time audio and speech processing are now a necessary component of modern mobile applications, enabling users to engage comfortably with voice-based inputs and speech recognition capabilities. Mobile environments do come with inherent challenges such as limited computation, battery constraints, and the need for low-latency noise-immune solutions. The present paper addresses the design and implementation of real-time speech recognition and command-detection systems optimized for mobile platforms. We focus on low-latency methods and algorithms for wake word spotting and command understanding in order to provide quick reaction to user feedback. We also address noise-robust modeling techniques to battle the various acoustic scenes of mobile scenarios, making the most out of deep neural networks architectures, noise-irrelevant feature learning, and data augmentation. For optimizing model performance on mobile, we discuss model compression, pruning, and the benefits of on-device inference for real-time processing. Next, we discuss case studies and identify future directions and challenges in deploying real-time speech systems on mobile, in terms of accuracy vs. latency vs. power consumption. The results discussed herein present a comprehensive framework for building mobile-based speech and audio processing technology.

Combined speech enhancement and auditory modelling for robust distributed speech recognition

This paper presents a set of 'strategies' that enabled the development of a real-time continuous speech recognition system for Czech language. The optimization strategies include efficient computation of HMM probability densities, pruning schemes applied to HMM states, words and word hypotheses, a bigram compression technique as well as parallel implementation of the real recognition system. In a series of off-line speaker-independent tests done with 1,600 Czech sentences based on 7,033-word lexicon we got 65% recognition rate. Several on-line tests proved that similar rates can be achieved under real conditions and with response time that is shorter than 1 second.

Recently, one of the most common approaches used in speech recognition is deep learning. The most advanced results have been obtained with speech recognition systems created using convolutional neural network (CNN) and recurrent neural networks (RNN). Since CNNs can capture local features effectively, they are applied to tasks with relatively short-term dependencies, such as keyword detection or phoneme- level sequence recognition. This paper presents the development of a deep learning and speech command recognition system. The Google Speech Commands Dataset has been used for training. The dataset contained 65.000 one-second-long words of 30 short English words. That is, %80 of the dataset has been used in the training and %20 of the dataset has been used in the testing. The data set consists of one-second voice commands that have been converted into a spectrogram and used to train different artificial neural network (ANN) models. Various variants of CNN are used in deep learning applications. The performance of the proposed model has reached %94.60.

Identification or recognition of speaker uttered short speech commands, known as speech command recognition (SCR), is a challenging problem in digital speech processing. It plays a vital role in various real-life applications such as Internet of things (IoT) devices and assistive technology to name a few. The deep learning models like convolutional neural networks (CNNs) have shown the potential ability to solve SCR tasks. However, these models’ performance heavily depends on the appropriate choice of associated hyperparameters values, which are very labor-intensive and time-consuming. This paper proposes a genetic algorithm (GA)-based hyperparameters optimization approach to fine-tune CNN hyperparameters for performing SCR tasks. The proposed method is trained and tested with the google speech command (GSC) dataset by considering eight speech commands. The experimental results demonstrate the efficiency and effectiveness of the proposed method for solving speech features-based classification problems like SCR.

We propose a novel one-way ring optical fiber Sagnac system for voice sensing. The system can realize real-time speech-to-text speech recognition using an open-source neural network model.

Automatic speech recognition by computer, in real-time and using only an input audio signal, has been developed over the last two decades to the point where it is useful for a number of practical applications, such as telephone enquiry services and office dictation. The purpose of talking to a computer is first examined and the inherent reasons for the difficulty of understanding speech are suggested. Three specific problems in the recognition process are highlighted: data rate reduction, speaker variability and application domain dependence. The currently successful metholology used in speech recognition systems is described in non-technical terms. The principal processes required for real-time automatic speech recognition are introduced, and the need for adaptation and customization discussed. Some suggestions for the directions of future research and applications are given

This paper proposes an architecture for real-time large vocabulary speech recognition on a mobile embedded device. The speech recognition system is based on Hidden Markov Model (HMM), which involves complex mathematical operations such as probability estimation and Viterbi decoding. This computational nature makes it power hungry and realtime recognition is not achieved by porting software solutions on embedded device. Our system architecture has a low power embedded processor and dedicated ASIC units for complex computations. These units operate at a low frequency of 50 MHz thus consuming low power. The system uses RAM for the intermediate values and flash memory to store acoustic and language models for speech recognition.

We study transfer learning in convolutional network architectures applied to the task of recognizing audio, such as environmental sound events and speech commands. Our key finding is that not only is it possible to transfer representations from an unrelated task like environmental sound classification to a voice-focused task like speech command recognition, but also that doing so improves accuracies significantly. We also investigate the effect of increased model capacity for transfer learning audio, by first validating known results from the field of Computer Vision of achieving better accuracies with increasingly deeper networks on two audio datasets: UrbanSound8k and Google Speech Commands. Then we propose a simple multiscale input representation using dilated convolutions and show that it is able to aggregate larger contexts and increase classification performance. Further, the models trained using a combination of transfer learning and multiscale input representations need only 50% of the training data to achieve similar accuracies as a freshly trained model with 100% of the training data. Finally, we demonstrate a positive interaction effect for the multiscale input and transfer learning, making a case for the joint application of the two techniques.

The paper reports one microcontroller car which processes speech - LINGYANG SPCE061A microchip forming speech recognition system. The speech recognition system not only has high recognizable veracity, small volume, economy-power consumption, lower cost, high operation speed and real-time speech recognition. Furthermore, the function of speech cue offers a favorable interface for human-computer interaction in the system. These characteristics embody fully predominance of the embedded speech recognition.