Speech recognition using microphone antenna array

A prosthetic hearing implant capable of independently and concurrently processing multiple audio input signals such that each audio input signal is separately utilized to stimulate the basilar membrane of the implant recipient. A prosthetic hearing implant of the present invention simultaneously receives discrete audio signals from each of one or more audio sources, generates a separate set of one or more stimulations signals for each audio signal, and concurrently applies the stimulation signals to the cochlear. Different channels of stimulation may be allocated to each audio input signal. Alternatively, different modes of stimulation may be used for each input signal. Or, in yet another example, the audio input signals may be time-multiplexed such that the input signals are applied in alternating timeslots. Or, in yet other examples, the input signals may be applied using a combination of stimulation channels, modes of stimulation, and alternating time slots. The stimulation signals may be delivered to the same or different electrodes depending on the quantity and frequency component of the received audio input signals, the quantity of available stimulation channels, and other factors.

Generally, a virtual reality (VR) system cannot provide the realistic sound formed by the multi-channel audio signals to a user with the stereo headphone environment. In addition, the VR system has a gap between the visual scene and the sound because it supplies the audio signal having only a constant sound scene without respect to the change of the user's position. To solve these problems, we introduce the sound scene control of binaural sound. Binaural sound is a stereo realistic sound that can be generated by convolving a 10.1 channel audio signal with a head related transfer function (HRTF) coefficients which features all the paths from the multi-channel speaker layout to the ears in free space. However, since a binaural sound is generated using a fixed multi-channel layout and HRTF coefficients, the binaural sound has a constant sound scene and cannot reflect the user's movements. So, we apply the sound scene control scheme that modifies the binaural sound to allow the user's movement in the VR system. Initially, a multichannel layout is re-created according to the user's azimuth change and the original multi-channel signal is mapped to a new ultra multi-channel layout using a constant power panning law. Secondly, the sound level of the new multichannel audio signal is controlled by the user's distance change, using the characteristics of the sound level inversely proportional to distance. Finally, the final multi-channel audio signal is convolved with the HRTF coefficients to produce a binaural sound with the controlled sound scene. As a result, the proposed realistic audio sound generation method allows the current VR system to provide true VR service without distinction between the visual scenes and sounds.

Provided are an apparatus and a method for reproducing a surround wave field using wave field synthesis. The apparatus includes an audio signal analyzer for analyzing a received multi-channel audio signal to check the number of audio signal channels, and extracting a sound source signal for each checked channel from the multi-channel audio signal; a wave field synthesis renderer for localizing the extracted sound source signal for each channel at a virtual sound image outside a narrow space using wave field synthesis so that the extracted sound source signal is suitable for the number of the checked audio signal channels; and an audio reproducer for reproducing the localized virtual sound source signal.

The advent of artificial intelligence (AI) has revolutionized human-computer interaction, making it more natural and intuitive. This research paper presents the development and implementation of an AI Desktop Voice Assistant designed to enhance productivity and accessibility for users. The voice assistant leverages advanced speech recognition, natural language processing (NLP), and machine learning techniques to understand and execute user commands. Key functionalities include voice-activated application control, web searches, personalized reminders, and real-time information retrieval. Our system integrates with widely-used APIs and services, providing a seamless user experience across various tasks. The paper discusses the architecture of the voice assistant, detailing the integration of components such as the speech recognition engine, NLP models, and the dialogue management system. We explore the challenges encountered during development, including accurate speech recognition in noisy environments and handling ambiguous user commands. Solutions implemented to address these challenges are also presented. Furthermore, we conduct a usability study to evaluate the effectiveness and user satisfaction of the voice assistant. The results indicate a high level of user engagement and satisfaction, demonstrating the practical benefits of incorporating AI-driven voice interfaces into desktop environments. Our findings contribute to the ongoing research in human-computer interaction, suggesting pathways for future enhancements in AI voice assistants..

Usually multimedia data have to be compressed before transmitting, and higher compression rate, or equivalently lower bitrate, relieves the load of communication channels but impacts negatively the quality. We investigate the bitrate lower bound for perceptually lossless compression of a major type of multimedia—multichannel audio signals. This bound equals to the perceptible information rate of the signals. Traditionally, Perceptual Entropy (PE), based primarily on monaural hearing measures the perceptual information rate of individual channels. But PE cannot measure the spatial information captured by binaural hearing, thus is not suitable for estimating Spatial Audio Coding (SAC) bitrate bound. To measure this spatial information, we build a Binaural Cue Physiological Perception Model (BCPPM) on the ground of binaural hearing, which represents spatial information in the physical and physiological layers. This model enables computing Spatial Perceptual Entropy (SPE), the lower bitrate bound for SAC. For real-world stereo audio signals of various types, our experiments indicate that SPE reliably estimates their spatial information rate. Therefore, “SPE plus PE” gives lower bitrate bounds for communicating multichannel audio signals with transparent quality.

Artificial Intelligence (AI) and Natural Language Processing (NLP) have significantly transformed human- computer interaction, enabling intelligent systems to process voice commands efficiently. Voice assistants, such as Google Assistant, Amazon Alexa, and Apple Siri, have set industry benchmarks, but they still face challenges in real-time response accuracy, handling ambient noise, and offline functionality. This paper presents the development of a custom AI -powered voice assistant, focusing on improving listening abilities, noise filtration, and command execution efficiency. The proposed system integrates Google Speech Recognition API for real-time speech-to-text conversion, pyttsx3 for text-to- speech synthesis, and natural language processing techniques to interpret and execute commands. Unlike cloud-dependent voice assistants, this system provides offline capabilities for essential commands, ensuring flexibility and usability even in low- connectivity environments. Experimental results demonstrate that the assistant achieves over 91% speech recognition accuracy in controlled environments, with an average command execution time of less than one second. Future enhancements include deep learning- based NLP models, real-time wake-word detection, and a graphical user interface for better interaction. The proposed system serves as a foundation for customizable, intelligent, and efficient AI-powered voice assistants.

Human computer interaction (HCI) is very crucial in our day-to-day activity. Speech is one of the essential and intuitive ways to interact with machines such as Smartphone, which has multiple sensors as microphone, camera, etc. An efficient performance speech recognition system improves interaction between man and machines by making latter more receptive to user needs. Such system has Automatic speech recognition (ASR) engine, which is facing a unique challenge of accuracy in recognition rate. By integrating acoustic signal feature vectors with the visual features, a more robust audiovisual speech recognition engine (AVSR) could be developed for real environmental scenarios. This paper presents past research and development in the field of ASR and AVSR technologies. It describes key technological perspective and admiration of the fundamental progress in ASR and AVSR. The objective of this review is to summarize and compare some of the well-known methods experimented by previous researchers, and to conclude with direction on future research proficiency in HCI system using ASR and AVSR engine.

GOL-SFSTS based few-shot learning mechanical anomaly detection using multi-channel audio signal

The global aging population faces considerable challenges, particularly in communication, due to the prevalence of hearing and speech impairments. To address these, we introduce the AVE speech, a comprehensive multimodal dataset for speech recognition tasks. The dataset includes a 100-sentence Mandarin corpus with audio signals, lip-region video recordings, and six-channel electromyography data, collected from 100 participants. Each subject read the entire corpus ten times, with each sentence averaging approximately two seconds in duration, resulting in over 55 hours of multimodal speech data per modality. Experiments demonstrate that combining these modalities significantly improves recognition performance, particularly in cross-subject and high-noise environments. To our knowledge, this is the first publicly available sentence-level dataset integrating these three modalities for large-scale Mandarin speech recognition. We expect this dataset to drive advancements in both acoustic and nonacoustic speech recognition research, enhancing cross-modal learning and human–machine interaction.

Automatic Speech Recognition (ASR) systems convert spoken words into text and play a critical role in voice assistants, transcription services, and other accessibility tools. However, their reliance on machine learning models makes them susceptible to adversarial attacks. By adding small, often imperceptible perturbations to the input audio, or by convolving it with a carefully designed room impulse response, attackers can cause ASR systems to produce incorrect or even malicious transcripts. In this work, we propose an ensemble detection method for the efficient identification of targeted adversarial attacks in ASR systems. Specifically, we introduce MELo-FEST, which calculates the minimum energy in the low-frequency band of an audio signal to detect convolutive adversarial attacks. We then combine two spectrogram-based detection methods with a noise-adding approach to form an ensemble detector capable of identifying both additive and convolutive targeted adversarial attacks. Through extensive experiments, we demonstrate that our proposed ensemble detector can accurately identify adversarial audio generated by both non-adaptive and adaptive CW, PGD, and AdvReverb attacks in Wav2Vec2 and Whisper ASR systems, achieving an F1 score of at least 0.97. Moreover, our method performs both speech recognition and adversarial detection for each input audio sample in an average of under 0.13 seconds for Wav2Vec2 and 0.29 seconds for Whisper, making it well-suited for real-time applications. Additionally, we find that Whisper is more vulnerable than Wav2Vec2 to both non-adaptive and adaptive targeted adversarial attacks.

An apparatus and method for selective distributed speech recognition includes a dialog manager (104) that is capable of receiving a grammar type indicator (170). The dialog manager (104) is capable of being coupled to an external speech recognition engine (108), which may be disposed on a communication network (142). The apparatus and method further includes an audio receiver (102) coupled to the dialog manager (104) wherein the audio receiver (104) receives a speech input (110) and provides an encoded audio input (112) to the dialog manager (104). The method and apparatus also includes an embedded speech recognition engine (106) coupled to the dialog manager (104), such that the dialog manager (104) selects to distribute the encoded audio input (112) to either the embedded speech recognition engine (106) or the external speech recognition engine (108) based on the corresponding grammar type indicator (170).

Audio-visual speech recognition based on joint training with audio-visual speech enhancement for robust speech recognition

Speech and language recognition is one of the most important requirements in daily life. Most people in the world use English as the highly preferable language for easy communication. As a result, English is the highest speaking language in the World. Speech is the easiest way of medium for communication. Hunan-human, human-machine, and machine-machine communication are very much dependent on extensive and robust speech recognition. There are various important features of speech. The accent is one of them. It is a unique feature of speech that varies according to region and dialect. Speech scientists are continuously working on the improvement of speech recognition techniques and performance. Various challenges are being faced day by day and new areas of research are opened to overcome those difficulties. Variation of accent is one of the biggest challenges in improving speech recognition performances. In this work, the native and non-native English accent has been considered. The classification task has been performed on the English speech of non-native speakers and native American English speakers. Three sets of male and female speaker data both for native and non-native speech have been collected from the Voice Cloning Toolkit corpus of the Centre for Speech Technology Research. Mel Frequency Cepstral Coefficient features were extracted from the audio signals and given as input parameters to the supervised machine learning models. Evaluation metrics have been generated, multiple results were produced and compared.

Introduction: In order to assess how well conventional and hybrid pitch detection techniques perform in speech processing applications, a comparative analysis of the two types of methods is conducted. Method: A proposed hybrid approach, Proposed PEF+CEP, is examined alongside five traditional algorithms, namely Normalized Correlation Function (NCF), Pitch Estimation Filter (PEF), Log- Harmonic Summation (LHS), Summation of Residual Harmonics (SRH) and Cepstrum Pitch Determination (CEP). The effectiveness is evaluated using performance metrics like accuracy, specificity, sensitivity, and Gross Pitch Error (GPE). Results and Discussion: Our findings show that the accuracy and specificity of the traditional methods are impressive; the accuracy and sensitivity of the suggested hybrid method surpass their performance, with an astounding 98.8% accuracy and 99.2% sensitivity. Conclusion: Furthermore, the Proposed PEF+CEP method is a promising solution for accurate and dependable pitch detection in speech processing applications because it strikes a strong balance between computational efficiency and robustness. These results open up new avenues for research in the field of speech processing and demonstrate the potential of hybrid approaches.

A voice detection circuit uses normal and customary signal peaks and pauses in speech to distinguish voice from mere noise. The voice detection circuit includes an audio input device for receiving an audio signal and comparing it to a threshold, and a detector for generating the VOX output signal responsive to the presence of the peaks and pauses in the audio signal which rise above the threshold. The switch is actuated upon the VOX output signal indicating that the audio signal includes selected patterns of peaks and pauses and thus comprises voice signals. Illustratively, the threshold may be varied in accordance with the input signals when the voice detection circuit indicates that the input audio signals do not comprise voice signals.