A Microprocessor for Speech Recognition

The increasing demand for real-time language translation in multilingual societies highlights the need for efficient speech translation systems. This research proposes the development of an English-to-Marathi speech translation framework leveraging a dictionary-based dataset to enhance linguistic accuracy and contextual understanding. Marathi, being a morphologically rich language with unique syntactic and semantic structures, poses significant challenges when paired with English, a structurally divergent language. The proposed system will integrate automatic speech recognition (ASR), machine translation (MT), and text-to-speech synthesis (TTS) to achieve seamless speech-to-speech conversion. A dictionary-based dataset will serve as the foundation, enriched with linguistic rules and context-aware mappings to address lexical and syntactic divergences. By incorporating rule-based techniques alongside advanced neural network models, such as sequence-to- sequence transformers and attention mechanisms, the system aims to capture the nuances of Marathi grammar, idiomatic expressions, and gendered verb forms. The hybrid approach will ensure both robustness and adaptability across diverse linguistic inputs. The research will evaluate translation accuracy, fluency, and contextual relevance using metrics such as BLEU and TER. This study aspires to contribute to the growing field of low-resource language translation by enabling effective communication between English and Marathi speakers, thus fostering inclusivity in education, commerce, and digital communication. In the proposed research work the focus will be given to translation of English language video to Marathi language. Considering the strong grammar and variety of word dictionaries of the Marathi language, a strong algorithm based on deep learning techniques is required. Speech to Speech machine translation using deep learning focuses on the language conversion from English to Marathi which allows for seamless communication between people who speak Marathi languages. Speech-to speech translation solves this problem by translating spoken words in real time, allowing people who speak Marathi languages to communicate with one another.

Time resolved imaging techniques, such as MEG and EEG, are unique in their ability to reveal the rich dynamic spatiotemporal patterning of neural activities. Here we propose a technique based on spatiotemporal searchlight Representational Similarity Analysis (RSA) of combined MEG and EEG (EMEG) data to directly analyze the multivariate pattern of information flow across the brain. This novel technique can recognize fine-grained dynamic neural computations both in space and in time. A prime example of such neural computations is our ability to understand spoken words in real time. A computational approach to these processes is suggested by the Cohort Model of spoken-word recognition. Here we show how spatiotemporal searchlight RSA applied to source estimations of EMEG data can provide insights into the neural correlates of the cohort model within bilateral front temporal brain regions.

Aiming at the problem that the visual SLAM algorithm is easily disturbed by complex environmental factors, resulting in poor closed-loop accuracy and real-time performance. An improved closed-loop detection algorithm based on a dynamic increase or decrease mechanism is proposed. The algorithm can dynamically increase or decrease the number of words in the visual bag of words in real time according to the complexity of the scene; and optimize the selected keyframes through a double-threshold keyframe selection strategy to reduce the redundancy of keyframes, thus improving the system's real-time performance. The algorithm in this paper can achieve localization and position correction in complex environments. Test results in public datasets and real scenes demonstrate the advantages of the proposed algorithm in terms of accuracy, real-time performance, and adaptability to the environment.

INTEX is a linguistic development environment that includes large-coverage dictionaries and grammars, and parses texts of several million words in real time. INTEX has tools to create and maintain large-coverage lexical resources as well as morphological and syntactic grammars. Dictionaries and grammars are applied to texts in order to locate morphological, lexical and syntactic patterns, remove ambiguities, and tag simple and compound words. INTEX can build lemmatized concordances and indices of large texts with respect to all types of Finite State patterns. INTEX is used as a corpus processor, to analyze literary, journalistic and technical texts. I describe here the subset of tools used to perform advanced search requests on large texts.

A foundational assumption of human communication is that speakers should say as much as necessary, but no more. Yet, people routinely produce redundant adjectives and their propensity to do so varies cross-linguistically. Here, we propose a computational theory, whereby speakers create referential expressions designed to facilitate listeners' reference resolution, as they process words in real time. We present a computational model of our account, the Incremental Collaborative Efficiency (ICE) model, which generates referential expressions by considering listeners' real-time incremental processing and reference identification. We apply the ICE framework to physical reference, showing that listeners construct expressions designed to minimize listeners' expected visual search effort during online language processing. Our model captures a number of known effects in the literature, including cross-linguistic differences in speakers' propensity to over-specify. Moreover, the ICE model predicts graded acceptability judgments with quantitative accuracy, systematically outperforming an alternative, brevity-based model. Our findings suggest that physical reference production is best understood as driven by a collaborative goal to help the listener identify the intended referent, rather than by an egocentric effort to minimize utterance length. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A new SOI substrate was formed by using artificial 1 µm × 1 µm seeds, solid phase growth, selective epitaxy and polishing technology. The NMOS and PMOS transistors were formed on the substrate. The electron and hole mobility of these devices were of ∼160 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> /v.sec and ∼100 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> /v.sec, respectively. The 21 stages ring oscillator gave ∼130 psec of the delay time per stage and ∼220 fJ power-delay product at Vdd of 5 volt. A 60 µm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> CMOS SRAM Cell was formed on this substrate using 1 µm design rule. The basic read/write operations were measured at Vdd of 5 volt.

The use of speech recognition in the area of office automation has received little experimental evaluation. Most speech recognition studies have involved testing in military command and control or aircraft environments. Speech control of spreadsheet and word processing programs is discussed by various manufacturers and in the popular press, yet no empirical data have been presented to substantiate its use in these applications. In one of the few studies applicable to the office environment (Murray, Van Praag, and Gilfoil, 1983), the use of speech to control cursor motion was found to be less efficient than the use of traditional cursor keys. To justify the expense of adding equipment to an office workstation, a significant increase in worker productivity needs to be demonstrated. The dedicated word processing operator, involved in the task of transcribing large quantities of textual data from hardcopy or dictation tapes, was chosen for examination in this study. The majority of word processing operator keystroke activity is within the QWERTY portion of the text entry keyboard. However, many of the necessary control operations used during text entry are located on dedicated or soft function keys. These function keys are often located well away from the QWERTY portion of the keyboard, interrupting the touch typist's activity in order to locate and activate the appropriate key. In this study, the use of speech input to select operations normally assigned to function keys was evaluated. It was hypothesized that the operator would experience less distraction in the primary task of text entry when operations were selected verbally rather than by manual activation of the function keys. This would be reflected in reduced text entry time and possible reduction in keystroke errors when selecting operations. In contrast to the earlier study examining speech control of a cursor positioning task in which speech was found to be inferior to manual control, this study used speech to control discrete operations. Subjects were experienced word processing operators. The primary task was transcription of text documents requiring frequent format changes. The subjects were placed in two groups. Group I was trained to operate a word processing package in which control operations were selected by using hard and soft function keys located above and to the right of the QWERTY keypad. Group II was trained to use the identical word processing software, with the difference being that control operations were selected by using speech input. Data collected and analyzed included time to complete the text entry task, number of entry errors, number of command errors, and subjective ratings of operator satisfaction with the system used.

In addressing the problem of achieving high-accuracy real-time speech recognition systems, we focus on recognizing speech from ARPA's 20,000-word Wall Street Journal (WSJ) task, using current UNIX workstations. We have found that our standard approach---using a narrow beam width in a Viterbi search for simple discrete-density hidden Markov models (HMMs)---works in real time with only very low accuracy. Our most accurate algorithms recognize speech many times slower than real time. Our (yet unattained) goal is to recognize speech in real time at or near full accuracy.We describe the speed/accuracy trade-offs associated with several techniques used in a one-pass speech recognition framework:• Trade-offs associated with reducing the acoustic modeling resolution of the HMMs (e.g., output-distribution type, number of parameters, cross-word modeling)• Trade-offs associated with using lexicon trees, and techniques for implementing full and partial bigram grammars with those trees• Computation of Gaussian probabilities are the most time-consuming aspect of our highest accuracy system, and techniques allowing us to reduce the number of Gaussian probabilities computed with little or no impact on speech recognition accuracy.Our results show that tree-based modeling techniques used with appropriate acoustic modeling approaches achieve real-time performance on current UNIX workstations at about a 30% error rate for the WSJ task. The results also show that we can dramatically reduce the computational complexity of our more accurate but slower modeling alternatives so that they are near the speed necessary for real-time performance in a multipass search. Our near-future goal is to combine these two technologies so that real-time, high-accuracy large-vocabulary speech recognition can be achieved.

The article discusses the system of dialogue control manipulation robots. The analysis of the basic methods of automatic speech recognition, speech understanding, dialogue management, voice response synthesis in dialogue systems has been carried out. Three types of dialogue management are considered as "system initiative", "user initiative" and "combined initiative". A system of object-oriented dialog control of a robot based on the theory of finite state machines with using a deep neural network is proposed. The main difference of the proposed system lies in the separate implementation of the dialogue process and robot’s actions, which is close to the pace of natural dialogue control. This method of constructing a dialogue control robot allows system to automatically correct the result of speech recognition, robot’s actions based on tasks. The necessity of correcting the result of speech recognition and robot’s actions may be caused by the users’ accent, working environment noise or incorrect voice commands. The process of correcting speech recognition results and robot’s actions consists of three stages, respectively, in a special mode and a general mode. The special mode allows users to directly control the manipulator by voice commands. The general mode extends the capabilities of users, allowing them to get additional information in real time. At the first stage, continuous speech recognition is built by using a deep neural network, taking into account the accents and speech speeds of various users. Continuous speech recognition is a real-time voice to text conversion. At the second stage, the correction of the speech recognition result by managing the dialogue based on the theory of finite automata. At the third stage, the actions of the robot are corrected depending on the operating state of the robot and the dialogue management process. In order to realize a natural dialogue between users and robots, the problem is solved in creating a small database of possible dialogues and using various training data. In the experiments, the dialogue system is used to control the KUKA manipulator (KRC4 control) to put the desired block in the specified location, implemented in the Python environment using the RoboDK software. The processes and results of experiments confirming the operability of the interactive robot control system are given. A fairly high accuracy (92 %) and an automatic speech recognition rate close to the rate of natural speech were obtained.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

Automatic speech recognition (ASR) for dysarthric speech is challenging. The acoustic characteristics of dysarthric speech are highly variable and there are often fewer distinguishing cues between phonetic tokens. Multimodal ASR utilises the data from other modalities to facilitate the task when a single acoustic modality proves insufficient. Articulatory information, which encapsulates knowledge about the speech production process, may constitute such a complementary modality. Although multimodal acoustic-articulatory ASR has received increasing attention recently, incorporating real articulatory data is under-explored for dysarthric speech recognition. This paper investigates the effectiveness of multimodal acoustic modelling using real dysarthric speech articulatory information in combination with acoustic features, especially raw signal representations which are more informative than classic features, leading to learning representations tailored to dysarthric ASR. In particular, various raw acoustic-articulatory multimodal dysarthric speech recognition systems are developed and compared with similar systems with hand-crafted features. Furthermore, the difference between dysarthric and typical speech in terms of articulatory information is systematically analysed by using a statistical space distribution indicator called Maximum Articulator Motion Range (MAMR). Additionally, we used mutual information analysis to investigate the robustness and phonetic information content of the articulatory features, offering insights that support feature selection and the ASR results. Experimental results on the widely used TORGO dysarthric speech dataset show that combining the articulatory and raw acoustic features at the empirically found optimal fusion level achieves a notable performance gain, leading to up to 7.6% and 12.8% relative word error rate (WER) reduction for dysarthric and typical speech, respectively. • Systematically compares the articulatory movement pattern mismatch between dysarthric and typical speech. This is achieved through the visualisation of the 3D point cloud of several real articulatory data samples and an analysis of the statistical space distribution across the dataset, employing the Maximum Articulator Motion Range (MAMR) as a metric. • Conduct comprehensive articulatory information analysis, including the computation of mutual information , to identify and select the most task-beneficial articulators, thereby enabling a more informed integration of articulatory features with acoustic features for optimal multimodal ADSR performance. • Investigate multimodal acoustic modelling for ADSR using real articulatory information in combination with acoustic features, encompassing both hand-crafted features and raw signal representations, e.g., raw waveform with non-parametric and parametric CNNs, magnitude spectrum, the raw source and filter components, and raw real and imaginary parts of the Fourier transform.

In the context of huge volume of data generated in the last couple of years and the need of processing audio stream in real time data processing flows, this paper aims to explore, identify and experiment Automatic Speech Recognition (ASR) models in the context of low latency applications depending on different contexts using specific data sets. Our work aims to explore whether if such models can be deployed in the real time stream audio workflow and for that we compare multiple size of the Whisper model for multiple data sets and analyze the results. Additionally, we tested the models for English and Romanian language. In the future, for the multi-language context it is highly important to analyze using model dedicated fine-tuned for specific language and compare with the results of multi-language models. This paper contributes to the field by providing evidence, comparison, statistics, level of accuracy in order to use ASR models in the context of low latency applications, exploring multiple versions of Whispers models and help researcher in dimensioning the environments for real time infrastructure and extend streaming capabilities with Artificial Intelligence models. Equally, the paper can used for IT professional in running critical systems in different industries to emphasize the techniques to be used in running ASR model in production.

The anticipation and remediation of operational problems while drilling an oilwell is the main goal of real time measurements of drilling parameters, such as bottomhole pressure, flow rate, pump pressure, torque, drag, among others. The petroleum industry has spent a great amount of financial resources to ensure the quality and availability of these data, but the knowledge for a correct analysis and interpretation of them is still far from being spread among the rigsite teams and drilling engineers. Nowadays, the interpretation of real time drilling data to identify possible operational problems is done by a drilling analysis specialist. However, this can be a very subjective job since it depends on the specialist experience. These analists also take their decisions based on intuition and qualitative rather than quantitative criteria. Petrobras has developed a computational tool (called PWDa) to interpret real time drilling data, predicting and analyzing drilling operational parameters (such as pump pressure, bottomhole pressure, torque and drag). The software detects abnormal behaviors (such as an unexpected increasing trend on bottomhole pressure) and establishes quantitative criteria in order to identify a possible cause, suggesting corrective and/or preventive actions. The main goal of the software is the establishment of an automated methodology to interpret operational parameters in real time helping the drilling engineers to take right and fast decisions. The software is being currently implemented at Petrobras Real Time Operations (RTO) rooms and is providing good results. Over 70 wells have already been monitored with PWDa and several operational problems (such as washout, mud losses, bit wear, downhole motor fail, deficient hole cleaning, pore pressure increments, etc) were successfully identified, allowing the operators to take fast decisions and avoiding riskier situations. The wells monitored include deep water exploratory wells (mostly), directional development wells and extended reach wells. This work aims to highlight the benefits generated by the implementation of the technology. The interaction with the drilling team, including operator and service company members will be discussed. Introduction The analyzis of PWD (Pressure While Drilling) data and other operational parameters (such as rate of penetration, standpipe pressure, flow rate, torque, drag, etc) is an important tool to identify and prevent several operational problems (Aragao et al). The real time interpretation of these data may be very useful to reduce non productive time, risks and operational costs. According to Teixeira et al, most of events and problems have direct or indirect impact on bottomhole pressure and standpipe pressure. Some of them may also affect torque and drag. Problems like poor hole cleaning, annular obstructions, wellbore collapse, kicks, washouts and mud losses will affect the amount of solids in the annular space and/or friction losses and, therefor, will directly affect standpipe and bottomhole pressure (Aragao, et al, 2005). Thus, the analysis of pressure data is a key element to identify and prevent operational problems. Additionaly, when other parameters are simultaneously analyzed (modlogging measurements, for instance), the interpretation becomes much richer.

In the educational process, the support for the presentations displayed at courses, seminars and laboratory hours are the PowerPoint slides. Usually, a simple on-screen slide is not enough for students to understand complex information, requiring additions like verbal explanations and sometimes, to complete the presentation, there are needed drawings, sketches, charts, flowcharts or arrows. To create a presentation, there are used various methods today, from the simplest ones (projecting the slides on the whiteboard and the usage of water-based markers to overwrite) to some more complex, like using interactive whiteboards which can be written directly with special pens. However, there are cases when the screen size is too large and the reader is not even near it. The same thing happens when there is not used a projection screen and the presentation is displayed on a white wall. In these cases, it is required a new approach for teaching methods, especially for emphasize and addition of information in real time. This paper presents the new methodology for using a Natural User Interface (NUI) dedicated device (e.g. Microsoft Kinect) in PowerPoint presentations, considering the following actions: achieving additions and changes in real time, on the same screen with the PowerPoint presentation. Kinect is a peripheral low-cost device that allows users to interact with data, using body gestures and voice commands. We propose a series of intuitive NUI actions based on body postures, gestures and voice commands used to make additions to the classical presentation such as underline, highlight, drawings, diagrams, various smart-arts. The GUI of the developed program is similar to the one of the classical Windows Paint program. There were performed tests and calibration of equipment, in order to modify / correct the positions / movements attached to each action. The advantage of the developed application is the usage of low cost equipment to create an intuitive human computer interface that improves the traditional presentations.

This article analyzes parallel implementation of GPU in the process of generating speech spectrogram images, which is most commonly used as a parameter in speech recognition. The convenience of generating and recognizing spectrogram images from speech signals is that they are rich in character sets compared to other speech recognition methods. Therefore, most modern speech recognition systems use spectrograms. Based on the results of the experiment, it can be seen that the production of spectrogram images from speech signals in the graphics processor was faster than the central processor. This will improve the quality of speech recognition and identification in real time.