Speech-based cursor control

Physical workload during use of speech recognition and traditional computer input devices

How productivity improves in hands-free continuous dictation tasks: lessons learned from a longitudinal study

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Eye tracking is a process that allows an observers gaze to be determined in real time by measuring their eye movements. Recent work has examined the possibility of using gaze control as an alternative input modality in interactive applications. Alternative means of interaction are especially important for disabled users for whom traditional techniques, such as mouse and keyboard, may not be feasible. This paper proposes a novel combination of gaze and voice commands as a means of hands free interaction in a paint style program. A drawing application is implemented which is controllable by input from gaze and voice. Voice commands are used to activate drawing which allow gaze to be used only for positioning the cursor. In previous work gaze has also been used to activate drawing using dwell time. The drawing application is evaluated using subjective responses from participant user trials. The main result indicates that although gaze and voice offered less control that traditional input devices, the participants reported that it was more enjoyable.

In this paper, the improvements in the recently implemented Kannada speech recognition system is demonstrated in detail. The Kannada automatic speech recognition (ASR) system consists of ASR models which are created by using Kaldi, IVRS call flow and weather and agricultural commodity prices information databases. The task specific speech data used in the recently developed spoken dialogue system had high level of different background noises. The different types of noises present in collected speech data had an adverse effect on the on line and off line speech recognition performances. Therefore, to improve the speech recognition accuracy in Kannada ASR system, a noise reduction algorithm is developed which is a fusion of spectral subtraction with voice activity detection (SS-VAD) and minimum mean square error spectrum power estimator based on zero crossing (MMSE-SPZC) estimator. The noise elimination algorithm is added in the system before the feature extraction part. An alternative ASR models are created using subspace Gaussian mixture models (SGMM) and deep neural network (DNN) modeling techniques. The experimental results show that, the fusion of noise elimination technique and SGMM/DNN based modeling gives a better relative improvement of 7.68% accuracy compared to the recently developed GMM-HMM based ASR system. The least word error rate (WER) acoustic models could be used in spoken dialogue system. The developed spoken query system is tested from Karnataka farmers under uncontrolled environment.

Human computer interaction (HCI) is very crucial in our day-to-day activity. Speech is one of the essential and intuitive ways to interact with machines such as Smartphone, which has multiple sensors as microphone, camera, etc. An efficient performance speech recognition system improves interaction between man and machines by making latter more receptive to user needs. Such system has Automatic speech recognition (ASR) engine, which is facing a unique challenge of accuracy in recognition rate. By integrating acoustic signal feature vectors with the visual features, a more robust audiovisual speech recognition engine (AVSR) could be developed for real environmental scenarios. This paper presents past research and development in the field of ASR and AVSR technologies. It describes key technological perspective and admiration of the fundamental progress in ASR and AVSR. The objective of this review is to summarize and compare some of the well-known methods experimented by previous researchers, and to conclude with direction on future research proficiency in HCI system using ASR and AVSR engine.

Speech recognition can be a powerful tool for use in human-computer interaction. Many researchers are investigating the use of speech recognition systems for dictation-based activities, resulting in dramatic improvements in recent years. However, this same experimentation has confirmed that recognition errors and the delays inherent with speech recognition result in unacceptably long task completion times and error rates for cursor control tasks. This study explores the potential of a speech-controlled grid-based cursor control mechanism. An experiment evaluated two alternative grid-based solutions, both using 3-3 grids. One provided a single cursor in the middle of the grid. The second allows users to select a target using any of nine cursors. The results confirm that the nine-cursor solution allowed users to select targets of varying size, distance and direction significantly faster than the one-cursor solution. Overall results are encouraging when compared to earlier evaluations of other speech-based cursor control solutions.

Automatic speech recognition systems can be enormously beneficial for people with physical disabilities, having the potential to provide a fast and easy-to-use input for computer access and control of the home environment. An overview, based on a review of the research literature and clinical observations, is given of the suitability and performance of speech recognition for computer access in two situations ± for people with no speech problems and for people with dysarthria ± and case studies are presented as examples. Commercial large vocabulary systems are often appropriate for people with no, mild or moderate speech problems, although adequate time, training and support are essential. Those with severe dysarthria achieve lower recognition rates, but speech recognition can still be a useful method of input for some individuals. Speech input as a means of controlling the home environment is mainly based on small vocabulary speech recognition, and recognition reliability is dependent upon the consistency of the speech of the individual. Although speech recognition may be a solution for some people, others reject the technology because of inadequate reliability. This is particularly a problem for people with dysarthria, where speech recognition rates can be unacceptably low due to inconsistent speech.

The accuracy of automatic speech recognition (ASR) systems is generally evaluated using corpora of grammatically sound read speech or natural spontaneous speech. This prohibits an accurate estimation of the performance of the acoustic modeling part of ASR because the language modeling performance is inherently integrated in the overall performance metric. In this work, ASR and human speech recognition (HSR) accuracies are compared for null grammar sentences in different signal-to-noise ratios and vocabulary sizes-1000, 2000, 4000, and 8000. The results shed light on differences between ASR and HSR in relative significance of bottom-up word recognition and context awareness.

In an era of advancing human-computer interaction, "GestureSpeak" emerges as a pioneering project that facilitates hands-free control over computing devices through intuitive hand gestures and voice commands. Leveraging the power of MediaPipe and OpenCV in Python, this innovative system enables users to seamlessly navigate their digital environments without the need for traditional input devices. By harnessing the capabilities of computer vision and machine learning, GestureSpeak interprets and responds to users' gestures and vocal instructions, opening new frontiers in accessibility and user experience. This abstract offers a glimpse into the transformative potential of GestureSpeak in revolutionizing the way we interact with computers, making technology more accessible and intuitive for all users. Advancements in human-computer interaction have led to the development of innovative projects that redefine how users interact with technology. This paper introduces a novel integration of two projects: GestureSpeak and VoiceRobot, which together enable intuitive and hands-free control over computing devices using hand gestures and voice commands. VoiceRobot complements GestureSpeak by adding voice command capabilities to the interaction model. Powered by speech recognition technology, VoiceRobot enables users to control devices and launch applications using natural language. With VoiceRobot, initiating the GestureSpeak project is as simple as speaking a command. The combined system offers unique benefits, including improved accessibility for users with mobility impairments and a more natural way to interact with computing devices. By integrating gesture recognition and voice commands, the project enhances user engagement and efficiency, paving the way for future advancements in human-computer interaction. Launching the Project with VoiceRobot:A notable feature of this integrated system is the ability to launch the GestureGenie project using VoiceRobot. By speaking a predefined command, such as "Activate GestureGenie," users can initiate the hand gesture control system effortlessly. This capability highlights the seamless integration of gesture recognition and voice command technologies, showcasing the project's versatility and user-friendly design. In conclusion, the integration of GestureGenie and VoiceRobot represents a significant advancement in human-computer interaction, offering a comprehensive solution for hands-free device control. This abstract provides insight into the combined capabilities of these projects and their potential impact on accessibility, user experience, and the future of interactive computing.

The application of speech recognition to the computer access needs of people with disabilities continues to grow, and a greater understanding of user performance with such systems is needed. This article reviews what is known about user performance with speech recognition systems, with a focus on its application to accommodation of physical disability. Although current systems offer the potential of text entry at 150 words per minute, the literature suggests that users actually achieve somewhere between 8 and 30 words per minute. Barriers that may contribute to this gap, such as the costs associated with correcting recognition errors, are reviewed, and directions for future research are proposed. A major need is for additional research involving users who have physical disabilities.

Auditory models that are capable of achieving human performance in tasks related to speech perception would provide a basis for realizing effective speech processing systems. Saving bits in speech coders, for example, relies on a perceptual tolerance to acoustic deviations from the original speech. Perceptual invariance to adverse signal conditions (noise, microphone and channel distortions, room reverberations) and to phonemic variability (due to nonuniqueness of articulatory gestures) may provide a basis for robust speech recognition. A state-of-the-art auditory model that simulates, in considerable detail, the outer parts of the auditory periphery up through the auditory nerve level is described. Speech information is extracted from the simulated auditory nerve firings, and used in place of the conventional input to several speech coding and recognition systems. The performance of these systems improves as a result of this replacement, but is still short of achieving human performance. The shortcomings occur, in particular, in tasks related to low bit-rate coding and to speech recognition. Since schemes for low bit-rate coding rely on signal manipulations that spread over durations of several tens of ms, and since schemes for speech recognition rely on phonemic/articulatory information that extend over similar time intervals, it is concluded that the shortcomings are due mainly to perceptually related rules over durations of 50-100 ms. These observations suggest a need for a study aimed at understanding how auditory nerve activity is integrated over time intervals of that duration. The author discusses preliminary experimental results that confirm human usage of such integration, with different integration rules for different time-frequency regions depending on the phoneme-discrimination task. >

Purpose :This investigation explored the speech-production behaviors associated with the use of automatic speech recognition (ASR) software for dictation of spontaneous and scripted material by individuals with and without spinal cord injury (SCI). The variables of interest in determining speaking workload and efficiency included syllables per breath group, frequency of breath groups, frequency of apnea, time needed for dictation and number of words spoken during a dictation task. Method : Twelve individuals participated, six with SCI and six able-bodied cohorts matched for age, sex and height. Subjects dictated with continuous-speech ASR, discrete-word ASR and no ASR in a spontaneous-speaking situation, as well as in a scripted speaking situation. Results : For all variables, differences amongst dictation conditions were significant. No significant differences were found between speaker groups. Dictation with both discrete-word and continuous-speech ASR resulted in a decrease in the number of syllables produced per breath group, increases in the frequency of breath groups and apnea, with differences from normal being greater for dictation with discrete-word ASR. In addition, when participants dictated with either type of ASR, the amount of time and number of words produced were significantly greater than that associated with production of the same message without ASR, requiring `more work' on the part of the speaker and ultimately reducing the efficiency with which a message was produced. Conclusions : From a human factors perspective, these results suggest that ASR software, especially discrete-word ASR, has the potential to increase energy expenditure during dictation over a prolonged period of time, thereby increasing speech workloads and the potential for overuse of the laryngeal system.

The purpose of this study was to determine the extent to which automatic speech recognition (ASR) systems and human speech output to ASR systems are affected when users perform complex tasks in a noisy, stressful simulated tank environment. A second objective was to determine whether the type of auditory information input (monaural or spatial) affected user task performance, including the output of speech to ASR systems. Subjects performed simultaneous radio communication and tank driving tasks. Independent variables included auditory display configuration, driving speed, and subject type (soldier or civilian). Dependent variables included 1) the proportion of radio channel changes correctly recognized by the ASR; 2) the proportion and types of speech errors in radio channel changes; 3) the number of accurate responses to the radio messages; and 4) subject ratings of stress and workload. Subjects were six male U.S. Army personnel and six male civilians. Results indicated that ASR and human speech output were not significantly affected by any of the independent variables. A mean of 95.5% of all subject verbal commands was recognized by the ASR system. Spatialized radio messages significantly enhanced the ability of the user to understand speech communications from multiple channels. However, spatial audio information input had no significant effect on user speech output or on ASR performance. In general, ASR and spatial audio displays have potential as technologies of interest in high noise, stressful tank environments in which complex, concurrent tasks are performed.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model