Vision Mate: An AI Based Mobile Assistive System for Visually Impaired Users

Recent research indicates that recognition of interrupted text can predict speech intelligibility under adverse listening conditions. However, factors underlying the relationship between perceptual processing of speech and text are not fully understood. We examined contributions of underlying linguistic and perceptual structure by comparing recognition of printed and spoken sentences interrupted at different rates (0.5—64 Hz) in 14 normal-hearing adults. The interruption method approximated deletion and retention of rate-specific linguistic information across the two modalities by substituting white space for silent intervals. Results indicate a remarkably similar U-shaped pattern of cross-rate variation for both modalities, with minima at 2 Hz. Nevertheless, at high and low interruption rates text recognition exceeded speech recognition, while the reverse trend was observed at middle rates. Surprisingly, no significant correlations were obtained in recognition accuracy between text and speech conditions. These findings indicate a high degree of perceptual constancy in recognition of interrupted text and speech, which may rely on retention of rate-specific linguistic and perceptual information retained after the interruptions. On the other hand, results also indicate rate-specific variation in perceptual processing of text and speech, which may potentially affect the degree to which recognition accuracy in one modality is predictive of the other.

ABSTRACT: The advancement of Artificial Intelligence (AI) and Natural Language Processing (NLP) has led to the development of voice- controlled systems that enhance Human–Computer Interaction (HCI). This paper presents the design and implementation of a Voice Controlled Desktop Assistant, capable of performing various tasks such as opening applications, searching the web, playing multimedia content, sending messages, and providing real-time system feedback. The assistant uses speech recognition for voice input and text- to-speech synthesis for generating responses. Developed using Python and integrated APIs, it aims to automate desktop functions through intuitive voice commands. The project demonstrates an effective approach for creating user-friendly, hands-free computing environments. Additionally, the system emphasizes lightweight processing to ensure efficient real-time performance without dependency on cloud-based services. It further contributes to the ongoing evolution of intelligent personal assistants designed for offline and secure desktop automation. KEYWORD: Voice Assistant, Artificial Intelligence, Natural Language Processing, Speech Recognition, Automation, Human– Computer Interaction. ABSTRACT: The advancement of Artificial Intelligence (AI) and Natural Language Processing (NLP) has led to the development of voice- controlled systems that enhance Human–Computer Interaction (HCI). This paper presents the design and implementation of a Voice Controlled Desktop Assistant, capable of performing various tasks such as opening applications, searching the web, playing multimedia content, sending messages, and providing real-time system feedback. The assistant uses speech recognition for voice input and text- to-speech synthesis for generating responses. Developed using Python and integrated APIs, it aims to automate desktop functions through intuitive voice commands. The project demonstrates an effective approach for creating user-friendly, hands-free computing environments. Additionally, the system emphasizes lightweight processing to ensure efficient real-time performance without dependency on cloud-based services. It further contributes to the ongoing evolution of intelligent personal assistants designed for offline and secure desktop automation. KEYWORD: Voice Assistant, Artificial Intelligence, Natural Language Processing, Speech Recognition, Automation, Human– Computer Interaction.

BackgroundRecent advances in natural language processing and artificial intelligence have led to widespread adoption of speech recognition technologies. In consumer health applications, speech recognition is usually applied to support interactions with conversational agents for data collection, decision support, and patient monitoring. However, little is known about the use of speech recognition in consumer health applications and few studies have evaluated the efficacy of conversational agents in the hands of consumers. In other consumer-facing tools, cognitive load has been observed to be an important factor affecting the use of speech recognition technologies in tasks involving problem solving and recall. Users find it more difficult to think and speak at the same time when compared to typing, pointing, and clicking. However, the effects of speech recognition on cognitive load when performing health tasks has not yet been explored.ObjectiveThe aim of this study was to evaluate the use of speech recognition for documentation in consumer digital health tasks involving problem solving and recall.MethodsFifty university staff and students were recruited to undertake four documentation tasks with a simulated conversational agent in a computer laboratory. The tasks varied in complexity determined by the amount of problem solving and recall required (simple and complex) and the input modality (speech recognition vs keyboard and mouse). Cognitive load, task completion time, error rate, and usability were measured.ResultsCompared to using a keyboard and mouse, speech recognition significantly increased the cognitive load for complex tasks (Z=–4.08, P<.001) and simple tasks (Z=–2.24, P=.03). Complex tasks took significantly longer to complete (Z=–2.52, P=.01) and speech recognition was found to be overall less usable than a keyboard and mouse (Z=–3.30, P=.001). However, there was no effect on errors.ConclusionsUse of a keyboard and mouse was preferable to speech recognition for complex tasks involving problem solving and recall. Further studies using a broader variety of consumer digital health tasks of varying complexity are needed to investigate the contexts in which use of speech recognition is most appropriate. The effects of cognitive load on task performance and its significance also need to be investigated.

The loss of vision restrained the visually impaired people from performing their daily task. This issue has impeded their free-movement and turned them into dependent a person. People in this sector did not face technologies revamping their situations. With the advent of computer vision, artificial intelligence, the situation improved to a great extent. The propounded design is an implementation of a wearable device which is capable of performing a lot of features. It is employed to provide visual instinct by recognizing objects, identifying the face of choices. The device runs a pre-trained model to classify common objects from household items to automobiles items. Optical character recognition and Google translate were executed to read any text from image and convert speech of the user to text respectively. Besides, the user can search for an interesting topic by the command in the form of speech. Additionally, ultrasonic sensors were kept fixed at three positions to sense the obstacle during navigation. The display attached help in communication with deaf person and GPS and GSM module aid in tracing the user. All these features run by voice commands which are passed through the microphone of any earphone. The visual input is received through the camera and the computation task is processed in the raspberry pi board. However, the device seemed to be effective during the test and validation.

Deep learning is one machine learning method based on the layers used in artificial neural networks. The breakthrough of deep learning in classification led to its rapid application in speech recognition, natural language understanding, and image processing and recognition. In the field of biodiversity informatics, deep learning efforts are being applied in rapid species identification and counts of individuals identified based on image, audio, video, and other data types. However, deep learning methods hold great potential for application in all aspects of biodiversity informatics. We present a case study illustrating how to increase data collection quality and efficiency using well-established technology such as optical character recognition (OCR) and some image classification. Our goal is to image data from the scanned documents of various butterfly atlases, add species, specimens, collections, photographs and other relevant information, and build a database of butterfly images. Information collection involves image annotation and text-based descritpion input. Although the work of image annotation is simple, this process can be accelerated by deep learning-based target segmentation to make the selection process easier, such as changing box select to a double click. The process of information collection is complicated, involving input of species names, specimen collection, specimen description, and other information. Generally, there are many images in atlases, the text layout is rather messy, and overall OCR success is poor. Therefore, the measures we take are as follows: Step A: select the screenshot of the text and then call the OCR interface to generate the text material; Step B: proceed with NLP- (natural language processing) related processing; Step C: perform manual operations on the results, and introduce the NLP function again to this process; Step D: submit the result. The deep learning applications we integrated in our client tool include: target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. Some simple machine learning methods such as k-nearest neighbor can be used to automatically determine gender, pose, and so on. While complex information such as collection place, time, and collector can be analyzed by deep learning-based NLP methods in the future. In our infomation collection process, ten fields are required to submit one single record. Of those, about 4-5 input fields can be dealt with the AI-assistant. It can thus be seen from the above process that deep learning has reduced the workload of manual information annotation by at least 30%. With improvements in accuracy, the era of using automatic information extraction robots to replace manual information annotation and collection is just around the corner.

In today’s digital environment, the pattern is everywhere. It is present in many aspects of our daily life. Algorithms can be used to detect or mathematically observe a pattern physically. Vector feature values represent a pattern in the digital world. With the advent of artificial intelligence (AI) techniques in the recent era, there are so many machine learning (ML) and deep learning (DL) models that have been developed. ML is the branch of AI, which can perform tasks like data analysis, analytical model building, and classification without being explicitly programmed. DL is the subset of ML in AI. In DL, mostly artificial neural networks (ANNs) are utilized. It has the capability of work based upon unsupervised learning from the data that is unlabeled and unstructured. Using these DL and ML models, extract the meaningful features from the given image or video is known as pattern recognition (PR). PR is used in many engineering applications such as computer vision, natural language processing, character recognition, robotics, speech recognition, and so on. It is also used in many medical image processing applications and telemedicine. The present work discussed the pattern recognition problem and its various stages in detail. In addition to that, the application of deep learning and machine learning in pattern recognition is also explained briefly.

In today's digital age, mobile apps have transformed our daily lives, but for people with visual impairments, these apps often present accessibility challenges. This research addresses the need for improved solutions by focusing on "voice-guided assistance" for Android smartphones. The existing options for visually impaired users are fragmented, and this study aims to create an integrated mobile application, "Voice Companion," designed to enhance their digital experience. "Voice-Companion" is a specialized Android application designed for visually impaired individuals, developed in Java within the Android Studio environment. It leverages the Android OS to provide comprehensive non-visual access. With features like voice commands, object detection, messaging, a voice-activated calculator, location and time announcements, text-to-speech capabilities, and battery status updates, the user-friendly app bridges accessibility gaps, empowering visually impaired users for equal participation in the digital realm. Its modular, user-centric design emphasizes accessibility through seamless integration of voice commands and gesture recognition, facilitating efficient interactions with mobile devices. This research aims to create "Voice-Companion," enhancing smartphone accessibility through seamless voice command integration and gesture recognition. Anticipated results include improved digital engagement and quality of life for users with visual impairments.

We describe a Chinese-Portuguese translation service, which is integrated in an Android application. The application is also enhanced with technologies such as Automatic Speech Recognition, Optical Character Recognition, Image Retrieval, and Language Detection. This mobile translation application, which is deployed on a portable device, relies by default on a server-based machine translation service, which is not accessible when no Internet connection is available. For providing translation support under this condition, we have developed a contextualized off-line search engine that allows the users to continue using the application. The system includes a search engine that is used to support our Chinese-Portuguese machine translation services when no Internet connection is available.

We have constructed a new speech data corpus consisting of the utterances of 221 elderly Japanese people (average age: 79.2) with the aim of improving the accuracy of automatic speech recognition (ASR) for the elderly. ASR is a beneficial modality for people with impaired vision or limited hand movement, including the elderly. However, speech recognition systems using standard recognition models, especially acoustic models, have been unable to achieve satisfactory performance for the elderly. Thus, creating more accurate acoustic models of the speech of elderly users is essential for improving speech recognition for the elderly. Using our new corpus, which includes the speech of elderly people living in three regions of Japan, we conducted speech recognition experiments using a variety of DNN-HNN acoustic models. As training data for our acoustic models, we examined whether a standard adult Japanese speech corpus (JNAS), an elderly speech corpus (S-JNAS) or a spontaneous speech corpus (CSJ) was most suitable, and whether or not adaptation to the dialect of each region improved recognition results. We adapted each of our three acoustic models to all of our speech data, and then re-adapt them using speech from each region. Without adaptation, the best recognition results were obtained when using the S-JNAS trained acoustic models (total corpus: 21.85% Word Error Rate). However, after adaptation of our acoustic models to our entire corpus, the CSJ trained models achieved the lowest WERs (entire corpus: 17.42%). Moreover, after readaptation to each regional dialect, the CSJ trained acoustic model with adaptation to regional speech data showed tendencies of improved recognition rates. We plan to collect more utterances from all over Japan, so that our corpus can be used as a key resource for elderly speech recognition in Japanese. We also hope to achieve further improvement in recognition performance for elderly speech.

Because neural networks specialize in handling ambiguous data, they are especially suited for such applications as speech recognition and optical character recognition (OCR). OCR applications are usually ambiguous because their data is generated by an inconsistent factor—the individual. This article provides an overview of neural networks and describes how this technology can be integrated with OCR technology to create neural OCR networks that can significantly improve the process of optical character recognition.

With the increased popularity of social media, editing and sharing selfies using augmented reality filters, face editors, and sticker features have become popular social trends. However, it can be challenging for people with visual impairments to edit and add fun elements to their selfies, although they actively participate in social media. We conducted an online survey in which 47 participants with visual impairments identified their experience with and demands for using such features. Based on the results, we designed and developed a selfie editing application with sticker features based on voice command and voice feedback for people with visual impairments. We then conducted a design probe study with four participants who were visually impaired to provide design guidelines to increase the accessibility of selfie editing apps with sticker features. Voice command and feedback were both highly appreciated by participants, and we also investigated their requirements for selfie editing features.

Artificial Intelligence (AI) is a computer system that simulates intelligent human behavior. The use of AI is rapidly shifting Healthcare. Speech recognition (SR) is a type of AI physicians use to operate Electronic Health records (EHR). This paper aims to demonstrate the technological advancements made thus far concerning speech recognition in health care and explore multiple scholarly studies to generate a wide-ranging and detailed assessment of its current progress. The effectiveness of speech recognition is the heart of this analysis. This review investigates published papers on the progress and effectiveness of speech recognition in Healthcare. Eight research papers exploring the progress and effectiveness of speech recognition in Healthcare were thoroughly reviewed. Articles were identified from Google Scholar, PubMed, and the World Wide Web. The five relevant papers generally discussed the progress and current effectiveness of SR in Healthcare, implementing SR in the EHR, adapting healthcare workers to SR and the problems they face, developing an intelligent healthcare system based on SR and using SR systems in other languages. Conclusion: This report demonstrates the technological improvements realized concerning SR in Healthcare. It proved that SR could be a tremendous help to providers if every medical and health institution continued to progress in using this technology.

: This report describes the function, operation, test and evaluation of a Neural Network that accomplishes unsupervised learning of binary input patterns by classifying them using Adaptive Resonance Theory. Keywords: Machine learning; Artificial intelligence; Speech recognition; Character recognition; Pattern recognition; Acoustic differentiation; Detection; Adaptive filters; Pattern classification; Unsupervised learning; Neural network.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

evelopment of OCRs for Indian script is an active area of activity today. Optical character recognition (OCR) is the mechanical or electronic translation of images of handwritten, typewritten or printed text (usually captured by a scanner) into machine-editable text. In simple words OCR is a visual recognition process that turns printed or written text into an electronic character-based file. OCR is a field of research in pattern recognition, artificial intelligence and machine vision. Though academic research in the field continues, the focus on OCR has shifted to implementation of proven techniques. A lot of work had been carried out for OCR at international scenario but in Indian context a concrete approach for character recognition is still required as scripts of Indian languages are from the group of most complex scripts and it is very hard to recognize them. Indian scripts present great challenges to an OCR designer due to the large number of letters in the alphabet, the sophisticated ways in which they combine, and the complicated graphemes they result in. The problem is compounded by the unstructured manner in which popular fonts are designed. There is a lot of common structure in the different Indian scripts. All existing OCR systems developed for various Indian scripts do not provide sufficient efficiency due to various factors. The objective of this paper is to discuss a more efficient character recognition technique. This paper introduces a new technical approach to recognize Indian script characters which are unpredictable due to different problems in other OCR‟s.