Visual, navigation and communication aid for visually impaired person

The paper proposes a self-driving car model also called autonomous, robotic or driver-less car is one that operates and navigates using its intelligence. The basic idea behind the paper is to develop a 1/10 scale RC car to portray an automated car. The model consists of the following software and hardware components such as CNN (Convolutional neural network), Monocular vision algorithm, Haar cascade classifier, Raspberry Pi Board model B+, Pi camera, Arduino, and an Ultrasonic sensor. The Pi camera and ultrasonic sensor are attached to the raspberry pi board to collect input images along with sensor data to stream these data to the server which in our case is the laptop. The (CNN) convolutional neural network running on the server will be used to enable lane detection to provide steering predictions that are left, right, forward based on the input image. The haar cascade classifier will be used to detect signals and stop sign and monocular vision algorithms to calculate distance from them. The ultrasonic sensor will be used for front collision avoidance by stopping the car at a certain distance before the obstacle ahead. The navigation commands such as right, left, forward, stop will be sent to the car through Arduino which is connected to the RC car's remote, this will make the car drive autonomously based on neural network predictions and some hard coded rules. Thus this model will enable autonomous driving by self-navigation via lane detection, stopping at detection of stop signs, red lights and moving on green signal and front collision avoidance in a cost-effective manner. Hardcoded rules, obstacle avoidance mechanisms, machine learning models, and smart object discrimination will help the system follow traffic rules and navigate the car using artificial intelligence.

Recent research indicates that recognition of interrupted text can predict speech intelligibility under adverse listening conditions. However, factors underlying the relationship between perceptual processing of speech and text are not fully understood. We examined contributions of underlying linguistic and perceptual structure by comparing recognition of printed and spoken sentences interrupted at different rates (0.5—64 Hz) in 14 normal-hearing adults. The interruption method approximated deletion and retention of rate-specific linguistic information across the two modalities by substituting white space for silent intervals. Results indicate a remarkably similar U-shaped pattern of cross-rate variation for both modalities, with minima at 2 Hz. Nevertheless, at high and low interruption rates text recognition exceeded speech recognition, while the reverse trend was observed at middle rates. Surprisingly, no significant correlations were obtained in recognition accuracy between text and speech conditions. These findings indicate a high degree of perceptual constancy in recognition of interrupted text and speech, which may rely on retention of rate-specific linguistic and perceptual information retained after the interruptions. On the other hand, results also indicate rate-specific variation in perceptual processing of text and speech, which may potentially affect the degree to which recognition accuracy in one modality is predictive of the other.

This paper presents a new method for token-by-token sequence labeling that can leverage not only lexical information but also speech information without any alignments. Our motivation is to detect disfluencies such as fillers and word fragments robustly from spontaneous speech. Disfluency detection is often modeled as a token-by-token sequence labeling using a transcribed text via automatic speech recognition. However, utilizing the lexical information alone is not sufficient because the disfluencies cause changes to speech information. One problem is that the speech and the transcribed text need to be aligned when we handle speech and lexical information simultaneously. This prevents introducing speech information to the disfluency detection. To solve this problem, we propose a method for token-by-token sequence labeling, one that can simultaneously use lexical and speech information without requiring any alignments. To this end, we introduce attention mechanisms into a method for neural sequence labeling based on bi-directional long short-term memory recurrent neural network conditional random fields. The attention mechanisms enable us to find the term of disfluencies from speech automatically. Our experimental results show that the proposed method using acoustic and prosodic features improves the labeling accuracy compared with that using lexical features alone.

ABSTRACT: The advancement of Artificial Intelligence (AI) and Natural Language Processing (NLP) has led to the development of voice- controlled systems that enhance Human–Computer Interaction (HCI). This paper presents the design and implementation of a Voice Controlled Desktop Assistant, capable of performing various tasks such as opening applications, searching the web, playing multimedia content, sending messages, and providing real-time system feedback. The assistant uses speech recognition for voice input and text- to-speech synthesis for generating responses. Developed using Python and integrated APIs, it aims to automate desktop functions through intuitive voice commands. The project demonstrates an effective approach for creating user-friendly, hands-free computing environments. Additionally, the system emphasizes lightweight processing to ensure efficient real-time performance without dependency on cloud-based services. It further contributes to the ongoing evolution of intelligent personal assistants designed for offline and secure desktop automation. KEYWORD: Voice Assistant, Artificial Intelligence, Natural Language Processing, Speech Recognition, Automation, Human– Computer Interaction. ABSTRACT: The advancement of Artificial Intelligence (AI) and Natural Language Processing (NLP) has led to the development of voice- controlled systems that enhance Human–Computer Interaction (HCI). This paper presents the design and implementation of a Voice Controlled Desktop Assistant, capable of performing various tasks such as opening applications, searching the web, playing multimedia content, sending messages, and providing real-time system feedback. The assistant uses speech recognition for voice input and text- to-speech synthesis for generating responses. Developed using Python and integrated APIs, it aims to automate desktop functions through intuitive voice commands. The project demonstrates an effective approach for creating user-friendly, hands-free computing environments. Additionally, the system emphasizes lightweight processing to ensure efficient real-time performance without dependency on cloud-based services. It further contributes to the ongoing evolution of intelligent personal assistants designed for offline and secure desktop automation. KEYWORD: Voice Assistant, Artificial Intelligence, Natural Language Processing, Speech Recognition, Automation, Human– Computer Interaction.

Human-computer interaction for virtual-real fusion

VisionMate is a mobile accessibility application that utilizes artificial intelligence (AI) and helps visually impaired and blind people to carry out daily tasks independently. The device proposed integrates computer vision, speech recognition, and GPS to utilize the camera, microphone, and location services on a user's smartphone to assist users with a variety of tasks. VisionMate also provides a multitude of functions for people with visual impairments, including Optical Character Recognition (OCR) for reading text, Object Detection for finding people, currency detection for identifying denomination, Barcode Scanning to read product/food labels, and Voice Navigation for providing direction to users via voice command. Uses of cloud services (Google Vision API & Speech-to-Text API) to store incoming images and voice commands allow for the analysis of received images and voice commands. The image results are returned to the user via real-time sound using Text-to-Speech (TTS) technology. VisionMate will be developed using the React Native framework, and MongoDB will serve as the backend database for user management and authentication. Ultimately the goal of the proposed system is to improve access to and independence for visually impaired users by allowing users to better understand the environment using intelligent audio guidance.

evelopment of OCRs for Indian script is an active area of activity today. Optical character recognition (OCR) is the mechanical or electronic translation of images of handwritten, typewritten or printed text (usually captured by a scanner) into machine-editable text. In simple words OCR is a visual recognition process that turns printed or written text into an electronic character-based file. OCR is a field of research in pattern recognition, artificial intelligence and machine vision. Though academic research in the field continues, the focus on OCR has shifted to implementation of proven techniques. A lot of work had been carried out for OCR at international scenario but in Indian context a concrete approach for character recognition is still required as scripts of Indian languages are from the group of most complex scripts and it is very hard to recognize them. Indian scripts present great challenges to an OCR designer due to the large number of letters in the alphabet, the sophisticated ways in which they combine, and the complicated graphemes they result in. The problem is compounded by the unstructured manner in which popular fonts are designed. There is a lot of common structure in the different Indian scripts. All existing OCR systems developed for various Indian scripts do not provide sufficient efficiency due to various factors. The objective of this paper is to discuss a more efficient character recognition technique. This paper introduces a new technical approach to recognize Indian script characters which are unpredictable due to different problems in other OCR‟s.

Auditory models that are capable of achieving human performance in tasks related to speech perception would provide a basis for realizing effective speech processing systems. Saving bits in speech coders, for example, relies on a perceptual tolerance to acoustic deviations from the original speech. Perceptual invariance to adverse signal conditions (noise, microphone and channel distortions, room reverberations) and to phonemic variability (due to nonuniqueness of articulatory gestures) may provide a basis for robust speech recognition. A state-of-the-art auditory model that simulates, in considerable detail, the outer parts of the auditory periphery up through the auditory nerve level is described. Speech information is extracted from the simulated auditory nerve firings, and used in place of the conventional input to several speech coding and recognition systems. The performance of these systems improves as a result of this replacement, but is still short of achieving human performance. The shortcomings occur, in particular, in tasks related to low bit-rate coding and to speech recognition. Since schemes for low bit-rate coding rely on signal manipulations that spread over durations of several tens of ms, and since schemes for speech recognition rely on phonemic/articulatory information that extend over similar time intervals, it is concluded that the shortcomings are due mainly to perceptually related rules over durations of 50-100 ms. These observations suggest a need for a study aimed at understanding how auditory nerve activity is integrated over time intervals of that duration. The author discusses preliminary experimental results that confirm human usage of such integration, with different integration rules for different time-frequency regions depending on the phoneme-discrimination task. >

Deep learning is one machine learning method based on the layers used in artificial neural networks. The breakthrough of deep learning in classification led to its rapid application in speech recognition, natural language understanding, and image processing and recognition. In the field of biodiversity informatics, deep learning efforts are being applied in rapid species identification and counts of individuals identified based on image, audio, video, and other data types. However, deep learning methods hold great potential for application in all aspects of biodiversity informatics. We present a case study illustrating how to increase data collection quality and efficiency using well-established technology such as optical character recognition (OCR) and some image classification. Our goal is to image data from the scanned documents of various butterfly atlases, add species, specimens, collections, photographs and other relevant information, and build a database of butterfly images. Information collection involves image annotation and text-based descritpion input. Although the work of image annotation is simple, this process can be accelerated by deep learning-based target segmentation to make the selection process easier, such as changing box select to a double click. The process of information collection is complicated, involving input of species names, specimen collection, specimen description, and other information. Generally, there are many images in atlases, the text layout is rather messy, and overall OCR success is poor. Therefore, the measures we take are as follows: Step A: select the screenshot of the text and then call the OCR interface to generate the text material; Step B: proceed with NLP- (natural language processing) related processing; Step C: perform manual operations on the results, and introduce the NLP function again to this process; Step D: submit the result. The deep learning applications we integrated in our client tool include: target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. target segmentation of the annotated image for automatic positioning and background removal, etc. to improve the quality of the image used for identification; making a preliminary judgment on various attributes of the labeled image and using the results to assist the automatic filling of relevant information in step B, including species information, specimen attributes (specimen image, nature photo, hand drawing pictures, etc.), insect stage (egg, adult, etc.); OCR in step A. Some simple machine learning methods such as k-nearest neighbor can be used to automatically determine gender, pose, and so on. While complex information such as collection place, time, and collector can be analyzed by deep learning-based NLP methods in the future. In our infomation collection process, ten fields are required to submit one single record. Of those, about 4-5 input fields can be dealt with the AI-assistant. It can thus be seen from the above process that deep learning has reduced the workload of manual information annotation by at least 30%. With improvements in accuracy, the era of using automatic information extraction robots to replace manual information annotation and collection is just around the corner.

In today’s digital environment, the pattern is everywhere. It is present in many aspects of our daily life. Algorithms can be used to detect or mathematically observe a pattern physically. Vector feature values represent a pattern in the digital world. With the advent of artificial intelligence (AI) techniques in the recent era, there are so many machine learning (ML) and deep learning (DL) models that have been developed. ML is the branch of AI, which can perform tasks like data analysis, analytical model building, and classification without being explicitly programmed. DL is the subset of ML in AI. In DL, mostly artificial neural networks (ANNs) are utilized. It has the capability of work based upon unsupervised learning from the data that is unlabeled and unstructured. Using these DL and ML models, extract the meaningful features from the given image or video is known as pattern recognition (PR). PR is used in many engineering applications such as computer vision, natural language processing, character recognition, robotics, speech recognition, and so on. It is also used in many medical image processing applications and telemedicine. The present work discussed the pattern recognition problem and its various stages in detail. In addition to that, the application of deep learning and machine learning in pattern recognition is also explained briefly.

Because neural networks specialize in handling ambiguous data, they are especially suited for such applications as speech recognition and optical character recognition (OCR). OCR applications are usually ambiguous because their data is generated by an inconsistent factor—the individual. This article provides an overview of neural networks and describes how this technology can be integrated with OCR technology to create neural OCR networks that can significantly improve the process of optical character recognition.

In pure-tone audiometry, hearing thresholds are typically measured up to 8 kHz. Recent research has shown that extended high-frequency (EHF; frequencies >8 kHz) speech information improves speech recognition. However, it is unclear whether the EHF benefit is present for different types of speech material. This study assesses the added value of EHF information for speech recognition in noise for digit triplets, consonant-vowel-consonant (CVC) words, and sentences; and for speech recognition in quiet for CVC. Twenty-four young adults with normal-hearing thresholds up to 16 kHz performed a listening experiment in quiet and in noise in a within-subject repeated measures design. Stimuli were presented monaurally. Steady state speech-shaped noise at a fixed signal to noise ratio was used for measurements in noise. Listening conditions varied only in terms of available EHF information. Stimuli were presented in three different conditions: (1) both speech and noise broadband, (2) speech broadband and noise low-pass filtered at 8 kHz, and (3) both speech and noise low-pass filtered at 8 kHz. In the speech-in-quiet experiment, stimuli (CVC) were high-pass filtered at 3 kHz and presented in two conditions: (1) with EHF information and (2) without EHF information. In the speech-in-noise experiment, for all speech material, the highest scores were achieved in the condition where the noise was low-pass filtered at 8 kHz and speech unfiltered; the lowest scores were obtained in the condition where both speech and noise were low-pass filtered at 8 kHz. Adding speech frequencies above 8 kHz improved the median recognition scores by 75.0%, 21.8%, and 23.8% for digit triplets, words, and sentences, respectively, at a fixed signal to noise ratio. In the speech-in-quiet experiment, median recognition scores were 7.8% higher in the condition where the EHF information was available, as opposed to when it was not. Speech information for frequencies above 8 kHz contributes to speech recognition in noise. It also contributes to speech recognition in quiet when information below 3 kHz is absent. Our results suggest that EHFs may be relevant in challenging listening conditions and should be measured in pure-tone audiometry to get a complete picture of a person's hearing. Further, results of speech recognition tests may vary when different recording and/or measurement equipment is used with different frequency responses above 8 kHz.

The field of automatic speech recognition (ASR) has enjoyed more than 30 years of technology advances due to the extensive utilization of the hidden Markov model (HMM) framework and a concentrated effort by the speech community to make available a vast amount of speech and language resources, known today as the Big Data Paradigm. State-of-the-art ASR systems achieve a high recognition accuracy for well-formed utterances of a variety of languages by decoding speech into the most likely sequence of words among all possible sentences represented by a finite-state network (FSN) approximation of all the knowledge sources required by the ASR task. However, the ASR problem is still far from being solved because not all information available in the speech knowledge hierarchy can be directly integrated into the FSN to improve the ASR performance and enhance system robustness. It is believed that some of the current issues of integrating various knowledge sources in top-down integrated search can be partially addressed by processing techniques that take advantage of the full set of acoustic and language information in speech. It has long been postulated that human speech recognition (HSR) determines the linguistic identity of a sound based on detected evidence that exists at various levels of the speech knowledge hierarchy, ranging from acoustic phonetics to syntax and semantics. This calls for a bottom-up attribute detection and knowledge integration framework that links speech processing with information extraction, by spotting speech cues with a bank of attribute detectors, weighting and combining acoustic evidence to form cognitive hypotheses, and verifying these theories until a consistent recognition decision can be reached. The recently proposed automatic speech attribute transcription (ASAT) framework is an attempt to mimic some HSR capabilities with asynchronous speech event detection followed by bottom-up knowledge integration and verification. In the last few years, ASAT has demonstrated good potential and has been applied to a variety of existing applications in speech processing and information extraction.

: This report describes the function, operation, test and evaluation of a Neural Network that accomplishes unsupervised learning of binary input patterns by classifying them using Adaptive Resonance Theory. Keywords: Machine learning; Artificial intelligence; Speech recognition; Character recognition; Pattern recognition; Acoustic differentiation; Detection; Adaptive filters; Pattern classification; Unsupervised learning; Neural network.

This study explores the perspectives of Indian library professionals on the use of artificial intelligence (AI) in libraries. It aims to understand their knowledge, awareness, and views on AI and its challenges and opportunities. The research adopts a quantitative approach, using a closed-ended survey to collect data from 386 library professionals in academic institutions across India. The survey covers various aspects, including socio-demographic information, AI knowledge, perspectives on AI in libraries, ethical considerations, and adopting AI tools and services. The findings indicate that Indian library professionals are generally aware of AI and its potential benefits in libraries. They believe that AI can enhance library activities, improve accessibility, and support decision-making. However, there are concerns about AI replacing human intelligence within libraries. Factors such as user privacy, funding, staff expertise, and alignment with institutional goals are essential when implementing AI applications. Ethical considerations, including bias and discrimination, intellectual freedom, and transparency, are also significant concerns. AI tools like smart shelving and optical character recognition are among the technologies already adopted in Indian libraries.