Sign Language Recognition and Translation into Pashto Language Alphabets

The development of Human-Computer Interaction (HCI) will not only empower the disabled with access to technology but also help to build tools for assisting the disable and their caregivers. Some disabled people are deaf, they use signs to communicate. Signs are emblematic forms which convey a specific meaning. These emblems construct the sign language, which is a non-verbal language used by the deaf community. This research attempts to build a Bangla Sign language recognition system that can recognize signs of both hands. Hands vary in shapes and sizes, as well as signs vary in orientations and motions. Accurate feature extraction is necessary for such systems. For such purposes, deep learning approaches can prove to be effective for the classification and feature engineering of images. In the beginning, an integrated CNN-LSTM model is proposed for building a Sign language Recognition System, a Bangla Sign Language (BSL) dataset consisting of Bangla lexical signs is considered. This dataset consists of 13,400 images comprising thirty-six classes of Bangla lexical signs. The model is trained using a CNN-LSTM model. This model produces a training accuracy of 90% and a testing accuracy of 88.5%. The proposed model is compared to CNN and other CNN state of the art models, namely VGG16, VGG9 and MobileNet. The CNN model and other CNN state undergo the problem of overfitting as their training accuracy is greater than the testing accuracy.

A way of communication on an equal footing with oral and written speech is sign language. Oral speech is a common way of communication, written speech has, with the development of information technologies, been taking up more and more space. In a parallel with them, communication can also take place through sign language, which is, to deaf people, as well as to all those who use it, a natural, simple and easy way to communicate. Relationship towards sign language has changed significantly throughout history from acceptance and isolated use, to complete rejection, and then to encouraging its adoption and emphasizing its importance for the cognitive, emotional, educational, social, and general development of deaf children. Serbian Sign Language (SSL) serves deaf people in Serbia as a means for everyday communication, for expressing desires, willingness, for learning, for intellectual discussions, for expressing personal style. Although the standardization of the Serbian Sign Language was completed in 2015, even nowadays we may still find certain gestures of expression varying in different regions. Different countries have different sign languages that are not reciprocally understood in use. They are distinguished by their own grammar (semantics, morphology, and syntax), different from the grammar of spoken languages. The distinguishing and recognition of sign languages in the world has led to changes in the field of education of deaf children. In bilingual schools, children acquire both sign and spoken languages, and teachers know both of the mentioned languages. The importance of sign language in the education of deaf children is emphasized.

A major challenge faced by our society is the difficulty individuals with disabilities encounter in expressing their emotions to others. People with disabilities often depend on sign (gesture) languages for communication. This project aims to develop a model capable of recognizing and translating sign language alphabets from hand gestures into text and audio. The primary objective is to enhance communication between individuals with hearing impairments and the broader society. We assessed our model's performance using a publicly accessible Indian Sign Language (ISL) dataset. For image classification, we employed Convolutional Neural Networks (CNNs) with the Inception V3 architecture. Hand gestures are captured through webcams, and our model identifies the corresponding alphabet for each gesture. This project seeks to overcome current challenges in Sign Language Recognition and aims to enhance its effectiveness and efficiency.

Communication is essential to humans because it allows the dissemination of knowledge and the formation of interpersonal connections. We communicate through speaking, facial expressions, hand gestures, reading, writing, and sketching, among other things. However, speaking is the most often utilized means of communication. People having speech and hearing disabilities can only communicate using hand gestures, making them extremely reliant on nonverbal modes of communication. Hearing-impaired persons can communicate via sign language. Globally, around 1 percent(5 million) of the Indian population falls into this group. ISL is a complete language with its own vocabulary, semantics, lexicon, and a variety of other distinctive linguistic features. In our work, we present the methods for Indian sign language recognition at the character and word levels. The Bag of Visual Words(BoVW) technique recognizes ISL at character level(A-Z, 0-9) with an accuracy of 99 percent. Indian Lexicon Sign Language Dataset - INCLUDE-50 dataset is used for word-level sign language recognition. Inception model, a deep Convolutional Neural Network(CNN) is used to train the spatial features and LSTM RNN(Recurrent Neural Network) is used to train the temporal features of the video. Using CNN predictions as input to RNN, we achieved an accuracy of 86.7 %. In order to optimize the training process, only 60 % of the dataset is trained using the Meta-Learning model along with LSTM RNN and obtained an accuracy of 84.4 %, thus reducing the training time by 70 % and reaching nearly as close accuracy as the previous pre-trained model.

Speech is the major way of human communication, but when it is limited, humans move to tactile kinaesthetic communication. People with speech-hearing impairments use sign language as an example of such adaptations. The deaf community uses Indian sign language (ISL) throughout India. In India, 250 licensed sign language interpreters are serving a deaf population of 1.8 to 7 million individuals. ISL interpreters are badly needed at institutes and places where persons with hearing impairments communicate. An Indian sign language picture database for English alphabets is established in this project. To prepare it for training, several pre-processing techniques were used. The effectiveness of deep learning neural networks is frequently influenced by the quantity of data available. As a result, data augmentation, a strategy for adding more and diverse samples to train datasets, was used to boost the effectiveness and outcomes of machine learning models. Our model is trained in CNN models utilizing transfer learning methodologies, with an accuracy of 95% for vgg16 and an accuracy of 92% for the inception model. More study on this research, as well as real-time implementation, has the potential to better connect people with hearing loss to society.

IntroductionEarly linguistic background, and in particular, access to language, lays the foundation of future reading skills in deaf and hard-of-hearing signers. The current study aims to estimate the impact of two factors – early access to sign and/or spoken language – on reading fluency in deaf and hard-of-hearing adult Russian Sign Language speakers.MethodsIn the eye-tracking experiment, 26 deaf and 14 hard-of-hearing native Russian Sign Language speakers read 144 sentences from the Russian Sentence Corpus. Analysis of global eye-movement trajectories (scanpaths) was used to identify clusters of typical reading trajectories. The role of early access to sign and spoken language as well as vocabulary size as predictors of the more fluent reading pattern was tested.ResultsHard-of-hearing signers with early access to sign language read more fluently than those who were exposed to sign language later in life or deaf signers without access to speech sounds. No association between early access to spoken language and reading fluency was found.DiscussionOur results suggest a unique advantage for the hard-of-hearing individuals from having early access to both sign and spoken language and support the existing claims that early exposure to sign language is beneficial not only for deaf but also for hard-of-hearing children.

Sign Language is a way of communication used by hearing-impaired people. As many normal people don’t know the Sign Language it is difficult to understand by them to understand thus, increasing the communication gap between normal and hearing-impaired people. To overcome this problem many systems,using Machine Learning(ML), Artificial Intelligence(AI), Deep Learning(DL), computer vision methods, were developed and being developed. These systems are used to translate Indian Sign Language(ISL) to understandable text, facilitating the communication between normal and hearing-impaired people. This study briefly explains various type of Indian Sign Language(ISL) recognition systems.

IsharaNet: A robust nested feature fusion coupled with attention incorporated width scaled lightweight architecture for Bengali sign language recognition

Sign language is a universal way of communication for challenged people with speaking and hearing limitations. Multiple mediums are accessible to translate or to acknowledge sign language and convert them to text. However, the text to signing conversion systems has been rarely developed; this is often thanks to the scarcity of any sign language dictionary. Our project aims at creating a system that consists of a module that initially transforms voice input to English text and which parses the sentence, then to which Indian sign language grammar rules are applied. This is done by eliminating stop words from the reordered sentence. Indian Sign Language (ISL) does not sustain the inflections of the word. Hence, stemming is applied to vary over the words to their root/ stem class. All words of the sentence are then checked against the labels in the dictionary containing videos representing each of the words. The present systems are limited to only straight conversion of words into ISL, whereas the proposed system is innovative, as our system aims to rework these sentences into ISL as per grammar in the real domain.

Sign Language Recognition (SLR) targets on interpreting the sign language into text or speech, so as to facilitate the communication between deaf-mute people and ordinary people. The task has broad social impact, but is still very challenging due to the complexity and large variations in hand actions. Existing dataset for Sign Language Recognition (SLR) in Bangla Sign Language (BdSL) is based on RGB images. Recent research on sign language recognition has shown better recognition accuracy using depth-based features. In this paper, we present a complete dataset for Bangla sign digits from Zero (Shunno in Bangla) to Nine (Noy in Bangla) using MediaPipe, a cross-platform depth-map estimation framework. The proposed method can utilize hand skeleton joint points containing depth information in addition to x, y coordinates from RGB images only. To validate the effectiveness of our proposed approach, we have run MediaPipe on a benchmark American Sign Language (ASL) dataset. Running different classifiers in our proposed dataset we got 98.65% using Support Vector Machine (SVM). Moreover, we compared our dataset with the existing Bangla digit dataset Ishara Bochon using deep learning based approach and achieved significantly higher accuracy.

<p>Sign language is a medium of communication for people with hearing and speaking impairment. It uses gestures to convey messages. The proposed system focuses on using sign language in search engines and helping specially-abled people get the information they are looking for. Here, we are using Marathi sign language. Translation systems for Indian sign languages are not much simple and popular as American sign language. Marathi language consists of words with individual letters formed of two letter = Swara + Vyanjan (Mulakshar). Every Vyanjan or Swara individually has a unique sign which can be represented as image or video with still frames. Any letter formed of both Swara and Vyanjan is represented with hand gesture signing the Vyanjan as above and with movement of signed gesture in shape of Swara in Devnagari script. Such letters are represented with videos containing motion and frames in particular sequence. Further the predicted term can be searched on google using the sign search. The proposed system includes three important steps: 1) hand detection; 2) sign recognition using neural networks; 3) fetching search results. Overall, the system has great potential to help individuals with hearing and speaking impairment to access information on the internet through the use of sign language. It is a promising application of machine learning and deep learning techniques.</p>

The Deaf are a large social group in society. Their unique way of communicating through sign language is often confined within their community due to limited understanding by individuals outside of this demographic. This is where sign language recognition (SLR) comes in to help normal people understand the meaning of sign language. In recent years, new methods of sign language recognition have been developed and achieved good results, so it is necessary to make a summary. This review mainly focuses on the introduction of sign language recognition techniques based on algorithms especially in recent years, including the recognition models based on traditional methods and deep learning approaches, sign language datasets, challenges and future directions in SLR. To make the method structure clearer, this article explains and compares the basic principles of different methods from the perspectives of feature extraction and temporal modelling. We hope that this review will provide some reference and help for future research in sign language recognition.

A lightweight network-based sign language robot with facial mirroring and speech system

This paper aims to shed light on the common discourse markers in Sign Language, particularly Pakistani Sign Language. Pakistani Sign Language is overlooked and emerges as a non-standardized. These gestures/ discourse markers will be studied in comparison to American Sign Language (ASL). The main features will be analysed that call for a detailed analysis based on the use, the universality and application. This research aims to study if there is any influence of gender on these discourse markers. The Pakistan Sign Language Corpus will be collected and analysed with the assistance of Sign Language Interpreters. Furthermore, to see for any consistent language specific functions that these languages may share will also be accounted and noted. Keywords: Sign Language, Discourse markers, gender, American Sign Language, Pakistani Sign Language

This article offers a brief overview of historical linguistics and explores the value of historical sign linguistics. The specific focus of the article is on the question of the extent to which the concept of "sign language families" is a legitimate and useful one. It is suggested that although lateral transmission and interference may account for a great deal of the similarity between and among sign languages, descent from a common linguistic ancestor (as in the cases of American Sign Language and French Sign Language) can also often be documented. It is observed that sign languages seem to change at a much more rapid rate than do spoken languages, but this is suggested to have at least as much to do with factors such as the intragenerational transmission of most sign languages as any other variable. Finally, it is argued that regardless of its status in historical linguistics more generally, lexicostatistics has consistently been demonstrated to be a powerful methodological tool in understanding the relationships between and among different sign languages.