Moment Based Sign Language Recognition for Indian Languages

Sign language is the communication language used by people with disabilities, especially deaf and hearing-impaired people. The communication between normal and disabled people using sign language will help them carry out their daily activities. Unfortunately, normal people are not aware of the importance of sign language because they are not directly dealing with disabled persons. Besides, some normal people found that sign language is difficult to learn. This study focuses on developing a mobile application for sign language with augmented reality features. This application is targeted at normal people as a sign language learning tool, and augmented reality will make the learning process effective and exciting. The methodology for this study is the ADDIE model that consists of five phases, namely Analysis, Design, Development, Implementation, and Evaluation. This application will assist the users in learning sign language interactively and interestingly. The evaluation of the application was done by the expert in the related background and the normal people. The usability test result revealed that the sign language application is usable for normal people to know and learn the basic sign language. In conclusion, the sign language application is an interesting application for normal people to use and learn sign language.

<p> Communication between normal and handicapped person such as deaf people, dumb people, and blind people has always been a challenging task. It has been observed that they find it really difficult at times to interact with normal people with their gestures, as only a very few of those are recognized by most people. Since people with hearing impairment or deaf people cannot talk like normal people so they have to depend on some sort of visual communication in most of the time. Sign Language is the primary means of communication in the deaf and dumb community. As like any other language it has also got grammar and vocabulary but uses visual modality for exchanging information. The importance of sign language is emphasized by the growing public approval and funds for international project. Interesting technologies are being developed for speech recognition but no real commercial product for sign recognition is actually there in the current market. So, to take this field of research to another higher level this project was studied and carried out. The basic objective of this research is to develop MATLAB based real time system for hand gesture recognition which recognize hand gestures, features of hands such as centroid, peak calculation, angle calculation and convert gesture images into voice and vice versa. To implement this system we used simple night vision web-cam with 20 megapixel intensity. The idea consisted of designing and building up an intelligent system using image processing, data mining and artificial intelligence concepts to take visual inputs of sign languages hand gestures and generate easily recognizable form of outputs in the form of text and voice with 82% accuracy.</p> <p><a href="https://journalnx.com/journal-article/20150118">https://journalnx.com/journal-article/20150118</a></p>

Deaf people are one of those with disabilities who cannot speak and hear. Deaf people using sign language for communication who use sign language with hand gestures, body and convey facial expressions using sign language. One important component in sign language is the alphabetical finger or the manual alphabet needed to complete communication. The alphabet of the finger is done by spelling words on spoken language, by spelling letter by letter using a finger. This method is used to spell the name or mention the word. However, not everyone understands sign language, so tools needed to bridge communication between people who are deaf with normal people. One solution that will be offered is to use computer technology as a tool to recognize sign language. The technology is an automatic language translator system that process input images using by using the Convolutional Neural Network (CNN). On this research consist of 3 class such us A, assalamualaikum and hallo which get the accuracy respectively 100% for each class.

Sign Language is a way of communication used by hearing-impaired people. As many normal people don’t know the Sign Language it is difficult to understand by them to understand thus, increasing the communication gap between normal and hearing-impaired people. To overcome this problem many systems,using Machine Learning(ML), Artificial Intelligence(AI), Deep Learning(DL), computer vision methods, were developed and being developed. These systems are used to translate Indian Sign Language(ISL) to understandable text, facilitating the communication between normal and hearing-impaired people. This study briefly explains various type of Indian Sign Language(ISL) recognition systems.

Hand gesture is one of the method used in sign language for non-verbal communication. It is most commonly used by deaf & dumb people who have hearing or speech problems to communicate among themselves or with normal people. Various sign language systems has been developed by many makers around the world but they are neither flexible nor cost-effective for the end users. Hence in this paper introduced software which presents a system prototype that is able to automatically recognize sign language to help deaf and dumb people to communicate more effectively with each other or normal people. Pattern recognition and Gesture recognition are the developing fields of research. Being a significant part in nonverbal communication hand gestures are playing key role in our daily life. Hand Gesture recognition system provides us an innovative, natural, user friendly way of communication with the computer which is more familiar to the human beings. By considering in mind the similarities of human hand shape with four fingers and one thumb, the software aims to present a real time system for recognition of hand gesture on basis of detection of some shape based features like orientation, Centre of mass centroid, fingers status, thumb in positions of raised or folded fingers of hand

Abstract. Inability to use speech interfaces greatly limits the deaf and hearing impaired people in the possibility of human-machine interaction. To solve this problem and to increase the accuracy and reliability of the automatic Russian sign language recognition system it is proposed to use lip-reading in addition to hand gestures recognition. Deaf and hearing impaired people use sign language as the main way of communication in everyday life. Sign language is a structured form of hand gestures and lips movements involving visual motions and signs, which is used as a communication system. Since sign language includes not only hand gestures, but also lip movements that mimic vocalized pronunciation, it is of interest to investigate how accurately such a visual speech can be recognized by a lip-reading system, especially considering the fact that the visual speech of hearing impaired people is often characterized with hyper-articulation, which should potentially facilitate its recognition. For this purpose, thesaurus of Russian sign language (TheRusLan) collected in SPIIRAS in 2018–19 was used. The database consists of color optical FullHD video recordings of 13 native Russian sign language signers (11 females and 2 males) from “Pavlovsk boarding school for the hearing impaired”. Each of the signers demonstrated 164 phrases for 5 times. This work covers the initial stages of this research, including data collection, data labeling, region-of-interest detection and methods for informative features extraction. The results of this study can later be used to create assistive technologies for deaf or hearing impaired people.

Sign language is a system for communication using visual gestures and signs for deaf and dumb people. This language is especially used by people who can’t speak or hear. Sign language is oldest and natural form of language for communication, since most of the people are not aware of sign language, hence it is tedious to understand, for solving this problem we have built a real time method using convolution neural networks (CNN) for hand-based gestures to detect sign language. In our CNN model, first the hand is passed through a filter and then the filter is applied. Finally, the hand is passed through a classifier which predicts the class of the hand gestures. Our model provides 98% accuracy for the alphabet A-Z letters. On the other hand, the most common hurdle deaf and dumb come across is communication with the normal people and with the fact i.e., each and every normal person doesn't know the sign language. Another main feature of the project is to create a communication system for the deaf people. The functionality of these part of the system is to translate audio message to the corresponding sign language. This part of the system takes the audio message as input, and converts the audio recorded message into the respective sign image and videos and displays the relevant American sign language or GIFs which we have already defined. With the aid of this part of the system, communication between normal and deaf people gets feasible. Overall, the main idea of the project is to make a system which can help deaf and dumb to interact with normal people.

Communication is very essential for each and every individual person, because it allows us to express our feelings, sentiments, and ideas etc. Humans can communicate with one another through a variety of means, including natural language, body language, and even writing. People who do not have any disabilities, such as hearing or speaking impairments, can use all of the foregoing modes of communication to communicate with other normal people. Natural language, which includes words, sentences, and so on, is the most often utilized type of communication by normal people. On the other hand, People with disabilities use Sign Language to communicate with others. Deaf and dumb people comprehend sign language easily and quickly, but it is difficult for them to communicate with normal people who do not understand sign language. Deaf and dumb persons are unable to effectively convey their feelings and thoughts to other normal people due to hearing and speaking issues, which is a serious problem all over the world. To address this issue, numerous systems were developed that interpret sign language (Hand movements, facial expression etc.), recognize signs (Hand Gestures), and display the relevant text on the system, allowing normal people to understand the language of the deaf and dumb. The fundamental purpose of this survey paper is to read all of the methods that were used in various systems and to identify the most efficient method that can be adopted in future research work.

This research was designed for analyse and compare patterns of EEG signals while blinded and normal people performing touching-leaning. The pattern analysis focuses on important EEG wavebands during touching, such as delta, theta, alpha and gamma waveband. The EEG waveband-datasets were detected and recorded with inexpensive device, the NeuroSky Mindwave, where it is connected to a computer for data analysis through Bluetooth communication and had electric noise reduction chipset inside. The analysis for the EEG waveband pattern-comparisons is performed by utilizing waveband power spectrum and statistical technique based on FFT algorithm, Area Under Curve (AUC), mean, S.D., T-score, and P-value testing. The experiment shown that dominant EEG signal wavebands of blinded people when touching-learning are delta, theta, alpha and gamma. These wavebands were higher than normal people. Moreover, by using statistical analysis T-score and P-values testing, analyzed results illustrate that normal and blinded people EEG wave patterns are significantly different on gamma waveband where blinded people have significantly higher gramma wave during touching-learning activities. The objects used for touching-leaning in the experiments are square, triangle, circle, and hexagon shape of tactile pictures. The observation also shown that blinded people use their muscle in movement more than normal people which also strongly related to gramma wave. In addition, another wave did not relate to statistic significant. This result illustrated to normal and blinded people thinking and imagination with the same pattern.

Sign language is a way of communicating using hand gestures, movements, and facial expressions, instead of spoken words. It is the medium of communication used by people who are deaf or have hearing impairments to exchange information between their own community and with normal people. In order to bridge the communication gap between people with hearing and speaking disabilities and people who do not use sign language, a lot of research work using machine learning algorithms has been done. Hence, Sign language translator came into picture. Sign Language Translators are generally used to interpret signs and gestures from deaf and hard hearing people and convert them into text. Key Word: Sign language, Hand Gestures, Helping Impaired People, Machine Learning, Deep Learning.

Real time Indian sign language recognition system

In this world, we have more than 466 million deaf dumb peoples. Normal peoples are communicating with voice, likewise deaf and dumb people are communicating by the use of sign language. Generally, without knowing the language, one cannot talk to other people in the world, similarly without knowing the sign language normal people cannot talk to deaf and dumb people. Within themselves, deaf and dumb people will communicate using sign language, which normal people cannot understand, but still hearing is one way to understand the people, so expression and gesture are also a way to understand thoughts of deaf and dumb people. To find a solution for this problem, either normal person needs to learn the sign language or we need to create a platform which will convert the sign language to voice and voice to sign language. Though we have many different types platform, still it not efficient so we need to use the advanced method which recognizes the input much efficiently. Since we are using gesture and voice as an input, this will act as bidirectional way communication. The focus of this work is to create a bidirectional-based system to identify sign language gestures from the sign language images and converting that to voice output. Since we are developing the application to computer and android, it should have two-way communication so that both can act as input and output node. For efficient voice input and output, we are using android mobile.KeywordsGestureSign languageBidirectionalDeaf and dumbTwo-way communication systemComputer visionSIFT algorithm

The inability to communicate verbally is widely acknowledged as a significant disability. The primary objective of this research is to create a practical system aimed at individuals with hearing impairments, particularly those who depend on sign language as their primary means of communication. This system aims to enable deaf individuals to express themselves, communicate effectively and facilitate understanding of their language which would otherwise be challenging since most people are unfamiliar with sign language. By employing human gesture interpretation and motion capture, this technology can facilitate the translation of sign language into spoken language and vice versa. Despite the existence of various methods to convert sign language into voice, none of them currently provide an entirely intuitive user interface. Our objective is to create a system that not only translates sign language but also integrates a natural user interface, thus enhancing accessibility for individuals who are blind or have visual impairments. This system will achieve this by recognizing facial expressions and effectively conveying emotions behind words assisting visually impaired individuals in expressing themselves more effectively. Hence, in this project, we tentatively aim to build a system that can ease the lives of blind, deaf and dumb people to some extent. Normal communication like normal people do might never be possible for differently abled people but through our project, we try to provide them with a tool that can help them experience normalcy while communicating with normal people.

소리가 잘 들리지 않는 청각장애인은 소리를 표시하는 글자를 익히고 복잡한 개념을 전달하는 글을 이해하는 데 어려움이 많다. 그래서 자연스럽게 표정, 몸짓, 손짓으로 의사를 전달하는 수화가 청각장애인들의 주요 의사소통수단으로 자리잡아왔지만 사회에서는 글과 말이 주요 정보전달 수단으로 이용되고 있어 청각장애인이 정보에 접근하고 지식을 넓혀 직업을 갖는데 큰 어려움이 있다. 특히 근래에 인터넷이 보편화됨에 따라 정상인의 정보 습득량은 크게 늘었지만, 글로 표현된 인터넷을 이해하기 어려운 청각장인들은 인터넷 정보에 접근하는데 어려움이 많다. 본 연구에서는 청각장애인을 위한 수화 웹페이지를 제작할 수 있는 수화 웹페이지 제작시스템을 개발하였다. 수화 웹페이지 제작 시스템은 청각장애인용 수화 웹페이지 제작자가 통상적인 인터넷 사용에 필요한 하드웨어 및 소프트웨어 환경만 갖추면 운영할 수 있도록 웹기반으로 개발하였다. 수화용 웹 페이지 제작자는 게시판 형태의 인터페이스를 이용하여 변역하고자하는 문장을 서버에 전송할 수 있다. 서버는 수신된 문장을 수화로 번역하여 3D 아바타 기반의 수화 그래픽 애니메이션을 랜더링한다. 수화 그래픽 애니메이션은 MP4 형식의 동영상으로 변환되며, 스트리밍 서버의 저장소에 보관된다. 저장된 수화 동영상의 이름과 주소는 수화 웹페이지 제작자가 처음 문장을 입력하였던 게시판에 표시되도록 하여 향후 청각장애인용 웹 페이지를 제작할 때 활용할 수 있도록 하였다. 또한 본 연구에서는 웹페이지 제작자가 공공기관의 웹페이지를 제작할 수 있도록, 이들 기관에서 현재 사용된 글들을 수화로 번역할 때 필요한 수화 단어를 수화 웹페이지 제작 시스템의 수화 사전에 추가하였다. 수화 단어를 추가함으로써 웹기반 수화 웹페이지 제작 시스템이 공공부문에 보다 더 많이 활용될 수 있을 것이다. Hearing-impaired people have difficulty in hearing, so it is also hard for them to learn letters that represent sound and text that conveys complex and abstract concepts. Therefore it has been natural choice for the hearing-impaired people to use sign language for communication, which employes facial expression, and hands and body motion. However, the major communication methods in daily life are text and speech, which are big obstacles for the hearing-impaired people to access information, to learn and make intellectual activities, and to get jobs. As delivering information via internet become common the hearing-impaired people are experiencing more difficulty in accessing information since internet represents information mostly in text forms. This intensifies unbalance of information accessibility. This paper reports web-based text-to-sign language translating system that helps web designer to use sign language in web page design. Since the system is web-based, if web designers are equipped with common computing environment for internet browsing, they can use the system. The web-based text-to-sign language system takes the format of bulletin board as user interface. When web designers write paragraphs and post them through the bulletin board to the translating server, the server translates the incoming text to sign language, animates with 3D avatar and records the animation in a MP4 file. The file addresses are fetched by the bulletin board and it enables web designers embed the translated sign language file into their web pages by using HTML5 or Javascript. Also we analyzed text used by web pages of public services, then figured out new words to the translating system, and added to improve translation. This addition is expected to encourage wide and easy acceptance of web pages for hearing-impaired people to public services.

In day-to-day life, deaf and dumb people are facing a lot of battles in speaking with typical individuals. Sign language plays a very important role for the deaf and people interact with the outside world. Due to the nonattendance of wonderful gesture-based communication interpreters still it leads the communication break. Powerful collaboration possible only if the deaf and dumb persons nearby sign language translator. To overcome this problematic circumstance in our approach utilizing smartphones for establishing active communication with normal people. The main effort of our proposed framework is to design smart mobile for helping the deaf and the dumb community to convey their thoughts anywhere they are located so it reduces the communication hole among normal, hearing, and speech-impaired people. Smartphones act as an interface to translate sign language to speech and vice versa. The various gestures-based signs captured automatically and translating sign language to speech. To process an image of gestures using several image processing techniques and deep learning models. Investigation result shows that this smartphone-based dynamic hand gesture recognition model provides an exact and effective system for the interaction between non-signer and signer.