Deep Learning Technology to Recognize American Sign Language Alphabet.

With its development, artificial intelligence has formed the basis for many studies aimed at facilitating people's lives. More successful results have been tried to be obtained with the increasing data and developing equipment in these studies. It is seen that these developments in artificial intelligence are reflected in the studies related to sign language conversion. In this study, a data set belonging to the letters in the Turkish Sign Language Alphabet was created, and the classification process was carried out with both the deep learning model we created and VGG16, Inceptionv3, Resnet, and Mobilnet models, which are frequently used in image classification. In addition, an open-source data set containing the letters in the American Sign Language Alphabet was organized similar to the data set containing the letters in the Turkish Sign Language Alphabet we created, and Deep Learning models were used to classify the letters in the American Sign Language Alphabet by using this data set. Performance evaluations of the classifications made by Deep Learning Models using both data sets were made. With this study, the results obtained from training Deep Learning methods with different data sets were compared. In addition, it is thought that the study will be useful in determining both the data set and the deep learning method to be used for the studies on the recognition of Sign Language Letters.

The predominant means of communication is speech; however, there are persons whose speaking or hearing abilities are impaired. Communication presents a significant barrier for persons with such disabilities. The primary objective of this work is to develop an efficient solution with machine learning for the speech and hearing impairment community with the help of AI. This study helps in the development of future technologies in the enhancement of communication accessibility. In the proposed work, a machine learning approach for the American Sign Language alphabet using image detection and processing techniques are utilized. The experiment was conducted using an American Sign Language recognition dataset obtained from Kaggle, which provides images of hand Gestures of the alphabet. The dataset was divided into training and testing subsets in the ratio of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$80: 20$</tex>, where 80 % of the data was used in training the model and the remaining 20 % was designated for testing the model. The dataset was normalized using denoising filters and proceeded with Canny edge detection to highlight sharp edge features. From the edge detected image, Histogram of Oriented Gradient features were extracted to detect the letter and shape information. These extracted features are then fed into the classifiers: Support Vector Machine, Random Forest, and KNearest Neighbors. The study was analyzed by experimental results, which show that the Support Vector Machine classifier achieved the highest accuracy of 93.33 %, closely followed by Random Forest and K-Nearest Neighbors. The analysis confirms that applying edge-based preprocessing along with the Histogram of Oriented Gradients feature extraction enables an effective classification of American Sign Language alphabets. This Comparative study provides a solid foundation for developing low-complexity, high-accuracy sign language recognition systems.

Hand gesture recognition is a rich area of research and covers a wide scope of applications from human machine interaction (e.g in games) to Deaf people computer interface and from 3D animationto control of mechanical systems. In this work a new algorithm for hand gesture recognition is proposed and evaluated. This algorithm uses moment invariants for feature extraction and neural networks for classification. To evaluate the algorithm a subset from the ASL (American Sign Language) is used, this subset consists of the 26 American one-handed sign language alphabet as a training set for the system. The system then tested using images that are different from the training set in size,orientation and position and the results for these tests are presented and discussed.

The use of computer vision along with machine learning techniques resulted in a breakthrough in medical and social fields. One such intervention is the sign language detection, enabling communication for differently-abled people. Hand gestures are the common method used for this. This paper deals with using image processing techniques like RGB to binary image conversion, skin detection, edge detection to extract the important features of the gesture and provide them as inputs to Convolution Neural Networks (CNN) to improve the accuracy in predicting American Sign Language alphabets (ASL). Using the preprocessed image data set of the hand gesture in our proposed mechanism has predicted the alphabets with higher accuracy up to 99% as compared to the model trained with original data set of images. KeywordsImage processingEdge detectionConvolution neural netsAmerican sign language

American sign language recognition and training method with recurrent neural network

Physical disability is one of the factor in human beings, which cannot be ignored. A person who can’t listen by nature is called deaf person. For the representation of their knowledge, a special language is adopted called ‘Sign-Language’. American Sign Language (ASL) is one of the most popular sign language that is used for learning process in deaf persons. For the representation of their knowledge by deaf persons, a special language is adopted ‘Sign-Language’. American Sign Language contains a set of digital images of hands in different shapes or hand gestures. In this paper, we present feature based algorithmic analysis to prepare a significant model for recognition of hand gestures of American Sign Language. To make a machine intelligent, this model can be used to learn efficiently. For effective machine learning, we generate a list of useful features from digital images of hand gestures. For feature extraction, we use Matlab 2018a. For training and testing, we use weka-3-9-3 and Rapid Miner 9 1.0. Both application tools are used to build an effective data modeling. Rapid Miner outperforms with 99.9% accuracy in auto model.

People around the world with speech and hearing impairment use a media of communication known as ‘Sign language’. In recent times, Sign language is omnipresent. However, there exists a challenge for people who do not know sign language, to communicate with people who can communicate exclusively using sign language. This gap can be bridged by using technologies of recent times to recognize gestures and design intuitive systems with deep learning. The aim of this paper is to recognise American Sign Language gestures dynamically and create an intuitive system which provides sign language translation to text and speech of various languages. The system uses Convolutional neural network, natural language processing, language translation and text-to-speech algorithms. It is capable of recognizing hand gestures dynamically and predicting the corresponding letters to form a desired sentence accurately.

This research introduces a system for translating American Sign Language (ASL) utilising Convolutional Neural Networks. (CNNs). The system under consideration has the capability to identify manual gestures executed by persons communicating in American Sign Language (ASL) and subsequently convert them into written language, thereby facilitating uninterrupted communication between the deaf and hearing populations. The CNN model was validated and trained using a set of data made up of 78,300 pictures of movements in the ASL Alphabet written in American Sign Language (ASL). In order to improve the performance of the model, the pre-processing of the data included a number of stages, such as converting the photos to grayscale, normalising the pixel values, and enhancing performance of the model. In order to ease classification, fully connected layers were added after a succession of pooling and convolutional layers in CNN's design. After 15 epochs of training, the model attained a validation accuracy of 99.85%. The findings of this research demonstrate the viability of employing Convolutional Neural Networks (CNNs) in the creation of precise and effective American Sign Language (ASL) translation systems, which can serve as a means of communication for people with auditory impairments.

Hand gesture for communication has proven effective for humans, and active research is ongoing in replicating the same success in computer vision systems. Human–computer interaction can be significantly improved from advances in systems that are capable of recognizing different hand gestures. In contrast to many earlier works, which consider the recognition of significantly differentiable hand gestures, and therefore often selecting a few gestures from the American Sign Language (ASL) for recognition, we propose applying deep learning to the problem of hand gesture recognition for the whole 24 hand gestures obtained from the Thomas Moeslund’s gesture recognition database. We show that more biologically inspired and deep neural networks such as convolutional neural network and stacked denoising autoencoder are capable of learning the complex hand gesture classification task with lower error rates. The considered networks are trained and tested on data obtained from the above-mentioned public database; results comparison is then made against earlier works in which only small subsets of the ASL hand gestures are considered for recognition.

Speech impairment affects an individual’s ability to communicate. People affected by this problem, suffer by the communication barrier with the normal people. Hence, to express their emotions and thoughts people will use sign language for their communication and. Although sign language is common in world, there may be a difficulty for people who do not have knowledge about sign language to communicate with speech impaired people who knows the sign language. Now there is a prominent growth in the field of computer vision and deep learning, which has been tremendous development in the fields of sign and motion recognition. The proposed model focusses on overcoming the challenges in verbal communication between non-sign language speakers and sign language speakers by building a deep learning model to recognize alphabets of American sign language. The model was trained on the dataset collected from Kaggle website which consisted of 26 American Sign Language alphabets. The model achieved 99.3% mean average precision in recognition of sign and average probability value for test image achieved was 0.99. The deep learning method used to build the model achieved more accuracy than the previous solutions.

American sign language is an important communication way to convey information among the deaf community in North America and is primarily used by people who have hearing or speech impairments. The deaf community faces a struggle in schools and other institutions because they usually consist primarily of hearing people. Besides, deaf people often feel misunderstood by people who do not knowsign language, for example, family members. In the last two decades, researchers have been proposing automatic sign language recognition systems to facilitate the learning of sign language, and nowadays, computer scientists have focused on using artificial intelligence in order to develop a system capable of reducing the communication gap between hearing and deaf people. In this paper, it is proposed a siamese convolutional neural network for American sign language alphabet recognition. This siamese architecture allows the computer to reduce the high interclass similarity and high intraclass variations. The results show that the proposed method outperforms the state of the art systems.

In the sign language alphabet, several hand signs are in use. Automatic recognition of performed hand signs can facilitate the communication between hearing and none hearing people. This framework proposes hand posture recognition of the American Sign Language alphabet based on a neural network (NN) which works on geometrical feature extraction of the hand. The user’s hand is captured by a 3D depth-based sensor camera. Consequently, the hand is segmented according to the depth features. The proposed system is called ‘Depth-based Geometrical Sign Language Recognition’ (DGSLR). The DGSLR adopted an easier hand segmentation approach, which is further used in other segmentation applications. The proposed geometrical feature extraction framework improves the accuracy of recognition due to unchangeable features against hand orientation or rotation compared to Discrete Cosine Transform (DCT) and Moment Invariant. As a support vector machine (SVM) is a type of artificial neural network (ANN), it is used to drive desired outcomes. Since there are 26 different signs in the Sign Language alphabet, a multi-class SVM versus a single SVM classifier with 26 classes by an RBF kernel was used to validate each class. The proposed framework is proficient to hand posture recognition and provides an accuracy of up to 96.78%. The findings of the iterations demonstrated that the combination of the extracted features resulted in a better accuracy rate in the recognition process in the classification step.

Over half a million people in the United States use American Sign Language (ASL) as their primary mode of communication. Automatic ASL recognition would enable Deaf and Hard of Hearing (DHH) users to interact with others who are not familiar with ASL as well as voice-controlled digital assistants (e.g., Alexa, Siri, etc.). While ASL recognition has been extensively studied, there is a little attention given to recognition of ASL non-manual body markers. The non-manual markers are typically expressed through head, torso and shoulder movements, and add essential meaning and context to the signed sentences. In this work, we present ExASL, a sentence-level ASL recognition system using millimeter-wave radars. ExASL can recognize manual markers (hand gestures) and non-manual markers (head and torso movements). It utilizes multi-distance clustering to recognize body parts and cluster mmWave point clouds. We then present a multi-view deep learning algorithm that can learn from clustered body part representation for an expressive sentence-level recognition. Our evaluation shows that ExASL can recognize ASL sentences with a word error rate of 0.79%, sentence error rate of 1.25%, and non-manual markers with an accuracy of 83.5%.

As part of the development of an educational tool that can help students achieve fluency in American Sign Language (ASL) through independent and interactive practice with immediate feedback, this paper introduces a near real-time system to recognize grammatical errors in continuous signing videos without necessarily identifying the entire sequence of signs. Our system automatically recognizes if a performance of ASL sentences contains grammatical errors made by ASL students. We first recognize the ASL grammatical elements including both manual gestures and nonmanual signals independently from multiple modalities (i.e. hand gestures, facial expressions, and head movements) by 3D-ResNet networks. Then the temporal boundaries of grammatical elements from different modalities are examined to detect ASL grammatical mistakes by using a sliding window-based approach. We have collected a dataset of continuous sign language, ASL-HW-RGBD, covering different aspects of ASL grammars for training and testing. Our system is able to recognize grammatical elements on ASL-HW-RGBD from manual gestures, facial expressions, and head movements and successfully detect 8 ASL grammatical mistakes.

American Sign Language (ASL) is a visual gestural language which is used by many people who are deaf or hard-of-hearing. In this paper, we design a visual recognition system based on action recognition techniques to recognize individual ASL signs. Specifically, we focus on recognition of words in videos of continuous ASL signing. The proposed framework combines multiple signal modalities because ASL includes gestures of both hands, body movements, and facial expressions. We have collected a corpus of RBG + depth videos of multi-sentence ASL performances, from both fluent signers and ASL students; this corpus has served as a source for training and testing sets for multiple evaluation experiments reported in this paper. Experimental results demonstrate that the proposed framework can automatically recognize ASL.