Temporal Modeling and Real-Time Recognition Approaches in SLR Systems

ABSTRACTSign language is a communication tool for deaf and dumb people that includes known signs or body gestures to transfer meanings. It uses shapes, directions, movements of hands, and also facial expressions. A sign not only transmits a word but also conveys a tone. Many of the deaf people are not only able to speak, but also not able to write or read a language, so developing sign language translation or in other words sign language recognition (SLR) system can be very vital in their life. The SLR is extremely desired because of its capability to overcome the obstacles between deaf and normal people. It is one of the most important research fields in the human computer interaction studies. Hence, this paper presents an overview of the recent main research works with the vision-based SLR system, and the existing recognition techniques are discussed. Next, we focus on video-based SLR system and perform continuous SLR within video sequences.

Sign Language is a meansof communication between audibly challenged people. To provide an interface between the audibly challenged community and the rest of the world we need Sign Language translators. A sign language recognition system computerizes the work of a sign language trEvery Sign Language Recognition (SLR) System is trained to recognizespecific sets of signs and they correspondingly output the sign in the required format. These SLR systems are built with powerful image processing techniques. The sign language recognition systems are capable of recognizing a specific set of signing gestures and output the corresponding text/audio. Most of these systems involve the techniques of detection, segmentation, tracking, gesture recognition and classification. This paperproposes a design for a SLR System.

An optical modeless Sign Language Recognition (SLR) system is presented. The system uses the HAusdorf-Voronoi NETwork (HAVNET), an artificial neural network designed for 2D binary pattern recognition. It uses adaptation of the Hausdorff distance to determine the similarity between an input pattern and a learned representation. A detailed review of the architecture, the learning equations, and the recognition equations for the HAVNET network are presented. Competitive learning has been implemented in training the network using a nearest-neighbor technique. The SLR system is applied to the optical recognition of 24 static symbols from the American Sign Language convention. The SLR system represents the target images in a 80 X 80 pixel format. The implemented HAVNET network classifies the inputs into categories representing each of the symbols, using an output layer of 24 nodes. The network is trained with 5 different formats for each symbol and is tested with all 24 symbols in 15 new formats. Results from the SLR system without competitive training show shape identification problems, when distinguishing symbols with similar shapes. Implementation of competitive learning in the HAVNET neural network improved recognition accuracy on this task to 89%. The hand gestures are identified through a window search algorithm. Feature recognition is obtained from edge enhancement by applying a Laplacian filter and thresholding, which provides robustness to pose, color and background variations.

Sign language recognition (SLR) system is helpful for individuals who can't talk or hear. One can easily communicate with this kind of people using SLR system. In this paper we are focusing on SLR approach where we have apply our proposed algorithm to recognize hand gestures of routine words like Morning, Night, Monday, Tuesday and so forth. Total 10 routine words we have recognize using our proposed technique. For recognize hand pose we have utilized skin color detection algorithm. Further we have applied correlation-coefficient algorithm to distinguish closeness features which are further continue in Neuro-fuzzy(NF) classification algorithm to recognize words. Presented algorithm tried on Matlab. With taken of 10 routine words our proposed framework of techniques got 92% correction rate.

Automating real-time sign language translation through deep learning and machine learning techniques can greatly enhance communication between the deaf community and the wider public. This research investigates how these technologies can change the way individuals with speech impairments communicate. Despite advancements, developing accurate models for recognizing both static and dynamic gestures remains challenging due to variations in gesture speed and length, which affect the effectiveness of the models. We introduce a hybrid approach that merges machine learning and deep learning methods for sign language recognition. We provide new model for the recognition of Kazakh Sign Language (QazSL), employing five algorithms: Support Vector Machine (SVM), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Networks (CNN) with VGG19, ResNet-50, and YOLOv5. The models were trained on a QazSL dataset of more than 4,400 photos. Among the assessed models, the GRU attained the highest accuracy of 100%, followed closely by SVM and YOLOv5 at 99.98%, VGG19 at 98.87% for dynamic dactyls, LSTM at 85%, and ResNet-50 at 78.61%. These findings illustrate the comparative efficacy of each method in real-time gesture recognition. The results yield significant insights for enhancing sign language recognition systems, presenting possible advancements in accessibility and communication for those with hearing impairments.

Over 300 sign languages are used worldwide, posing challenges for effective communication between deaf and hearing individuals. This study presents a bilingual sign language recognition (SLR) system using deep learning to enhance accessibility for the deaf and mute communities. The system processes real-time video input, leveraging MediaPipe for hand and body landmark extraction. For static gesture classification (e.g., alphabet recognition), a Support Vector Machine (SVM) with a linear kernel is employed. For dynamic gesture sequences (e.g., word-level recognition), a Long Short-Term Memory (LSTM) network is used to model temporal patterns. The models were trained on large-scale datasets of Arabic and English sign languages, achieving recognition accuracies exceeding 99% for English letters and over 93% for selected Arabic words. The training dataset consists of images from Kaggle and real-time videos, and the test dataset uses independent real-time videos not seen during training. The system supports sign-to-text translation as well as voice and text-to-sign conversion through avatars or image sequences, promoting inclusive, real-time communication across linguistic boundaries.

Sign language recognition devices are effective approaches to breaking the communication barrier between signers and non-signers and exploring human-machine interactions. Wearable gloves have been developed for gesture recognition and virtual reality applications by employing flexible sensors for motion detection and machine learning for data analysis. However, most existing wearable devices present limited sign language translating capacity due to the sensors' design and distribution. Here, we propose a cost-effective dual-hand soft fiber optic glove system consisting of multimode soft liquid-core fiber optic sensors, gyroscopes, wireless printed circuit boards, and batteries for sign language translation. In combination with different deep learning techniques and recognition strategies, the glove system can recognize static gestures and dynamic gestures of American Sign Language, and deduce the meaning of sentences by the sequence of gestures. The soft glove system exhibits a broad sign language range (10 numbers, 26 alphabets, 18 words, and 5 sentences meaning prediction), and high recognition accuracy (98.6% for static gestures, 95% for dynamic gestures). The results also present the recognizing capacity for high-correlated gestures (e.g., “M” and “N”). Finally, we demonstrate its application for controlling the motion of a virtual character through 7 discrete commands in the VR interface.

The essential research related to sign language recognition states that there is a need of remarkable progress in this domain. Selecting features is decisive to gesture recognition, as hand gestures are very fine in shape variation, motion and textures. For static posture recognition, although it is possible to recognize hand posture by extracting some geometric features such as fingertips, finger directions it is not reliable due to self -- occlusion and lighting condition. Silhouette and textures, however, they are inadequate in recognition process. Therefore this work focuses on two of the research problems comprising automatic sign language recognition, namely robust segmentation techniques for consistent detection, preserving the shape contour which is therefore useful for textural feature extraction. The discrimination ability of the two segmentation methods for texture computation is observed and compared by objective parameters. Experiments are done in various camera based images and that it explores the need for prior effective contour segmentation and textural features for constructing a precision based recognition system for sign language.

Sign language recognition is one of the most promising sub-fields in gesture recognition research. A sign language is a language which, instead of acoustically conveyed sound patterns, uses visually transmitted sign patterns. Sign languages are commonly developed for hearing impaired communities, which can include interpreters, friends and families of hearing impaired people as well as people who are hard of hearing themselves. Developing a sign language recognition system will help the hearing impaired to communicate more fluently with the normal people. A simple sign language recognition system employing skin color segmentation and Neural Network has been developed. A simple segmentation process is carried out to separate the right and left hand regions from the image frame and in the preprocessing stage a simple vertical interleaving method has been proposed to reduce the size of the image. The 2D moment invariant features of the right and left hand interleaved image is obtained as features. Using the interleaved 2D-moment Invariant features, a simple neural network model has been developed. The system has been implemented and tested for its validity. Experimental results show that the system has a recognition rate of 92.58%.

Sign is the way of communication when there is no appropriate verbal communication for deaf and dump persons. Sign Language Recognition System (SLRS) acts as a contrivance to perform the conversion of sign language into text or voice. Many classification techniques and algorithms with a variety of methods already exist in SLRS. But still only a few considerable contributions are there in Tamil Sign Language Recognition. The objective of this paper is to extract the features from the pre-processed image and propose an optimization based classifier to improve the classification accuracy in Tamil Sign Language images. The experimental result of the proposed work demonstrated that BPN-PSO outperforms existing classification techniques.

Sign language is the main communication language for deaf and dumb people. There have many difficulties for deaf and dumb people when they communicate with normal people. Only a few people can use Sign Language proficiently and most of the people don't know how to communicate the deaf people. From the point of view of computer vision, we can develop the sign language recognition for deaf and dumb people. Most of the sign language recognition systems are based on the static sign. In motion-based sign language recognition system, we proposed feature extraction method and recognition using machine learning. Every nation has its own sign language and our proposed system is based on Myanmar National Sign Language.

Sign language is a main communication tool for hearing impaired people. The sign language recognition systems are designed to convert it into text or speech, breaking the interaction barrier for deaf person. The paper reviews the state of the art in sign language recognition. The study focuses on data acquisition methods, because they put the major restrictions on a further recognition process. We perform the sign language recognition using data collected by the Leap Motion Controller. The device detects user's hands and transmits it into a three-dimensional model, providing hands and fingers location and posture. This paper presents a system that is capable of recognition and classification of the sign language using deep-convolutional neural network. We highlight the main features of the Leap Motion Controller that improves and simplifies feature extraction. Also, the paper discusses the problems we faced during the sign language recognition.

Human–Computer Interaction (HCI) refers to the interaction between the computers and human. One of the most important applications of (HCI) is sign language recognition. Several research works aimed to interpret and translate the sign language to a spoken language to help the hear impaired persons in integrating with communities. Sign language is the main way of communication for the deaf persons and hearing impaired which enable them to communicate with their societies and between each other’s. According to the World Health Organization, there are 466 million hearing loss people (i.e. 5% of the world population), 432 million or 93% of them are adults and 34 million or 17% of them are children. The hearing-impaired persons often need to the same level of mental capability as the normal persons. The main problem is the most of the hearing persons who cannot understand the sign language and most of hearing impaired cannot read or write our spoken language, this represents a barrier between the deaf persons and their societies so developing an automatic sign language recognition system is very necessary. This research introduces dynamic Arabic Sign Language recognition system using Microsoft Kinect. The recognition depends on using two machine learning algorithms (a) Decision Tree and (b) Bayesian Network then applied Ada-Boosting technique to enhance the recognition of the system, we compared the results with two direct matching techniques: (a) Dynamic Time Wrapping and Hidden Markov Model the system was applied on 42 Arabic gestures related to medical field. The experimental results showed that the proposed system recognition rate reached 91.18% for Decision Tree classifier, 92.50% for Bayesian classifier and 93.7% after applying Ada-Boosting.

Sign Language Recognition systems have been thoroughly researched in recent times to bridge the gap between the signer and non-signer community. This paper focuses on fulfilling the need for a thorough review of the works done in building such systems. The systems discussed herein for static Sign Language Recognition are broadly classified on Support Vector Machine, Convolutional Neural Network as classifiers. The identified phases for the reviewed systems include preprocessing, feature extraction and classification. The methods discussed in these phases include Principal Component Analysis, Histograms of Orientations Gradient, Hybrid Sign Invariant Feature Transformation. Further the limitations and improvements in approaches are relatively portrayed to give a better analysis for future work. The overall aim for this paper is to provide insight into making effective systems for Sign Language Recognition. This will make these systems easily available for educating the masses about static Sign Language. Thus, sign language will no longer remain a barrier.

The population of deaf-mute in Taiwan is increasing every year. These deaf-mute persons usually use sign language to communicate with each other. However, most hearing persons cannot understand sign languages because they have not learned it. In order to let normal persons understand sign language, a sign language recognition system is necessary. In this paper, such a sign language recognition system is proposed. In the proposed system, a depth image is captured by Kinect sensor, palm area is segmented from depth image, palm binary image is thresholded by using Otsu thresholding method, and background noises are removed by morphological closing operators, and SURF features and descriptors are extracted to identify the sign language. Experimental results show that the proposed method is effective to detect and identify the numerals and letters in the English Alphabet used in the sign language.