Supervised learning classifiers for Arabic gestures recognition using Kinect V2

ABSTRACTSign language is a communication tool for deaf and dumb people that includes known signs or body gestures to transfer meanings. It uses shapes, directions, movements of hands, and also facial expressions. A sign not only transmits a word but also conveys a tone. Many of the deaf people are not only able to speak, but also not able to write or read a language, so developing sign language translation or in other words sign language recognition (SLR) system can be very vital in their life. The SLR is extremely desired because of its capability to overcome the obstacles between deaf and normal people. It is one of the most important research fields in the human computer interaction studies. Hence, this paper presents an overview of the recent main research works with the vision-based SLR system, and the existing recognition techniques are discussed. Next, we focus on video-based SLR system and perform continuous SLR within video sequences.

Sign Language is a meansof communication between audibly challenged people. To provide an interface between the audibly challenged community and the rest of the world we need Sign Language translators. A sign language recognition system computerizes the work of a sign language trEvery Sign Language Recognition (SLR) System is trained to recognizespecific sets of signs and they correspondingly output the sign in the required format. These SLR systems are built with powerful image processing techniques. The sign language recognition systems are capable of recognizing a specific set of signing gestures and output the corresponding text/audio. Most of these systems involve the techniques of detection, segmentation, tracking, gesture recognition and classification. This paperproposes a design for a SLR System.

A way of communication on an equal footing with oral and written speech is sign language. Oral speech is a common way of communication, written speech has, with the development of information technologies, been taking up more and more space. In a parallel with them, communication can also take place through sign language, which is, to deaf people, as well as to all those who use it, a natural, simple and easy way to communicate. Relationship towards sign language has changed significantly throughout history from acceptance and isolated use, to complete rejection, and then to encouraging its adoption and emphasizing its importance for the cognitive, emotional, educational, social, and general development of deaf children. Serbian Sign Language (SSL) serves deaf people in Serbia as a means for everyday communication, for expressing desires, willingness, for learning, for intellectual discussions, for expressing personal style. Although the standardization of the Serbian Sign Language was completed in 2015, even nowadays we may still find certain gestures of expression varying in different regions. Different countries have different sign languages that are not reciprocally understood in use. They are distinguished by their own grammar (semantics, morphology, and syntax), different from the grammar of spoken languages. The distinguishing and recognition of sign languages in the world has led to changes in the field of education of deaf children. In bilingual schools, children acquire both sign and spoken languages, and teachers know both of the mentioned languages. The importance of sign language in the education of deaf children is emphasized.

Sign is the way of communication when there is no appropriate verbal communication for deaf and dump persons. Sign Language Recognition System (SLRS) acts as a contrivance to perform the conversion of sign language into text or voice. Many classification techniques and algorithms with a variety of methods already exist in SLRS. But still only a few considerable contributions are there in Tamil Sign Language Recognition. The objective of this paper is to extract the features from the pre-processed image and propose an optimization based classifier to improve the classification accuracy in Tamil Sign Language images. The experimental result of the proposed work demonstrated that BPN-PSO outperforms existing classification techniques.

American sign language is an important communication way to convey information among the deaf community in North America and is primarily used by people who have hearing or speech impairments. The deaf community faces a struggle in schools and other institutions because they usually consist primarily of hearing people. Besides, deaf people often feel misunderstood by people who do not knowsign language, for example, family members. In the last two decades, researchers have been proposing automatic sign language recognition systems to facilitate the learning of sign language, and nowadays, computer scientists have focused on using artificial intelligence in order to develop a system capable of reducing the communication gap between hearing and deaf people. In this paper, it is proposed a siamese convolutional neural network for American sign language alphabet recognition. This siamese architecture allows the computer to reduce the high interclass similarity and high intraclass variations. The results show that the proposed method outperforms the state of the art systems.

An optical modeless Sign Language Recognition (SLR) system is presented. The system uses the HAusdorf-Voronoi NETwork (HAVNET), an artificial neural network designed for 2D binary pattern recognition. It uses adaptation of the Hausdorff distance to determine the similarity between an input pattern and a learned representation. A detailed review of the architecture, the learning equations, and the recognition equations for the HAVNET network are presented. Competitive learning has been implemented in training the network using a nearest-neighbor technique. The SLR system is applied to the optical recognition of 24 static symbols from the American Sign Language convention. The SLR system represents the target images in a 80 X 80 pixel format. The implemented HAVNET network classifies the inputs into categories representing each of the symbols, using an output layer of 24 nodes. The network is trained with 5 different formats for each symbol and is tested with all 24 symbols in 15 new formats. Results from the SLR system without competitive training show shape identification problems, when distinguishing symbols with similar shapes. Implementation of competitive learning in the HAVNET neural network improved recognition accuracy on this task to 89%. The hand gestures are identified through a window search algorithm. Feature recognition is obtained from edge enhancement by applying a Laplacian filter and thresholding, which provides robustness to pose, color and background variations.

Sign language recognition (SLR) system is helpful for individuals who can't talk or hear. One can easily communicate with this kind of people using SLR system. In this paper we are focusing on SLR approach where we have apply our proposed algorithm to recognize hand gestures of routine words like Morning, Night, Monday, Tuesday and so forth. Total 10 routine words we have recognize using our proposed technique. For recognize hand pose we have utilized skin color detection algorithm. Further we have applied correlation-coefficient algorithm to distinguish closeness features which are further continue in Neuro-fuzzy(NF) classification algorithm to recognize words. Presented algorithm tried on Matlab. With taken of 10 routine words our proposed framework of techniques got 92% correction rate.

This article is dedicated to analyzing advanced approaches in temporal modeling and real-time gesture recognition within sign language recognition (SLR) systems. Sign glosses are expressed through the spatio-temporal characteristics of visual information, which requires the use of sequence-processing models for their automatic recognition. The study primarily evaluates the effectiveness of three key models: Long Short-Term Memory (LSTM) networks, Temporal Convolutional Networks (TCN), and Transformer-based architectures. The article also examines methods applied for real-time analysis of sign glosses, including: Sliding window segmentation of video streams; Self-attention mechanisms for identifying dependencies between gestures; Gloss mapping algorithms for linking sign movements to linguistic units; Ontological integration techniques for enhancing semantic accuracy. Practical results indicate that combining temporal modeling with semantic analysis and contextual verification algorithms ensures continuous and high-accuracy recognition of sign movements. In particular, multimodal systems (video + sensor + gloss) utilizing Transformer-based approaches achieved superior performance in real-time conversion of continuous sign gloss streams into text. The findings of this study hold practical significance for the development of smart assistive devices for automatic sign language translation, interactive interfaces for hearing-impaired users, and specialized SLR platforms for educational and instructional purposes.

This paper is concerned with the automatic recognition of German continuous sign language. For the most user-friendliness only one single color video camera is used for image recording. The statistical approach is based on the Bayes decision rule for minimum error rate. Following speech recognition system design, which are in general based on subunits, here the idea of an automatic sign language recognition system using subunits rather than models for whole signs will be outlined. The advantage of such a system will be a future reduction of necessary training material. Furthermore, a simplified enlargement of the existing vocabulary is expected. Since it is difficult to define subunits for sign language, this approach employs totally self-organized subunits called fenone. K-means algorithm is used for the definition of such fenones. The software prototype of the system is currently evaluated in experiments.

The development of Human-Computer Interaction (HCI) will not only empower the disabled with access to technology but also help to build tools for assisting the disable and their caregivers. Some disabled people are deaf, they use signs to communicate. Signs are emblematic forms which convey a specific meaning. These emblems construct the sign language, which is a non-verbal language used by the deaf community. This research attempts to build a Bangla Sign language recognition system that can recognize signs of both hands. Hands vary in shapes and sizes, as well as signs vary in orientations and motions. Accurate feature extraction is necessary for such systems. For such purposes, deep learning approaches can prove to be effective for the classification and feature engineering of images. In the beginning, an integrated CNN-LSTM model is proposed for building a Sign language Recognition System, a Bangla Sign Language (BSL) dataset consisting of Bangla lexical signs is considered. This dataset consists of 13,400 images comprising thirty-six classes of Bangla lexical signs. The model is trained using a CNN-LSTM model. This model produces a training accuracy of 90% and a testing accuracy of 88.5%. The proposed model is compared to CNN and other CNN state of the art models, namely VGG16, VGG9 and MobileNet. The CNN model and other CNN state undergo the problem of overfitting as their training accuracy is greater than the testing accuracy.

Sign Language Recognition (SLR) has become an essential area of research due to its potential to promote understanding between the deaf and hearing communities through communication. This paper provides an in-depth study of various methodologies and models employed in SLR, focusing on Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). We analyze their application to datasets from various sign languages, such as Arabic Sign Language (ArSL), American Sign Language (ASL), and British Sign Language (BSL), and explore how these models improve the recognition of dynamic, multi-dimensional hand gestures. This research not only advances the understanding of deep learning applications in sign language recognition but also addresses critical challenges in data processing and real-time applications, paving the way for inclusive technologies in informatics and human-computer interaction. Despite the progress in applying deep learning techniques to SLR, several challenges remain, particularly in dataset limitations, handling large vocabularies, and ensuring consistent performance across diverse environments and signers. The paper also investigates the broader applications of SLR, such as virtual reality, healthcare, education, and accessibility, and discusses the integration of SLR with human-computer interaction systems. Furthermore, it highlights current limitations in the field, such as difficulties with video data handling, the need for standard datasets, and issues related to training computational models. Finally, the paper outlines future research directions, including developing more robust SLR systems that can function effectively in uncontrolled environments, improving data collection methodologies, and creating real-time, user-friendly applications to assist the community of deaf and hard-of-hearing individuals.

Sign Language Recognition (SLR) systems have gained significant popularity in recent years. Despite the existence of various Deep Learning (DL) models to classify sign languages, the deployment of these models on resource-constrained devices remains a challenge. Majority of the research on SLR focuses on popular sign languages, such as American Sign Language (ASL), German Sign Language (GSL) etc., while Kannada Sign Language (KSL) remains significantly underexplored. This study focuses on creating a word-level custom dataset of 33 classes consisting of 2319 videos and training three DL models, namely, LSTM, BiLSTM, and Encoder-only Transformer for dynamic KSL recognition. Trained models were then optimized with quantization techniques and converted to .tflite format, for efficient deployment on resource-constrained devices like smartphones. Dynamic Post Training Quantization (PTQ) model, for the BiLSTM model achieved the highest accuracy of 95.71%, followed by the LSTM model with 94.7%, and the Transformer-based architecture achieved 94.19%. Transformer model deployed on smartphone achieved the smallest inference time of 16.2 ms with a model size of 1097.77 KB, followed by LSTM recording 19.6 ms as inference time with a model size of 1254.49 KB and BiLSTM recording the highest inference time of 27.8 ms with a model size of 3862.01 KB.

Sign language is the main communication language for deaf and dumb people. There have many difficulties for deaf and dumb people when they communicate with normal people. Only a few people can use Sign Language proficiently and most of the people don't know how to communicate the deaf people. From the point of view of computer vision, we can develop the sign language recognition for deaf and dumb people. Most of the sign language recognition systems are based on the static sign. In motion-based sign language recognition system, we proposed feature extraction method and recognition using machine learning. Every nation has its own sign language and our proposed system is based on Myanmar National Sign Language.

The population of deaf-mute in Taiwan is increasing every year. These deaf-mute persons usually use sign language to communicate with each other. However, most hearing persons cannot understand sign languages because they have not learned it. In order to let normal persons understand sign language, a sign language recognition system is necessary. In this paper, such a sign language recognition system is proposed. In the proposed system, a depth image is captured by Kinect sensor, palm area is segmented from depth image, palm binary image is thresholded by using Otsu thresholding method, and background noises are removed by morphological closing operators, and SURF features and descriptors are extracted to identify the sign language. Experimental results show that the proposed method is effective to detect and identify the numerals and letters in the English Alphabet used in the sign language.

Hand gestures help individuals communicate in daily life. Sign languages, however, employ hand motions extensively. People with hearing impairments use sign language. Pattern recognition in computer vision may be used to interpret and translate Arabic Sign Language (ArSL) for deaf and dumb persons using image processing-based software systems. Arab hearing-impaired and deaf persons communicate with Arabic Sign Language Recognition (ArSLR).We propose an image-based ArSLR system employing gesture recognition to enable people with disabilities to engage with the outside world. This work proposes the design and development of an automated Arabic sign language-to-Arabic text translation system. This study extracts stem Arabic words and differentiates related terms. Hand motions capture alphabet letters in the proposed approach. Image isolation occurs after acquisition. Extraction, conclusion, and hand sign pattern characteristics comprise the third step. The fourth phase classifies hand gestures using a powerful classification algorithm. Finally, the sign language letter is interpreted and translated to the Arabic character.