Classifiers in Arab Gloss Annotation System for Arabic Sign Language

<p><span>Arabic sign language (ArSL) is the natural language of the deaf community in Arabic countries. Deaf people have a set of difficulties due to poor services available. They have problems accessing essential information or receiving an education, communicating with other communities, and engaging in activities. Thus, a machine translation system of Arabic to ArSL has been developed using avatar technologies. Firstly, a dictionary of ArSL was constructed using eSign editor Software. The constructed dictionary has three thousand signs. It can be adopted for the translation system in which written text can be transformed into sign language. The dictionary will be available as a free resource for researchers. It is complex and time-consuming, but it is an essential step in the machine translation of whole Arabic text to ArSL with 3D animations. Secondly, the translator has been developed. It performs syntactic and morphological analysis and then applies a set of rules to translate an Arabic text into ArSL text based on the structure and grammar of ArSL. The system is evaluated according to the parallel corpus that consists of 180 sentences using the metric for evaluation of translation with explicit ordering </span><span lang="IN">metric for evaluation of translation with explicit ordering</span><span>(METEOR) our system achieves a relative score of (86%).</span></p>

Sign language is a common language that deaf people around the world use to communicate with others. However, normal people are generally not familiar with sign language (SL) and they do not need to learn their language to communicate with them in everyday life. Several technologies offer possibilities for overcoming these barriers to assisting deaf people and facilitating their active lives, including natural language processing (NLP), text understanding, machine translation, and sign language simulation. In this paper, we mainly focus on the problem faced by the deaf community in Saudi Arabia as an important member of the society that needs assistance in communicating with others, especially in the field of work as a driver. Therefore, this community needs a system that facilitates the mechanism of communication with the users using NLP that allows translating Arabic Sign Language (ArSL) into voice and vice versa. Thus, this paper aims to purplish our created dataset dictionary and ArSL corpus videos that were done in our previous work. Furthermore, we illustrate our corpus, data determination (deaf driver terminologies), dataset creation and processing in order to implement the proposed future system. Therefore, the evaluation of the dataset will be presented and simulated using two methods. First, using the evaluation of four expert signers, where the result was 10.23% WER. The second method, using Cohen’s Kappa in order to evaluate the corpus of ArSL videos that was made by three signers from different regions of Saudi Arabia. We found that the agreement between signer 2 and signer 3 is 61%, which is a good agreement. In our future direction, we will use the ArSL video corpus of signer 2 and signer 3 to implement ML techniques for our deaf driver system.

Sign language is significant in its linguistic complexity and sociocultural values to its users. The linguistic situation of sign languages in the Middle East is controversial. For Arab deaf communities, each country has a rich and complex national sign language without the need for a superior, more standard unifying Arabic sign language (ArSL). There have been attempts to unify sign languages by creating ArSL. This initiative has been mostly taken by nondeaf persons and has been met with opposition and resistance. This paper investigates sign languages in Kuwait and the attitudes towards both Kuwaiti Sign Language (KSL) and ArSL from both deaf individuals and sign language translators/teachers using surveys, informal interviews, and fieldwork visits. The research question is: What sign language would the deaf community and sign language interpreters/teachers in Kuwait choose (KSL, ArSL, or both) to better serve the deaf community. This paper is designed to uncover the impact of language policies on communities' representations of self and explore the richness and complexity of national sign languages that can be underestimated by language policymakers. The findings of this study indicate a preference for KSL or the use of both KSL and ArSL, but not ArSL alone.

There are more than 10 million deaf people in the Arab nations presently—Egypt alone accounts for 4 million deaf people. Hence, an Arabic sign language is necessary for facilitating communication between the hearing-impaired people and the larger society. However, everyone does not understand this language. Furthermore, the previous systems were designed for SLR, which were extremely expensive, and several sensors were required to interpret all hand and finger motions. For these reasons, a new prototype of the smart glove is produced, which can be used as a part of an electronic system to translate the Arabic sign language into a spoken or written Arabic. This design includes all Arabic sign language signs (words level) by using 4-flex sensors, 4-contact pad sensor, gyroscope, and accelerometer, controller, and Bluetooth Module per hand. The design is driven from a statistical analysis of the Arabic sign language – 1300 words in terms of single hand signs, double hand signs, single-stage signs, multi-stage signs, fingers angle, and orientation. This paper explores the design analysis and implementation of this system. The result of the (53words) samples of Arabic SL signs (involving all classification) is 85% pass from the first time and 15% pass after many trials (double-hand multi-stage signs classification) due to the synchronisation problem of the two hands data. The system is low cost, requires low power, and has a fast response.

<span>The arabic sign language (ArSL) is the natural language of the deaf community in Arabic countries. ArSL suffers from a lack of resources such as unified dictionaries and corpora. In this work, a dictionary of Arabic language to ArSL has been constructed as a part of a translation system. The Arabic words are converted into hamburg notation system (HamNoSys) using eSign editor Software. HamNoSys was used to create manual parameters (handshape, hand orientation, hand location, and hand movement), while non-manual parameters (facial expressions, shoulder raising, mouthing gesture, head tilting, and body movement) added by using (mouth, face, and limbs) in the eSign editor software. The sign then converted to the sign gesture markup language (SiGML) file, and later 3D avatar interprets the SiGML file scripts to the animated sign. The constructed dictionary has three thousand signs; therefore, it can be adopted for the translation system in which written text can be transformed into sign language and can be utilized for the education of deaf people. The dictionary will be available as a free resource for researchers. It is hard and time-consuming work, but it is an essential step in machine translation of whole Arabic text to ArSL with 3D animations. </span>

Sign Language is a widely known medium of communication used by the deaf, hard of hearing individuals, and those who are unable to speak physically because of any reason. There is no common or universal version of sign language. This is especially true for Arabic Sign Language (ArSL) which has various versions depending on the Arab country where it is being used. In this paper, we have designed an intelligent system which can parse Arabic text, keeping in mind the context of the text, and then translate the text into Arabic Sign Language. Preliminary testing of the system has shown an overall accuracy of 85% in correctly translating Arabic words into the Arabic Sign Language.

Deaf and hearing-impaired individuals who communicate using sign language face several communication difficulties. Because the vast majority of people do not know sign language, the need for a sign language translator is growing significantly, especially for Arabic sign language (ArSL). Today, technology plays a significant role in people’s lives. Leap Motion technology can be used to address the aforementioned issues and improve communication between Saudi Arabia’s deaf and hearing individuals. In this study, we investigated the possibility of using a Leap Motion system to provide continuous ArSL recognition for two-way communication to improve communication between deaf and hearing individuals in terms of speed and independence. The system translates ArSL into spoken words for hearing individuals and transcribes spoken Arabic language into text for deaf individuals.

Arabic sign language (ArSL) is a full natural language that is used by the deaf in Arab countries to communicate in their community. Unfamiliarity with this language increases the isolation of deaf people from society. This language has a different structure, word order, and lexicon than Arabic. The translation between ArSL and Arabic is a complete machine translation challenge, because the two languages have different structures and grammars. In this work, we propose a rule-based machine translation system to translate Arabic text into ArSL. The proposed system performs a morphological, syntactic, and semantic analysis on an Arabic sentence to translate it into a sentence with the grammar and structure of ArSL. To transcribe ArSL, we propose a gloss system that can be used to represent ArSL. In addition, we develop a parallel corpus in the health domain, which consists of 600 sentences, and will be freely available for researchers. We evaluate our translation system on this corpus and find that our translation system provides an accurate translation for more than 80% of the translated sentences.

Sign language is used as a primary means of communication by millions of people who suffer from hearing problems. The unhearing people used visual language to interact with each other, Represented in sign language. There are features that the hearing impaired use to understand each other, which are difficult for normal people to understand. Therefore, deaf people will struggle to interact with society. This research aims to introduce a system for recognizing hand gestures in Arabic Sign Language (ArSL) through training the Convolutional Neural Network (CNN) on the images of ArSL gestures launched by the University of Prince Mohammad Bin Fahd, Saudi Arabia. A Scale Invariant Feature Transform (SIFT) algorithm is used for creating the feature vectors that contain shape, finger position, size, center points of palm, and hand margin by extracting the Important features for images of ArSL and transforming them to points of the vector. The accuracy of the proposed system is 97% using the SIFT with CNN, and equal to 94.8% nearly without SIFT. Finally, the proposed system was tried and tested on a group of persons and its effectiveness was proven after considering their observations.

Sign Language Recognition by using Spatio-Temporal Features

The Arabic Sign Language (ArSL) plays a vital role in facilitating communication for deaf and hard-of-hearing individuals within Arabic-speaking communities, serving as a primary medium for social interaction, education, and access to services. However, the automatic recognition of ArSL remains a significant challenge due to its linguistic complexity, lack of standardized datasets, and the substantial variation in signing styles across regions, age groups, and social contexts. To address these challenges, we propose a word-level ArSL recognition system based on the transformer-based VideoMAE (masked autoencoder) model, which is pre-trained on the Kinetics-400 dataset and fine-tuned on a curated ArSL video dataset. We employ data augmentation techniques, such as downsampling, color inversion, and multiplication, which simulate real-world variability in gesture appearance and recording conditions, to enhance the model’s adaptability to real-world signing variations. Our system achieved a 97% accuracy, outperforming the 49% baseline and surpassing the traditional convolutional neural network-, recurrent neural network-, and long short-term memory-based approaches. Additionally, the system demonstrated consistent performance across 15 data subsets and supported real-time inference at 30 frames per second on GPU hardware. This performance demonstrates the effectiveness of VideoMAE’s transformer-based architecture and data augmentation in capturing the spatial–temporal patterns of sign language. It offers a robust foundation for future research in sentence-level and real-time sign language translation. Lastly, the results contribute to the development of inclusive communication technologies for Arabic-speaking deaf and hard-of-hearing communities.

Social Networking Sites (SNSs) impact on deaf and hard of hearing people (D/HoH) and characteristics of deaf people in Jordan were investigated in this paper. The main purpose of this paper is to show whether deaf people can effectively use and communicate through SNSs. To this end, a questionnaire was used to find the preferences and experiences of deaf people on SNSs. It was distributed to 138 deaf people. Furthermore, some demographic characteristics of the sample were discussed in the paper. This survey shows how SNSs are integrated with deaf people as a communication tool and the positive impact resulted from using them. It was found that Arabic sign language is most frequently used more than any other communication mode. In addition, they communicate with individuals with hearing loss using sign language more often than using written and spoken language. It was reported that they used SNSs once or more times per day. Furthermore, the most used SNSs website is Facebook, then WhatsApp comes second. Smartphones are used more frequently than any other devices. Last but not least, they use SNSs to exchange photos and watch videos more often than any other purposes and this is highly related to their reading and writing skills.

This study aims to propose a low-cost, smart glove-based sensory recognition system. This involves designing a glove containing multiple sensors to record data on static signs in Arabic Sign Language. The research was divided into several successive phases. The first phase included an analysis of the hand structure and movement in Arabic Sign Language to extract essential features that would help define the requirements of the recognition system. The study examined the language from several morphological and kinetic perspectives. The second phase focused on the design and development of the DataGlove through iterative refinement processes based on the requirements of the first phase, with the goal of improving its efficiency and performance. In the third phase, a sensory dataset was created based on the proposed glove, adopting a clear methodology for data collection and processing. The subsequent phase involved building a recognition system using machine learning algorithms, with a thorough evaluation of the effectiveness of the proposed methodology. The glove enabled the production of new, large-scale datasets for Arabic Sign Language (ARSL) using a sensory approach. The results demonstrated a number of significant benefits. The experiments demonstrated that the data glove design is an effective means of identifying similarities in hand postures and recording a wide variety of movements. The experiments achieved high accuracy, reaching 100%, in recognizing words, letters, and numbers using the Extra Trees algorithm. These results confirm that the proposed system achieves high recognition rates, and that the ARSL dataset will provide a supportive foundation for future research in Arabic sign language recognition systems. The results are encouraging, contributing to bridging the communication gap between the deaf and hard-of-hearing community and society.

Arabic sign language (ArSL) is a visual-manual language which facilitates communication among Deaf people in the Arabic-speaking nations. Recognizing the ArSL is crucial due to variety of reasons, including its impact on the Deaf populace, education, healthcare, and society, as well. Previous approaches for the recognition of Arabic sign language have some limitations especially in terms of accuracy and their capability to capture the detailed features of the signs. To overcome these challenges, a new model is proposed namely DeepArabianSignNet, that incorporates DenseNet, EfficientNet and an attention-based Deep ResNet. This model uses a newly introduced G-TverskyUNet3+ to detect regions of interest in preprocessed Arabic sign language images. In addition, employing a novel metaheuristic algorithm, the Crisscross Seed Forest Optimization Algorithm, which combines the Crisscross Optimization and Forest Optimization algorithms to determine the best features from the extracted texture, color, and deep learning features. The proposed model is assessed using two databases, the variation of the training rate was 70% and 80%; Database 2 was exceptional, with an accuracy of 0.97675 for 70% of the training data and 0.98376 for 80%. The results presented in this paper prove that DeepArabianSignNet is effective in improving Arabic sign language recognition.

Hand gestures help individuals communicate in daily life. Sign languages, however, employ hand motions extensively. People with hearing impairments use sign language. Pattern recognition in computer vision may be used to interpret and translate Arabic Sign Language (ArSL) for deaf and dumb persons using image processing-based software systems. Arab hearing-impaired and deaf persons communicate with Arabic Sign Language Recognition (ArSLR).We propose an image-based ArSLR system employing gesture recognition to enable people with disabilities to engage with the outside world. This work proposes the design and development of an automated Arabic sign language-to-Arabic text translation system. This study extracts stem Arabic words and differentiates related terms. Hand motions capture alphabet letters in the proposed approach. Image isolation occurs after acquisition. Extraction, conclusion, and hand sign pattern characteristics comprise the third step. The fourth phase classifies hand gestures using a powerful classification algorithm. Finally, the sign language letter is interpreted and translated to the Arabic character.