Static Alphabet American Sign Language Recognition using Convolutional Neural Networks

Sign Language Recognition (SLR) has become an essential area of research due to its potential to promote understanding between the deaf and hearing communities through communication. This paper provides an in-depth study of various methodologies and models employed in SLR, focusing on Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). We analyze their application to datasets from various sign languages, such as Arabic Sign Language (ArSL), American Sign Language (ASL), and British Sign Language (BSL), and explore how these models improve the recognition of dynamic, multi-dimensional hand gestures. This research not only advances the understanding of deep learning applications in sign language recognition but also addresses critical challenges in data processing and real-time applications, paving the way for inclusive technologies in informatics and human-computer interaction. Despite the progress in applying deep learning techniques to SLR, several challenges remain, particularly in dataset limitations, handling large vocabularies, and ensuring consistent performance across diverse environments and signers. The paper also investigates the broader applications of SLR, such as virtual reality, healthcare, education, and accessibility, and discusses the integration of SLR with human-computer interaction systems. Furthermore, it highlights current limitations in the field, such as difficulties with video data handling, the need for standard datasets, and issues related to training computational models. Finally, the paper outlines future research directions, including developing more robust SLR systems that can function effectively in uncontrolled environments, improving data collection methodologies, and creating real-time, user-friendly applications to assist the community of deaf and hard-of-hearing individuals.

Sign Language Recognition (SLR) targets on interpreting the sign language into text or speech, so as to facilitate the communication between deaf-mute people and ordinary people. The task has broad social impact, but is still very challenging due to the complexity and large variations in hand actions. Existing dataset for Sign Language Recognition (SLR) in Bangla Sign Language (BdSL) is based on RGB images. Recent research on sign language recognition has shown better recognition accuracy using depth-based features. In this paper, we present a complete dataset for Bangla sign digits from Zero (Shunno in Bangla) to Nine (Noy in Bangla) using MediaPipe, a cross-platform depth-map estimation framework. The proposed method can utilize hand skeleton joint points containing depth information in addition to x, y coordinates from RGB images only. To validate the effectiveness of our proposed approach, we have run MediaPipe on a benchmark American Sign Language (ASL) dataset. Running different classifiers in our proposed dataset we got 98.65% using Support Vector Machine (SVM). Moreover, we compared our dataset with the existing Bangla digit dataset Ishara Bochon using deep learning based approach and achieved significantly higher accuracy.

The Shape of Air: American Sign Language as Narrative Prosthesis in 21st Century North American Media

The project "Transforming Expression into Language: American Sign Language Recognition using Image Processing" intends to employ image processing techniques to identify gestures in American Sign Language (ASL). The goal of this project is to develop an efficient system for translating ASL signs using visual cues generated from hand motions and gestures. The program intends to use image processing technology to bridge the communication gap between ASL users and nonusers. The use of image processing for ASL identification may improve communication in a range of situations while also increasing accessibility for the deaf community. The paper provides a realistic approach to real-time ASL motion interpretation using image processing techniques, thereby advancing the field of sign language recognition technology. KEYWORDS ASL Recognition, Image Processing Techniques, Convolutional Neural Networks (CNNs), Feature Extraction in Image Processing, Transforming Expressions into Text

A major challenge faced by our society is the difficulty individuals with disabilities encounter in expressing their emotions to others. People with disabilities often depend on sign (gesture) languages for communication. This project aims to develop a model capable of recognizing and translating sign language alphabets from hand gestures into text and audio. The primary objective is to enhance communication between individuals with hearing impairments and the broader society. We assessed our model's performance using a publicly accessible Indian Sign Language (ISL) dataset. For image classification, we employed Convolutional Neural Networks (CNNs) with the Inception V3 architecture. Hand gestures are captured through webcams, and our model identifies the corresponding alphabet for each gesture. This project seeks to overcome current challenges in Sign Language Recognition and aims to enhance its effectiveness and efficiency.

Contents: Preface. Part I: Reminiscences. L. Fant, Two Memorable Meals With Ursula and Ed. R.M. Battison, American Sign Language Linguistics 1970-1980: Memoir of a Renaissance. B. Bragg, A Fish Story for Ursula and Other Remembrances. Part II: Historical and Comparative Analyses of Sign Languages. J. Woodward, Sign Languages and Sign Language Families in Thailand and Vietnam. D. McKee, G. Kennedy, Lexical Comparisons of Signs From American, Australian, British, and New Zealand Sign Languages. H. Lane, R.C. Pillard, M. French, Origins of the American Deaf-World: Assimilating and Differentiating Societies and Their Relation to Genetic Patterning. Part III: Language in the Visual-Spatial Modality. E.L. Newport, T. Supalla, Sign Language Research at the Millennium. P. Siple, Attentional Resources and Working Memory: A New Framework for the Study of the Impact of Deafness on Cognition. M. Wilson, K. Emmorey, When Does Modality Matter? Evidence From ASL on the Nature of Working Memory. P.B. Braem, T. Bram, A Pilot Study of the Expressive Gestures Used by Classical Orchestra Conductors. N. Frishberg, An Interpreter Creates the Space. Part IV: Linguistic Analysis of Sign Languages. S.D. Fischer, More Than Just Handwaving: The Mutual Contributions of Sign Language and Linguistics. R.B. Wilbur, Phonological and Prosodic Layering of Nonmanuals in American Sign Language. R.C. Loew, C.T. Akamatsu, M. Lanaville, A Two-Handed Manual Alphabet in the United States. E. Pizzuto, V. Volterra, Iconicity and Transparency in Sign Languages: A Cross-Linguistic Cross-Cultural View. M. Kuntze, Codeswitching in ASL and Written English Language Contact. S.K. Liddell, Indicating Verbs and Pronouns: Pointing Away From Agreement. Part V: Language Acquisition. J.G. Bettger, Viewing Deaf Children in a New Way: Implications of Bellugi and Klima's Research for Education. R.P. Meier, Shared Motoric Factors in the Acquisition of Sign and Speech. A. Abrahamsen, Explorations of Enhanced Gestural Input to Children in the Bimodal Period. D. Lillo-Martin, Early and Late in Language Acquisition: Aspects of the Syntax and Acquisition of Wh-Questions in American Sign Language. J. Reilly, Bringing Affective Expression Into the Service of Language: Acquiring Perspective Marking in Narratives. C.A. Padden, V.L. Hanson, Search for the Missing Link: The Development of Skilled Reading in Deaf Children. L.A. Petitto, On the Biological Foundations of Human Language. Part VI: The Neural Organization of Sign Language. A.R. Damasio, H. Damasio, Language and the Brain. D.P. Corina, Some Observations Regarding Paraphasia in American Sign Language. H. Poizner, D. Brentari, M.E. Tyrone, J. Kegl, The Structure of Language as Motor Behavior: Clues From Signers With Parkinson's Disease. V.A. Fromkin, On the Uniqueness of Language.

As a natural language, sign language is a rich and expressive form of communication with properties similar to spoken languages. In this paper, we explore the application of transfer learning techniques to develop an intelligent system for recognizing American Sign Language. Instead of building a new deep convolutional neural network from scratch, we leveraged pre-trained models to capitalize on their advantages and reduce experimentation time. Specifically, we focused on YOLOv3 combined with advanced recurrent layers, including LSTM, GRU, and Bidirectional LSTM. The results were highly promising: YOLOv3 + Bidirectional LSTM achieved the highest accuracy, averaging over 94%, followed by YOLOv3 + LSTM at ~93%, and YOLOv3 + GRU at ~91%. These findings highlight the effectiveness of combining YOLOv3 with recurrent architectures for sign language recognition, demonstrating robust precision, recall, and F1 scores across models, with Bidirectional LSTM standing out as the top-performing configuration.

Sign language serves as the primary medium of expression serves as the primary medium of expression for deaf and hard-of-hearing individuals. However, interpersonal interaction remains challenging, as they primarily rely on sign language to express their thoughts. To address this issue, this study proposes an automated sign language recognition and interpretation framework, integrating static and dynamic recognition components. Specifically, a Convolutional Neural Network (CNN) is employed for static gesture classification, while a hybrid CNN- Long Short-Term Memory (CNN-LSTM) architecture is utilized to capture the spatiotemporal features of dynamic signs. Furthermore, MediaPipe is leveraged for robust landmark localization to enhance feature extraction. The American Sign Language (ASL) dataset used in this research ensures diversity in sign representation, including variations in hand shapes, positions, and movements. The proposed models achieved high accuracy, with the CNN model reaching 93.1% and the CNN-LSTM model achieving 94.1% on test datasets, confirming their effectiveness in ASL recognition tasks.

Sign language recognition system for communicating to people with disabilities

Hearing Impaired individuals routinely encounter limitations in their involvement in social interactions, access to intriguing information, and participation in everyday activities, among various other aspects. However, the hardest part of their interactions with regular people is communication, because sign language is the primary language of those who are hearing impaired. However, the general public is unaware of sign language. Each country has its own sign language. However, there are some striking similarities between them. In Sri Lanka, hearing impaired people use Sri Lankan Sign Language (SLSL) as their communication language. There is several research done on Sign Language recognition and translation. But no fully functioning system is utilized for Sri Lankan Sign Language translation. To find the gap in this area, we conducted a systematic literature review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method that analyses 12 studies on Sign Language Translation (SLT). As per the literature review, Image Processing (IP) and Convolutional Neural Networks (CNN) are the most used techniques for Sign Language translation. But these methods have limitations: not enough data, differences in how people use sign language, difficulty in translating in real-time, not capturing cultural aspects, needing specific equipment, and understanding the context of conversations. Recognizing and solving these problems is important, especially for languages like SLSL. Future research should focus on getting more data, making translation work for different cultures, and improving real-time translation. This will help hearing impaired people communicate better with others.

Sign language plays a crucial role as a communication tool for both the deaf and hard-of- hearing communities, enabling them to engage and interact effectively within their own community as well as with others.However, communication barriers arise when individuals unfamiliar with sign language engage with those who rely on it, underscoring the need for inclusive solutions. Real-time sign language interpretation systems, leveraging machine learning and computer vision technologies, present a promising approach to bridging this gap. These systems convert sign language gestures into spoken or written language by utilizing gesture recognition algorithms, neural networks, and natural language processing. By analyzing hand movements, facial expressions, and body language, the systems provide accurate, context-aware translations of various sign languages, such as American Sign Language (ASL), with minimal delay. This enables seamless, natural interactions, making such technologies essential for fostering inclusive communication in diverse settings. Key Words: Sign language recognition, real- time interpretation, machine learning, com- puter vision, gesture recognition, neural networks, natural language processing, communication barriers, inclusivity, American Sign Language (ASL).

A variety of algorithms allows gesture recognition in video sequences. Alleviating the need for interpreters is of interest to hearing impaired people, since it allows a great degree of self-sufficiency in communicating their intent to the non-sign language speakers without the need for interpreters. State-of-theart in currently used algorithms in this domain is capable of either real-time recognition of sign language in low resolution videos or non-real-time recognition in high-resolution videos. This paper proposes a novel approach to real-time recognition of fingerspelling alphabet letters of American Sign Language (ASL) in ultra-high-resolution (UHD) video sequences. The proposed approach is based on adaptive Laplacian of Gaussian (LoG) filtering with local extrema detection using Features from Accelerated Segment Test (FAST) algorithm classified by a Convolutional Neural Network (CNN). The recognition rate of our algorithm was verified on real-life data.

Introduction: Communication with the hearing impaired ( deaf/mute) people is a great challenge in our society today; this can be attributed to the fact that their means of communication (Sign Language or hand gestures at a local level) requires an interpreter at every instance. Conversion of images to text as well as speech can be of great benefit to the non-hearing impaired and hearing impaired people (the deaf/mute) from circadian interaction with images. To effectively achieve this, a sign language (ASL – American Sign Language) image to text as well as speech conversion was aimed at in this research. Methodology: The techniques of image segmentation and feature detection played a crucial role in implementing this system. We formulate the interaction between image segmentation and object recognition in the framework of FAST and SURF algorithms. The system goes through various phases such as data capturing using KINECT sensor, image segmentation, feature detection and extraction from ROI, supervised and unsupervised classification of images with K-Nearest Neighbour (KNN)-algorithms and text-to-speech (TTS) conversion. The combination FAST and SURF with a KNN of 10 also showed that unsupervised learning classification could determine the best matched feature from the existing database. In turn, the best match was converted to text as well as speech. Result: The introduced system achieved a 78% accuracy of unsupervised feature learning. Conclusion: The success of this work can be attributed to the effective classification that has improved the unsupervised feature learning of different images. The pre-determination of the ROI of each image using SURF and FAST, has demonstrated the ability of the proposed algorithm to limit image modelling to relevant region within the image.

A lightweight network-based sign language robot with facial mirroring and speech system

Sign Language Recognition is a vast domain with a variety of solutions available on the market which achieve the same results, translating hand gestures to text and/or audio. There is a plethora of sign languages used globally, but the mostreputed and commonly used sign language is the American Sign Language (ASL). ASL uses hand gestures and finger spelling to convey the English alphabets and words. The two methods of implementing sign language translation are: Image Processing and Gesture Recognition. The gesture recognition gloves available in the current market utilize expensive flex sensors to map the bend of the users’ fingers. This makes the overall cost of the gloves expensive. Through ‘‘ASL Sign Language Recognition Gloves we aim to implement gesture recognition with MEMS accelerometers that are comparatively cheaper and make the glove more affordable.