ADVANCEMENT IN SIGN LANGUAGE RECOGNITION TECHNOLOGIES: A COMPREHENSIVE REVIEW

A major challenge faced by our society is the difficulty individuals with disabilities encounter in expressing their emotions to others. People with disabilities often depend on sign (gesture) languages for communication. This project aims to develop a model capable of recognizing and translating sign language alphabets from hand gestures into text and audio. The primary objective is to enhance communication between individuals with hearing impairments and the broader society. We assessed our model's performance using a publicly accessible Indian Sign Language (ISL) dataset. For image classification, we employed Convolutional Neural Networks (CNNs) with the Inception V3 architecture. Hand gestures are captured through webcams, and our model identifies the corresponding alphabet for each gesture. This project seeks to overcome current challenges in Sign Language Recognition and aims to enhance its effectiveness and efficiency.

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

Sign language identification and recognition technique is composed by the gesture images detection and the hand gestures recognition. Gesture images detection is to locate the image part of palm and fingers from the captured pictures, and rotating them to the appropriate gesture posture. Both of them are the important pre-processing for sign language identification and recognition. Lose them, the correctness rate of the sign language recognition algorithms will be dropped down to an unacceptable level. This paper presents novel processing algorithms for the gesture images detection and recognition. In the detection process, it rotates an askew gesture to right position, and to delete the elbow and forearm parts from the captured pictures. In the recognition process, it includes two phases with the model construction and the sign language identification. In the model construction phase, the static hand gesture of sign language is constructed by the Gaussian mixture model, and the unknown gesture image is identified by Gaussian model match. Based on this presented static sign language detection and recognition algorithms, the correct recognition rate is about 94% in average.

A lightweight network-based sign language robot with facial mirroring and speech system

Barriers in expressive and receptive communication for individuals with hearing and speech impairments remain a significant challenge to achieving equitable social interaction and digital accessibility. These limitations restrict their participation in everyday conversations, education, and professional environments, emphasizing the urgent need for intelligent assistive communication technologies. This research introduces an AI-driven real-time Indian Sign Language (ISL) recognition framework that integrates advanced computer vision and deep learning techniques to translate hand gestures into textual outputs. The primary objective of this study is to design a lightweight, accurate, and real-time sign language translator suitable for Divyangjan, ensuring inclusivity in educational and social interactions. The proposed research employs Mediapipe-based hand landmark extraction to detect 21 key points in each gesture frame, followed by preprocessing and normalisation to create robust spatial representations. A hybrid Convolutional Neural Network processes these landmark vectors through a Bidirectional Long Short-Term Memory (CNN–BiLSTM) model that captures both spatial and temporal dependencies in gesture motion, allowing it to recognise static and dynamic gestures such as J and Z. The system was trained on a self-collected dataset of over 26,000 gesture images covering all 26 ISL alphabets. Experimental analysis demonstrates that the proposed model achieved 97.8 percent accuracy, 97.4 percent precision, 97.2 percent recall, 97.3 percent F1-score, and a validation loss of only 0.08, outperforming traditional classifiers such as Random Forest and SVM by a significant margin. The trained model performs robustly under varying lighting, background, and hand orientation conditions, ensuring high reliability for real-world deployment. The novelty of this study lies in the fusion of Mediapipe landmark extraction with a temporal deep learning framework for continuous ISL gesture translation within an interactive CustomTkinter GUI. This human-centric, computationally efficient design enables accessible, real-time communication for the deaf and hard-of-hearing community, contributing to socially assistive AI systems that promote digital inclusivity and empowerment for Divyangjan.

Abstract— Sign language is an essential mode of communication for individuals with hearing impairments, yet real-time translation remains a challenge due to complex hand gestures, facial expressions, and language variations. Traditional deep learning approaches, such as CNNs and RNNs, struggle with sequential dependencies and spatial feature extraction, limiting recognition accuracy. Recent advancements in Vision Transformers have significantly improved image-based learning by utilizing self- attention mechanisms to capture both spatial and temporal dependencies, making them highly effective for gesture recognition. This paper presents a bidirectional Sign Language Recognition and Translation System that employs ViT for sign recognition and a Pre-Recorded Gesture Database for text-to-sign conversion. The system captures real-time video input, extracts gesture features using ViT’s attention-based encoding, and converts recognized gestures into text. Conversely, it maps typed text to a corresponding pre-recorded sign animation, ensuring smooth and natural communication. By eliminating the need for gloss-based intermediaries and improving processing efficiency, the proposed system enhances accuracy, computational efficiency, and real-time performance, offering a scalable. solution for bridging communication gaps in the hearing-impaired community. KEYWORDS— SIGN LANGUAGE RECOGNITION, VISION TRANSFORMER, GESTURE-TO-TEXT, TEXT- TO- GESTURE, DEEP LEARNING, TRANSFORMER MODELS, REAL-TIME TRANSLATION

Sign language recognition is an assistive technology that has garnered significant attention from researchers, particularly with respect to its potential benefits for individuals with hearing impairments. This paper proposes an effective technique for sign language recognition based on the Contourlet Transform (CT) and deep learning. The CT is employed in the pre-processing stage to reduce complexity and processing time, while deep learning is utilized to extract and classify sign language features. The proposed method was evaluated using two sign language databases: a direct feed database and an American sign language database. The experimental analysis demonstrated that the proposed method gives good results in processing time by more than 70% while maintaining high accuracy

Sign Language Recognition (SLR) has become an essential area of research due to its potential to promote understanding between the deaf and hearing communities through communication. This paper provides an in-depth study of various methodologies and models employed in SLR, focusing on Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN). We analyze their application to datasets from various sign languages, such as Arabic Sign Language (ArSL), American Sign Language (ASL), and British Sign Language (BSL), and explore how these models improve the recognition of dynamic, multi-dimensional hand gestures. This research not only advances the understanding of deep learning applications in sign language recognition but also addresses critical challenges in data processing and real-time applications, paving the way for inclusive technologies in informatics and human-computer interaction. Despite the progress in applying deep learning techniques to SLR, several challenges remain, particularly in dataset limitations, handling large vocabularies, and ensuring consistent performance across diverse environments and signers. The paper also investigates the broader applications of SLR, such as virtual reality, healthcare, education, and accessibility, and discusses the integration of SLR with human-computer interaction systems. Furthermore, it highlights current limitations in the field, such as difficulties with video data handling, the need for standard datasets, and issues related to training computational models. Finally, the paper outlines future research directions, including developing more robust SLR systems that can function effectively in uncontrolled environments, improving data collection methodologies, and creating real-time, user-friendly applications to assist the community of deaf and hard-of-hearing individuals.

Sign language recognition, as an interdisciplinary field involving computer vision, pattern recognition, and natural language processing, holds profound research significance and extensive application value. This technology not only helps people with hearing impairments and those with normal hearing achieve barrier-free communication, but it also enhances their daily living experience while driving the development of sciences such as computer vision and artificial intelligence technologies. The subsequent text offers a thorough examination of the technologies involved in sign language recognition. It starts by detailing the methods for gathering data in sign language recognition, giving particular attention to hand modeling and the techniques used for visual feature extraction. Then, it discusses in detail the two methods of sign language recognition, namely traditional methods and artificial intelligence methods. These two methods have their advantages and disadvantages, providing different ideas for developing sign language recognition technology. Finally, the article proposes a prospect for the future development of sign language recognition technology, hoping that it can play a significant role in more fields and create a more convenient and barrier-free communication environment for people with hearing impairments.

A survey on recent advances in Sign Language Production

Speech impairment limits a person’s capacity for oral and auditory communication. Improvements in communication between the deaf and the general public can be progressed by a real-time sign language detector. Recent studies have contributed to make progress in motion and gesture identification processes using Deep Learning (DL) methods and computer vision. But the development of static and dynamic sign language recognition (SLR) models is still a challenging area of research. The difficulty is in obtaining an appropriate model that addresses the challenges of continuous signs that are independent of the signer. Different signers’ speeds, durations, and many other factors make it challenging to create a model with high accuracy and continuity. This study mainly focused on SLR using a modified DL and hybrid optimization approach. Notably, spatial and geometric-based features are extracted via the Visual Geometry Group 16 (VGG16), and motion features are extracted using the optical flow approach. A new DL model, CNNSa-LSTM, is a combination of a Convolutional Neural Network (CNN), Self-Attention (SA), and Long-Short-Term Memory (LSTM) to identify sign language. This model is developed for feature extraction by combining CNNs for spatial analysis with SA mechanisms for focusing on relevant features, while LSTM effectively models temporal dependencies. The proposed CNNSa-LSTM model enhances performance in tasks involving complex, sequential data, such as sign language processing. Besides, a Hybrid Optimizer (HO) is proposed using the Hippopotamus Optimization Algorithm (HOA) and the Pathfinder Algorithm (PFA). The proposed model has been implemented in Python, and it has been evaluated over the existing models in terms of accuracy (98.7%), sensitivity (98.2%), precision (98.5%), Word Error Rate (WER) (0.131), Sign Error Rate (SER) (0.114), and Normalized Discounted Cumulative Gain (NDCG) (98%) as well. The proposed model has recorded the highest accuracy of 98.7%.

Sign language is mandatory and mostly used by huge number of individuals (particularly disabled) in society and it is another way of communication; The form of the hand, the contour of the movement, and the posture of the hand, face, and body parts are all distinct in each sign language; as a result, visual sign language identification is a challenging area of computer vision research; various academics have suggested several models in recent years that have been greatly improved using deep learning approaches. This study looked at the vision-based sign language recognition models suggested utilizing deep learning approaches in the last five years. Although the suggested model’s general trend reveals that sign language recognition accuracy has substantially increased, specific problems still have to be overcome. There are various and multiple languages in the world, people from different languages, have their alphabets and sign to communicate with each other. Many language speakers have developed a way to communicate with disabled people like Indian sign Language, French sign language several more; In this research paper, a novel Pashto sign language has been worked on. The most spoken language in southern Afghanistan is the Pashto language, and it has its grammar and history; it has approximately 40 million population in the world. The aim is to make a system by which disabled people of this language can communicate with the society. The system will take the signs and gestures of disabled people to change them into the Alphabet of Pashto language. 2500 dataset images are prepared and CNN model i s a ppl i e d to recognize the hand gestures and have executed the model with an accuracy of 98%.

Aiming at the feature of signer-independent sign language recognition the training data complexity caused by mass data and noticeable distinctions between different people data, the weighted KNN/HMM model is presented in this paper. This model is made of two blocks, which is part of sign language classification and recognition. In classing part, the KNN (K-Nearest Neighbor, KNN) is used to learn the training samples. Considering the different contributions of sign language features to pattern classification we give different weight to different characteristics. And the category of test sample is decided by the sum of weighted distance. In recognition part, weighted KNN classification result is taken as the state-input of HMM (Hidden Markov models, HMM) to implement sign language recognition, combine with the ability to temporal data modeling and fuzzy inference of HMM model. Experiment results show that weighted KNN/HMM sign language recognition model are efficient on either recognition speed or recognition rate.

Communication is essential to humans because it allows the dissemination of knowledge and the formation of interpersonal connections. We communicate through speaking, facial expressions, hand gestures, reading, writing, and sketching, among other things. However, speaking is the most often utilized means of communication. People having speech and hearing disabilities can only communicate using hand gestures, making them extremely reliant on nonverbal modes of communication. Hearing-impaired persons can communicate via sign language. Globally, around 1 percent(5 million) of the Indian population falls into this group. ISL is a complete language with its own vocabulary, semantics, lexicon, and a variety of other distinctive linguistic features. In our work, we present the methods for Indian sign language recognition at the character and word levels. The Bag of Visual Words(BoVW) technique recognizes ISL at character level(A-Z, 0-9) with an accuracy of 99 percent. Indian Lexicon Sign Language Dataset - INCLUDE-50 dataset is used for word-level sign language recognition. Inception model, a deep Convolutional Neural Network(CNN) is used to train the spatial features and LSTM RNN(Recurrent Neural Network) is used to train the temporal features of the video. Using CNN predictions as input to RNN, we achieved an accuracy of 86.7 %. In order to optimize the training process, only 60 % of the dataset is trained using the Meta-Learning model along with LSTM RNN and obtained an accuracy of 84.4 %, thus reducing the training time by 70 % and reaching nearly as close accuracy as the previous pre-trained model.

The move to standardize Indian Sign Language has created an opportunity for researchers to focus on solving local problems, to increase its reach. In this paper, a survey and assessment of the techniques applied to the recognition and conversion of Indian Sign Language are performed. An overview of the techniques used in sign language recognition for Indian Sign Language is provided to understand the status of research in this field. Following this, a comparison of techniques aimed at rendering a more detailed picture of the research results is presented. The challenges faced by researchers, the limitations of current techniques, and the need for improved research in this area are highlighted. With the intent of spurring more in-depth research, key areas within the approaches and techniques in need of improvement are summarized.