Hybrid feature based deep ensemble approach for Indian sign language recognition

Sign languages are crucial in promoting independent living for deaf and hard-of-hearing individuals. They offer a means of communication that facilitates access to various aspects of daily life and supports autonomy. The deaf community in India, estimated at over 18 million individuals, faces significant communication barriers due to the limited use of Indian sign language (ISL) by the general population. This chapter explores the potential of machine learning (ML) for Indian sign language detection as a bridge between the deaf and hearing communities. The authors delve into existing research on sign language recognition (SLR) systems, analyze the challenges specific to ISL detection, and propose future directions leveraging advancements in ML. By examining the sociolinguistic landscape and the potential impact of ML-powered ISL detection, this chapter aims to contribute to a more inclusive society.

The development of Human-Computer Interaction (HCI) will not only empower the disabled with access to technology but also help to build tools for assisting the disable and their caregivers. Some disabled people are deaf, they use signs to communicate. Signs are emblematic forms which convey a specific meaning. These emblems construct the sign language, which is a non-verbal language used by the deaf community. This research attempts to build a Bangla Sign language recognition system that can recognize signs of both hands. Hands vary in shapes and sizes, as well as signs vary in orientations and motions. Accurate feature extraction is necessary for such systems. For such purposes, deep learning approaches can prove to be effective for the classification and feature engineering of images. In the beginning, an integrated CNN-LSTM model is proposed for building a Sign language Recognition System, a Bangla Sign Language (BSL) dataset consisting of Bangla lexical signs is considered. This dataset consists of 13,400 images comprising thirty-six classes of Bangla lexical signs. The model is trained using a CNN-LSTM model. This model produces a training accuracy of 90% and a testing accuracy of 88.5%. The proposed model is compared to CNN and other CNN state of the art models, namely VGG16, VGG9 and MobileNet. The CNN model and other CNN state undergo the problem of overfitting as their training accuracy is greater than the testing accuracy.

Communication is essential to humans because it allows the dissemination of knowledge and the formation of interpersonal connections. We communicate through speaking, facial expressions, hand gestures, reading, writing, and sketching, among other things. However, speaking is the most often utilized means of communication. People having speech and hearing disabilities can only communicate using hand gestures, making them extremely reliant on nonverbal modes of communication. Hearing-impaired persons can communicate via sign language. Globally, around 1 percent(5 million) of the Indian population falls into this group. ISL is a complete language with its own vocabulary, semantics, lexicon, and a variety of other distinctive linguistic features. In our work, we present the methods for Indian sign language recognition at the character and word levels. The Bag of Visual Words(BoVW) technique recognizes ISL at character level(A-Z, 0-9) with an accuracy of 99 percent. Indian Lexicon Sign Language Dataset - INCLUDE-50 dataset is used for word-level sign language recognition. Inception model, a deep Convolutional Neural Network(CNN) is used to train the spatial features and LSTM RNN(Recurrent Neural Network) is used to train the temporal features of the video. Using CNN predictions as input to RNN, we achieved an accuracy of 86.7 %. In order to optimize the training process, only 60 % of the dataset is trained using the Meta-Learning model along with LSTM RNN and obtained an accuracy of 84.4 %, thus reducing the training time by 70 % and reaching nearly as close accuracy as the previous pre-trained model.

Indian Sign Language (ISL) is the primary visual communication medium for approximately 63 million deaf and hard-of-hearing individuals in India, yet computational resources for ISL processing remain severely limited compared to spoken languages. This paper presents a comprehensive framework for developing an AI-powered mobile application capable of bidirectional translation between English/Hindi audio-visual content and Indian Sign Language. The system integrates three critical components: large-scale dataset aggregation from sources including iSign benchmark (118,000 video-sentence pairs) and ISLTranslate (31,222 sentence pairs), advanced natural language processing pipelines for grammatical transformation, and computer vision techniques for sign generation and recognition. We discuss the architectural design of such systems, the processing pipeline from audio input through NLP tokenization, lemmatization, and part-of-speech tagging to final ISL video output, and present evaluation methodologies using Dynamic Time Warping and machine translation metrics. The paper incorporates findings from recent research on ISL datasets, linguistic properties unique to sign languages, and practical implementation considerations for real-world deployment on mobile platforms. Index Terms—Indian Sign Language, bidirectional translation, natural language processing, computer vision, mobile application, accessibility technology, sign language datasets, neural machine translation.

This paper presents innovative approaches to various aspects of sign language recognition and learning, catering to the diverse needs within this field. It introduces a system utilizing 2D image sampling and concatenation, trained with convolutional neural networks, achieving accurate and robust sign recognition even with low-cost cameras. Another system, SignQuiz, offers a cost-effective web-based solution for learning finger-spelled signs in Indian Sign Language (ISL), outperforming traditional printed mediums. Additionally, a dynamic hand gesture recognition system employing deep learning architectures demonstrates superior performance over existing methods, addressing challenges such as hand segmentation and sequence modeling. Furthermore, an efficient deep convolutional neural network approach for hand gesture recognition, including transfer learning, achieves high recognition rates across various datasets. Finally, a robust model for static sign recognition in ISL, utilizing CNNs and evaluating on a diverse dataset, attains exceptional accuracy and outperforms previous works. These contributions collectively advance the field of sign language recognition and learning, offering solutions that are accurate, cost-effective, and efficient, thereby facilitating better communication and interaction for the hearing-impaired community. Key Words: CNN, Sign language recognition, Hand gesture recognition, Machine learning, Transfer Learning, SVM.

Sign Language is a meansof communication between audibly challenged people. To provide an interface between the audibly challenged community and the rest of the world we need Sign Language translators. A sign language recognition system computerizes the work of a sign language trEvery Sign Language Recognition (SLR) System is trained to recognizespecific sets of signs and they correspondingly output the sign in the required format. These SLR systems are built with powerful image processing techniques. The sign language recognition systems are capable of recognizing a specific set of signing gestures and output the corresponding text/audio. Most of these systems involve the techniques of detection, segmentation, tracking, gesture recognition and classification. This paperproposes a design for a SLR System.

Sign Language is a way of communication used by hearing-impaired people. As many normal people don’t know the Sign Language it is difficult to understand by them to understand thus, increasing the communication gap between normal and hearing-impaired people. To overcome this problem many systems,using Machine Learning(ML), Artificial Intelligence(AI), Deep Learning(DL), computer vision methods, were developed and being developed. These systems are used to translate Indian Sign Language(ISL) to understandable text, facilitating the communication between normal and hearing-impaired people. This study briefly explains various type of Indian Sign Language(ISL) recognition systems.

Classifier fusion based on Bayes aggregation method for Indian sign language datasets

A major challenge faced by our society is the difficulty individuals with disabilities encounter in expressing their emotions to others. People with disabilities often depend on sign (gesture) languages for communication. This project aims to develop a model capable of recognizing and translating sign language alphabets from hand gestures into text and audio. The primary objective is to enhance communication between individuals with hearing impairments and the broader society. We assessed our model's performance using a publicly accessible Indian Sign Language (ISL) dataset. For image classification, we employed Convolutional Neural Networks (CNNs) with the Inception V3 architecture. Hand gestures are captured through webcams, and our model identifies the corresponding alphabet for each gesture. This project seeks to overcome current challenges in Sign Language Recognition and aims to enhance its effectiveness and efficiency.

Indian sign language (ISL) is used by deaf people in India for their internal communication. Sign language recognition (SLR) system recognizes gestures of sign language and converts them into text and voice, thus enabling deaf people to communicate with others. In this paper, we have presented an automatic system for ISL hand gesture recognition. Modern features like Histogram of Oriented Gradient (HOG), Speeded Up Robust Features (SURF), Fourier Descriptors, Hu Moments, and Zernike Moments have been extracted from training images of our own created dataset. Using these features, most commonly used classifiers like Support Vector Machine (SVM), Artificial Neural Network (ANN), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA) have been trained and evaluated. Comparative performance analysis of these classifiers along with experimental results is the main topic of discussion included in this paper. Moreover, a user-friendly GUI of the system allows user to select any combination of features and classifiers to perform gesture recognition from both stored file and live camera. During experiment, it is found that the system is able to recognize gestures with a recognition rate up to 98.70%.

A hand sign for a disabled patient is a useful tool for human-to-human communication that can be used for a variety of purposes. They are frequently utilised by speech-impaired people around the world for communication since they are a natural form of engagement. In actuality, this group makes up around 1% of the Indian population. To enable easy communication among the signer and non-signer communities, an effective sign language recognition system (SLRS) can identify sign language motions. This research proposes a deep learning-based SLRS for classification that is computer vision-based. A sample of Auslan (Australian Sign Language (ASL)) signs is included in the dataset. Using top-notch position trackers, 27 examples of each of the 95 Auslan signs were recorded by a native signer. However, films captured Indian Sign Language (ISL) was filmed utilising numerous signs. Three individuals submitted 26 letters (A–Z) and 10 numbers (0–9). Few background removal techniques were performed on the samples and various skin color segmentation process was done. Third, a brand-new and reliable model for classifying ISL motions has been suggested utilising the mutation denoising autoencoder (MDAE). ISL and ASL datasets are publicly available and have been used to measure the effectiveness of the proposed system. In terms of measures like precision, recall, F1-Score, and accuracy, the proposed system is contrasted with well-known computer vision algorithms like the Support Vector Machine (SVM), Convolution Neural Network (CNN), Enhanced Convolution Neural Network (ECNN), and DAE. Python has been used to implement and test these categorisation techniques.

Sign language recognition (SLR) system is helpful for individuals who can't talk or hear. One can easily communicate with this kind of people using SLR system. In this paper we are focusing on SLR approach where we have apply our proposed algorithm to recognize hand gestures of routine words like Morning, Night, Monday, Tuesday and so forth. Total 10 routine words we have recognize using our proposed technique. For recognize hand pose we have utilized skin color detection algorithm. Further we have applied correlation-coefficient algorithm to distinguish closeness features which are further continue in Neuro-fuzzy(NF) classification algorithm to recognize words. Presented algorithm tried on Matlab. With taken of 10 routine words our proposed framework of techniques got 92% correction rate.

In recent years, for recognizing sign language, several hardware approaches have been developed using the leap motion controller and Kinect sensors. The sensor-based approaches were costly, and the complexity of designing was high. In machine learning approaches, it was found that less accuracy of prediction occurs as compared to other approaches. As a solution, machine learning-based approaches using image processing evolved as an approach with better prediction and accuracy. In this paper, deep learning approach for sign language recognition (SLR) system has been proposed using AlexNet model. AlexNet is pretrained model, and it is being tested on Indian Sign Language (ISL) Dataset pf Stanford University comprising of alphabets (A–Z) and numerals (0–9). The proposed model has yielded an accuracy of 99.6%.KeywordsAlexNetDeep learningSLRRectified linear unit layer

The move to standardize Indian Sign Language has created an opportunity for researchers to focus on solving local problems, to increase its reach. In this paper, a survey and assessment of the techniques applied to the recognition and conversion of Indian Sign Language are performed. An overview of the techniques used in sign language recognition for Indian Sign Language is provided to understand the status of research in this field. Following this, a comparison of techniques aimed at rendering a more detailed picture of the research results is presented. The challenges faced by researchers, the limitations of current techniques, and the need for improved research in this area are highlighted. With the intent of spurring more in-depth research, key areas within the approaches and techniques in need of improvement are summarized.

Sign language simplifies communication for the deaf community.A kind of communication known as sign language uses visual sign patterns to convey feelings.When a deaf community wants to convey their ideas, speech, and hearing with sighted people, there is a communication gap.At the moment, two communities use human interpreters, which can be expensive and inconvenient.It might be challenging to distinguish across sign languages since their alphabets differ from one another.American Sign Language (ASL) has a very different alphabet than Indian Sign Language or Italian Sign Language (ISL).Thus, regional differences in sign language exist.We use convolutional neural network for capturing various image features and for feature extraction.The model is able to generalize on users and surroundings and the accuracy is high compared to other methods.