Translating Gestures: Utilizing CNN to Enhance ASL Communication and Understanding

This paper presents the development of a real time American Sign Language (ASL) detection system using Raspberry Pi and a Sequential Convolutional Neural Network (CNN) model. The system aims to bridge the communication gap between the Deaf and Hard of Hearing (DHH) community and the hearing population by translating ASL gestures into text and audio outputs. The proposed system leverages the Raspberry Pi 5, a cost effective and scalable hardware platform, combined with deep learning techniques to achieve high accuracy in gesture recognition. The system utilizes the custom dataset for training and employs hand landmark detection using MediaPipe for real-time gesture analysis. The results demonstrate an 85% accuracy in recognizing ASL gestures, with real-time text and audio outputs. The system is designed for personal, educational, and public applications, offering a practical solution for enhancing communication accessibility for the DHH community. Key Words: American Sign Language(ASL), Raspberry Pi, Machine Learning, Gesture recognition, Sign Language Translation, Text-to-Speech conversion

Optimizing convolutional neural networks on multi-core vector accelerator

People around the world with speech and hearing impairment use a media of communication known as ‘Sign language’. In recent times, Sign language is omnipresent. However, there exists a challenge for people who do not know sign language, to communicate with people who can communicate exclusively using sign language. This gap can be bridged by using technologies of recent times to recognize gestures and design intuitive systems with deep learning. The aim of this paper is to recognise American Sign Language gestures dynamically and create an intuitive system which provides sign language translation to text and speech of various languages. The system uses Convolutional neural network, natural language processing, language translation and text-to-speech algorithms. It is capable of recognizing hand gestures dynamically and predicting the corresponding letters to form a desired sentence accurately.

Handwritten digit recognition, a task in computer vision, is critical for applications such as postal automation, banking, and digitization of forms. Traditional approaches have leveraged statistical models, but the rise of deep learning, particularly Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM), has revolutionized the field. However, a comprehensive performance comparison of CNN and LSTM architectures in the context of handwritten digit classification remains underexplored. This study aimed to address this gap by evaluating and comparing CNN and LSTM models for the classification of handwritten digits. Two machine learning models – Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) were trained with the preprocessed handwritten digit dataset. The CNN model was designed with multiple convolutional and pooling layers, along with dropout for regularization. The LSTM model was designed with LSTM layers to capture sequential patterns in the data, followed by a dense layer for classification. The models were implemented in python, evaluated and compared based on accuracy, precision, recall and F1-score. The evaluation and comparison results indicate that CNN achieved 99.31% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score, while LSTM achieved 98.90% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score. The results demonstrated that CNN outperformed LSTM in terms of accuracy and misclassification errors, making it the optimal choice for image-based handwritten digit recognition. This finding underscores the efficiency of CNN in addressing challenges related to digit recognition, contributing to the advancement of automated digit classification systems and improving the accuracy of image-based classification tasks.

Increasing adaptability of a speech into sign language translation system

Understanding the learning mechanism of convolutional neural networks in spectral analysis

This research proposes a novel hardware-optimized design to accelerate Convolutional Neural Networks (CNNs) using Verilog HDL. The design is specifically developed for the DARKNET-19 system model, which serves as the backbone of the YOLOv3-tiny algorithm, a widely used framework for real-time object detection in dynamic environments. The CNN architecture was implemented in Verilog HDL and synthesized using Synopsys Design Compiler, with a focus on improving both object detection accuracy and hardware resource efficiency. The proposed design efficiently performs key CNN operations, including convolution, pooling, and activation, enabling faster real-time object detection compared to many existing methods. To improve performance, the hardware design incorporates parallel processing techniques, allowing multiple computations to be executed simultaneously. This significantly reduces the system latency and power consumption. The convolutional layers of the DARKNET-19 architecture are efficiently mapped onto the hardware platform, ensuring optimized data storage and fast memory access, which further enhances processing speed and detection accuracy. An innovative feature of the design is a 2-dimensional image preprocessing module that prepares input images before they are fed into the CNN. This preprocessing stage includes image resizing, brightness normalization, and color adjustment, which helps the CNN process visual data more effectively. After preprocessing, the images pass through several CNN layers. The convolutional layers extract key features from the images, while the pooling and activation layers refine these features to improve detection performance. Finally, the processed data is analyzed by the YOLOv3-tiny algorithm, which identifies and locates objects in the images with high precision. Experimental results demonstrate that the proposed high-speed and resource-efficient hardware architecture is well-suited for real-time object detection applications, particularly in highly dynamic and unpredictable environments.

The project "Transforming Expression into Language: American Sign Language Recognition using Image Processing" intends to employ image processing techniques to identify gestures in American Sign Language (ASL). The goal of this project is to develop an efficient system for translating ASL signs using visual cues generated from hand motions and gestures. The program intends to use image processing technology to bridge the communication gap between ASL users and nonusers. The use of image processing for ASL identification may improve communication in a range of situations while also increasing accessibility for the deaf community. The paper provides a realistic approach to real-time ASL motion interpretation using image processing techniques, thereby advancing the field of sign language recognition technology. KEYWORDS ASL Recognition, Image Processing Techniques, Convolutional Neural Networks (CNNs), Feature Extraction in Image Processing, Transforming Expressions into Text

This critical literature review examines how African American English (AAE) and American Sign Language (ASL)-including Black ASL-shape literacy development in Black Deaf students. It situates these linguistic systems within broader historical and contemporary contexts of literacy acquisition, highlighting systemic inequities in educational practices that marginalize nonstandard dialects and visual languages. Understanding these dynamics is essential to developing more inclusive and effective literacy instruction that honors the linguistic identities of Black Deaf learners. Drawing from sociocultural theory, (socio)cognitive theory, poststructuralism, and critical race theory, this literature review employs a thematic synthesis to analyze how these frameworks illuminate the complex interplay between language, identity, and literacy development among Black Deaf students. The review integrates empirical and theoretical studies across hearing and Deaf populations, situating findings within broader historical and societal literacy trends. It also addresses contemporary challenges such as multilingualism and the role of nonpublic educational settings in shaping literacy outcomes. The findings reveal that prevailing educational systems frequently rely on deficit-based models that marginalize AAE and ASL, thereby limiting literacy opportunities for Black Deaf students. The literature advocates for a shift toward diversity-affirming frameworks that recognize AAE and ASL as vital cognitive and cultural assets supporting literacy acquisition. Key themes include the importance of culturally responsive pedagogy and inclusive educational policies that validate students' linguistic identities, dismantle systemic biases, and foster equitable literacy development. These insights underscore the need for educational practices that integrate students' home languages and social contexts to promote academic success. Affirming AAE and ASL in educational settings is essential for promoting equitable literacy development. The review recommends integrating these theoretical perspectives into policy reform and within the fields of education and speech-language pathology. Future research should explore the roles of family, community, and multimodal literacy practices in supporting Black Deaf learners across diverse educational contexts.

Development in Computer vision applications, most prominently in Artificial Intelligence (AI) over the past decade, has encouraged researchers to use computational algorithms. Machine Learning (ML) and Deep Learning (DL) algorithms are getting attention in the biomedical area to early diagnosis and prognosis of lethiferous diseases with higher accuracy using statistical analysis. Cervical cancer is one of the predominant gynecological infectious forms of cancer in females ecumenically, generally transpiring in less-developed nations. DL algorithms are more optimized and convenient due to the automatic feature extraction in Cervical lesion detection at an early phase. Here we analyze the convolutional neural network (CNN) models' performance using variation in channels depth comprises in the convolution layers to classify multi-class liquid-based cytology (LBC) Whole slide images (WSI). We propose three CNN models having two convolution layers with the number of channels (4,8), (8,16), and (32,64) and two pooling layers. CNN model with the highest number of channels in the convolution layers gives the leading performance with an accuracy of 96.89%, precision of 93.38%, the sensitivity of 93.75%, and F-score of 94.15%.

Background: Hearing loss affects 1.6 billion people, making it the third most common disability worldwide. Patients who can hear are approximately 2 times more likely to secure primary care appointments than those who are deaf and communicate via American Sign Language (ASL). Compounding factors such as inadequate sign language interpreter services and the COVID-19 pandemic have exacerbated the existing healthcare service disparities received by the deaf population. Additional studies show that despite having similar health insurance coverage percentages (Deaf=96% covered; Hearing=94% covered), deaf patients rated their doctors as having lower patient-centered communication skills than those who hear (Deaf patient centered communication satisfaction mean=60.99 [SD=25.89]; Hearing patient centered communication satisfaction mean=82.25 [SD=21.14]; t test=16.97, P value<.001). Despite this, there is still a lack of formal training for frontline health professionals working with this disadvantaged population. Methods The dire need for ASL interpreter services can be alleviated by introducing an ASL elective to medical school programs and other health professional school curriculums. Language electives, including medical Spanish, are currently offered by 78% of 125 U.S. medical schools in response to the growing Spanish-speaking population, yet very few medical programs offer an elective to teach ASL. Our proposed ASL elective would be modeled similarly to current language electives to confront this seemingly timeless disparity. Results Through collaboration with local ASL community centers, including churches and colleges, health professional programs across the nation can develop a 4-week ASL elective through clubs or service-learning projects. Offering such an elective would ultimately help produce better prepared future healthcare professionals, especially those interested in ENT, audiology, and primary care specialties. Conclusion The Americans with Disabilities Act (ADA) mandates equal access and effective communication to all patients, especially those with disabilities; however, ASL interpreter services are not readily available or reimbursed by all healthcare systems and insurances. Furthermore, the COVID-19 pandemic enforced mask mandates which prevents deaf patients from reading lips, thus hindering one of few alternative communication methods apart from ASL. The call for reformation in the medical healthcare system to alleviate the perpetual struggles of the deaf population is one that needs to be heard now.

Empirical-statistical Downscaling (SD) techniques are valuable tools able to generate high-resolution climate information needed to carry out impact studies. In this regard, Convolutional Neural Networks (CNNs) are promising SD techniques capable of handling large amounts of data and extracting relevant predictor information for each particular site. These characteristics of the CNN represent a major advantage over traditional SD methods, which typically rely on human-guided predictor selection. Notwithstanding, an adequate tuning of the CNN is key for optimising their potential.In southern South America (SSA), CNNs has proven to be skilful in representing daily extreme temperatures and extrapolating into future scenarios. Although the selection of the activation function introduces a source of uncertainty in the future projections.&amp;#160;In this context, this study aims to explore the use of CNNs as a statistical downscaling tool to simulate the wet bulb temperature (Tw) over SSA, a multivariate heat-stress index estimated from temperature and humidity. Tw has been widely used as a heat-stress proxy in different parts of the world, however, its characterisation and modelling in SSA remain as a pending task. To this end, four different CNN architectures regarding the activation function (ReLU or linear), domain size and configuration of the CNN layers were tested. All CNN models were trained during summer days using a cross-validation (CV) scheme in the period 1991-2020 and then evaluated in four unseen summers between 2021 and 2024. For comparison purposes, CNN models were similarly trained and validated to simulate maximum temperature (Tx).&amp;#160;Overall, CNN models well represented all the features evaluated, including the heat-waves that took place in the summers evaluated independently. In particular, CNN models presents a better performance in simulating Tw than Tx with smaller errors in terms of mean and extremes aspects. Regarding the domain size, for both temperatures, the configuration with the smaller domain yields the best results. Also in this latter case, the reduction of the number of filter size in the last layer slightly improves the representation of Tx. When considering the large domain, the differences between the CNNs based on different activation functions increase, and CNN models with linear configuration outperform the ones with ReLu.&amp;#160;The findings of this work reinforces the potential of CNNs for climate downscaling in SSA, especially for its use to simulate multivariate impact indices.

Sentiment Analysis was a process for identifying whether a source of text contains certain opinions, emotions, and polarity. Twitter Sentiment Analysis was a process for identifying sentiment and polarity on tweet. Twitter Sentiment Analysis provided a way for did survey about public sentiment to product, or particular even through collections of tweet. Main problem in sentiment identifying was how to determine classification model that gave high accuracy to classifying sentiment of tweet. One of the method for classifying sentiment of tweet was Deep Learning. Convolutional Neural Network (CNN) was special type of architecture from Deep Learning that its architecture had convolution layer. Convolution layer was important for extract relevant feature from text for classifying sentiment. The objective of this research was for found out the best CNN model for classifying sentiment of tweet. By using a dataset of tweets about public opinion on the Smartfren 4G network service, we searched the best CNN model using 6 combination parameters, that is the computational eficiency method, window size, and dimension of word embedding for parameters in Word2Vec Skip-gram model, then activation function in convolution layer, dropout rate, and pool size in pooling layer for parameters in CNN. The test is done using 10-fold cross validation for each search for the best parameter value and produced the best CNN model with an accuracy value of 88,21%.

A well-known vegetable crop farmed all throughout the world is the potato. The possibility of a large number of diseases, however, could also impair the harvest. In order to quickly deal with the crop, it is vital for the planter to recognise the type of contamination. It had been determined that the leaves were a simple indicator of many diseases. Many Machine Learning (ML) techniques as well as Convolution Neural Network (CNN) models are superior in the literature for the detection of tomato crop diseases. CNN models rely on deep learning and neural networks, which are superior than traditional machine learning methods like decision trees and kNN. Although well before CNN creates beautiful artwork, the wide range of subjects they cover renders them operationally intensive. This painting suggests a less complex CNN model with eight convolution layer At the publicly accessible dataset PlantVillage, the recommended compact model performs better than the standard machine learning techniques and pre-trained fashions with an accuracy of 98.4%. The PlantVillage dataset includes 39 classes of different crops, including tomato diseases. These crops include apple, potato, maize, grapes, and more. In comparison to VGG16, which has an amazing accuracy of 93% 5% in pretrained patterns, k-NN has the absolute best accuracy of 94% in conventional ML approaches. After image augmentation, picture pre-processing was used to improve the suggested CNN's overall performance by having the ability to modify the brightness of the image using the knowledge of a prospective variable involving a randomized width of the image. The suggested model, which has a 98% accuracy rate, is also incredibly effective with datasets other than PlantVillage.

Sign language is a type of language that uses manual communication to convey meaningful messages to other people. This includes simultaneous employing of hand gestures, movement, orientation of the fingers, arms or body, and facial expressions to convey a speaker's ideas. American Sign Language is one of the popular sign language used by most of deaf and dumb people to communicate with each other. American Sign Language is also referred to as ASL. A real-time sign language translator is required for facilitating communication between the deaf community and the general public. We propose a system called Dynamic tool for American Sign Language (ASL) finger spelling interpreter which can consistently classify the letters a-z. The dataset consists of a set of American Sign Language videos. Our approach first converts the videos into frames and then pre-processes the frames to convert them into greyscale images. Then the Convolutional Neural Network (CNN) classifier is used for building the classification model which classifies the frames into 26 different classes representing 26 English alphabets. Finally, the evaluation of the classification model is carried out with test data providing the output in the form of text or voice. The cross-validation accuracy results of 98.66% is achieved from our approach.