Speech generation from hand gestures based on space mapping

In this paper we have implemented a wireless vision based mobile robot control through hand gesture recognition based on perceptual color space such as HIS, HSV/HSB, HSL. Vision-based hand gesture recognition is an important problem in the field of human-computer interaction, since hand motions and gestures could potentially be used to interact with computers in more natural ways. The robot control was done purely based on the orientation histograms a simple and fast algorithm on the system which would recognize static hand gestures with HSV color spaces as major parameters. The wireless based mobile robot system using hand gestures is a new innovative user interface that resolves the complications of using numerous remote controls for various applications. Based on one unified set of hand gestures, this system interprets the user hand gestures into pre-defined commands to control the remote robot. The experimental results are very encouraging as the system produces real-time responses and highly accurate recognition towards various gestures under different lighting conditions.

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

With new changes seen in computer technology day by day, it has become quite essential for us to find specific new ways of interaction with computer systems as its need is increasing in society every day. Today, every device is making the use of touch screen technology on its systems, which isn't affordable to be used in all applications. A specific interactive module like a virtual mouse that makes use of Hand Tracking and Gestures that will help us to interact can be an alternative way for the traditional touch screen and the physical mouse. The objective is to create a Hand Tracking application that interacts with the system. This system proposed is a Computer Vision-based mouse cursor control system, which uses hand gestures that are being captured from a webcam through a Hand detection technique. This system allows the user to navigate the system cursor using their hand gestures that the computer webcam tracks and perform mouse operations like left- click, right-click, and double click using different hand gestures. Python and OpenCV library are used for real- time computer vision to implement the system. The camera output is displayed on the monitor.

The online version contains supplementary material available at 10.1007/s00521-022-08090-8.

Gesture Mouse is a real-time hand gesture recognition system that uses deep convolutional neural networks to mimic mouse functionalities for controlling computers. The system requires only a standard webcam, and it can detect and recognize various hand movements and gestures, such as clicking, scrolling, zooming, movements, and hovering. The aim of this project is to improve and ease the human-computer interaction, especially for users who are physically challenged or have limited mobility. The Gesture Mouse system consists of three main components: hand detection, hand landmark extraction, and gesture recognition. The hand detection component is based on the Single Shot Multi Box Detector (SSD) model, which is used to detect the hand region in each frame of the webcam stream. The hand landmark extraction component uses a convolutional neural network (CNN) to predict the 3D coordinates of 21 key hand landmarks, including fingertips, palm canter, and wrist. The gesture recognition component is based on a CNN classifier that is trained on a large dataset of hand gesture images. The system performs well in real-time and can be used to control a computer with a high degree of accuracy and reliability. The proposed system has the potential to provide a more natural and intuitive way of interacting with computers, especially for individuals with motor impairments and also in technologies such as Virtual Reality and Augmented Reality.

In this paper we propose a way to accomplish Human Computer Interface absolutely in electronic way (without mechanical sensors). The idea is to extirpate old techniques of controlling Robotic arm using joysticks, buttons and supersede with more intuitive technique ie., to control robotic arm by hand motion or gesture. Here we propound an approach to achieve the aforementioned idea employing Image processing technique using web camera. We detect the vital features of hand: fingers by computational geometry calculation enabling real time interaction between hand gestures and Robot. Our system can meticulously locate fingers even when fore-arm is involved. And the system can sustain a certain rotation of palm and fore-arm, which augments the freedom of use in palm center estimation.

It is extremely difficult for people who are deaf to communicate their message to normal people. Communication becomes extremely difficult in case of an emergency, while travelling or among unfamiliar people because most people are not trained in hand sign language. We propose a voice aid system that allows deaf persons to communicate with regular people by using hand motions and gestures. This system uses motion and flex sensors, as well as a speaker unit. The circuitry that runs this system is fuelled by batteries. The system includes various pre-programmed messages such as "need help," "where is the washroom," and other common phrases that enable deaf individuals communicate basic information. The technology analyses a person's hand motions for different variants. It also has a trigger sensor to provide the impression that the person wants to activate the device and say something.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded system. In this work, we propose an HCI system for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially collected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efficiently with a sliding window approach. For the CNN model, several 3-dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver simulator operating in real-time. We make DriverMHG dataset and our source code publicly available 1.1https://www.mmk.ei.tum.de/DriverMHG/

This paper talks about the significance of uniformity in the Nritta Hastas of Kathak dance across gharanas and eras. Kathak, an Indian classical dance form, hails from North India. Nritta Hastas refer to hand gestures or movements used in Indian classical dance. Often interchanged with” “hasta mudra”, “Nritta Hastas” encompass all hand movements and gestures, whereas “mudras” are more indicative of emotions and literal depictions of objects present in our environment.Kathak has evolved to appeal and cater to the spectators of today’s times - with lightning fast spins and hand movements along with intricate footwork. However, this has caused unfavorable negotiations with the original technique and overall uniformity of Nritta Hastas, thus resulting in the deterioration of quality in dance performances. For example, prioritizing the ability to execute multiple spins in quick succession, aiming to impress the audience with their speed and agility, leads to neglecting the intricate hand movements and mudras, which are essential for conveying the nuanced emotions and storytelling aspects of the dance. The purpose of this research paper is to shed light on the importance of maintaining uniformity by adhering to techniques, methods and pedagogies mentioned in ancient scriptures pertinent to Indian classical dance.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. As the technology advances and communication between humans and machines becomes more complex, HCI systems should also be scaled accordingly in order to accommodate the introduced complexities. In this paper, we propose a methodology to scale hand gestures by forming them with predefined gesture-phonemes, and a convolutional neural network (CNN) based framework to recognize hand gestures by learning only their constituents of gesture-phonemes. The total number of possible hand gestures can be increased exponentially by increasing the number of used gesture-phonemes. For this objective, we introduce a new benchmark dataset named Scaled Hand Gestures Dataset (SHGD) with only gesture-phonemes in its training set and 3-tuples gestures in the test set. In our experimental analysis, we achieve to recognize hand gestures containing one and three gesture-phonemes with an accuracy of 98.47% (in 15 classes) and 94.69% (in 810 classes), respectively. Our dataset, code and pretrained models are publicly available.

The hand gesture recognition technology brought a new era to the artificial intelligence branch of human-computer interaction. This initiative will nevertheless provide a more natural and dynamic form of communication by use of gestures. The gesture is made by the user who is detected by the machine via the image processing and the operation unique to the machine is carried out, thereby eliminating the requirement of any hardware input device. In the application is included an additional database of distinct gestures. The entire system is greatly dependent on the surrounding lighting condition. A well-lit room is a must as the major constraint system. The system utilizes nothing but a camera with good quality and can follow the hand of the user in 2 dimensions and identify up to four mouse-defined hand motions. The input picture from the camera is transformed first into the color space of HSV, which detects the skin and removes the backdrop. The edge detection is then done in order to get the contour of the hand utilized to track the hands and to recognize gesture. This application will detect the numbers written by hand gestures. This will work as a virtual blackboard where numbers and letters can be written by using gesture. The purpose of this post is to create a virtual blackboard which can be used to make online confrontation by moving fingers as the hand gesture. Everyone will help to reduce the use of a whiteboard or smart whiteboard. This application will add a new era of HCI implementation in educational purpose. Experiments will demonstrate that implementation is sufficiently dependable for practical application.

In this paper, a novel gesture spotting and recognition technique is proposed to handle hand gesture from continuous hand motion based on Conditional Random Fields in conjunction with Support Vector Machine. Firstly, YCbCr color space and 3D depth map are used to detect and segment the hand. The depth map is to neutralize complex background sense. Secondly, 3D spatio-temporal features for hand volume of dynamic affine-invariants like elliptic Fourier and Zernike moments are extracted, in addition to three orientations motion features. Finally, the hand gesture is spotted and recognized by using the discriminative Conditional Random Fields Model. Accordingly, a Support Vector Machine verifies the hand shape at the start and the end point of meaningful gesture, which enforces vigorous view invariant task. Experiments demonstrate that the proposed method can successfully spot and recognize hand gesture from continuous hand motion data with 92.50% recognition rate.

Hand gestures aid the user in unspoken communication and help interpret sign language. Modern nonverbal communication detection employs gesture recognition systems and allows us to recognize shapes and signals made by hand movements using numerous machine learning techniques. Gestures have a very high value in human-computer interaction and sign language detection systems. To implement the gesture recognition and detection the existing methodologies rely on the application of bodily sensors. Due to this, the effectiveness in communication becomes challenging and cumbersome. Hence targeting on the major fallacies of the existing approaches, the primary aspiration of this research work is to recognize hand gestures in real-time using depth sensors that support natural communication avoiding on body sensor dependency. A novel approach is presented to detect hand gestures using minimalistic training and the efficiency of other algorithms concerning this proposed technique has been compared. Experimental investigations on the proposed methodology have successfully confirmed that, the research exertion has obtained an overall gesture detection accuracy of 90.86% and a high precision of 98.25%.

In this paper, a multimodal hand gesture detection and recognition system using differential Pyroelectric Infrared (PIR) sensors and a regular camera is described. Any movement within the viewing range of the differential PIR sensors are first detected by the sensors and then checked if it is due to a hand gesture or not by video analysis. If the movement is due to a hand, one-dimensional continuous-time signals extracted from the PIR sensors are used to classify/recognize the hand movements in real-time. Classification of different hand gestures by using the differential PIR sensors is carried out by a new winner-take-all (WTA) hash based recognition method. Jaccard distance is used to compare the WTA hash codes extracted from 1-D differential infrared sensor signals. It is experimentally shown that the multimodal system achieves higher recognition rates than the system based on only the on/off decisions of the analog circuitry of the PIR sensors(1).

One of the non-verbal communication techniques utilized in sign language is the hand gesture. Deaf persons who struggle with speech or hearing most frequently use it to converse with each other and hearing people. Several makers around the world have created various sign language systems, but they are neither adaptable nor cost-effective for end users. The majority of programs only provide text- based results. So, we are going to offer a system prototype in an application that solves the aforementioned issue. Hand gestures can be translated into text and then spoken by the system. Several algorithms will be used by the system, including CNN for classifying hand movements and SVM for detection of hand movement & TTS for conversion of Text To Speech.