An efficient interpretation of hand gestures to control smart interactive television

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

Technology becomes a part of human lives for decades, especially in human—computer interaction (HCI) that considered as the important research area involving with an assistive technology and a medical system. Hand gesture is classified as an intuitive method for human to interact with the computer. It is useful for elderly people who cannot express their feelings by words. This paper proposed the hand gesture recognition technique for the elderly by using contour detection with convex hull feature extraction and rule-based classification. Vision-based hand gesture recognition is considered in classifying six hand gestures as lingual description. From the experimental results, the hand gesture system provides good detection and classification results for all the six static hand gestures.

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded system. In this work, we propose an HCI system for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially collected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efficiently with a sliding window approach. For the CNN model, several 3-dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver simulator operating in real-time. We make DriverMHG dataset and our source code publicly available 1.1https://www.mmk.ei.tum.de/DriverMHG/

Stationary hand gesture recognition can be applied in a variety of domains such as human-computer interaction (HCI), remote control, virtual reality etc. Hand gesture recognition is mainly the study of detection and recognition of various hand gestures like Sign Language hand gestures, by a computer. This work is focused on three main issues in developing a gesture recognition system. These are (i) Threshold (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. It is very demanding to segment hand regions from the static hand gesture color images, due to unreliable light conditions and composite background. Since skin pixels can vary with different explanation condition, to find the range of skin pixels, becomes a hard task in case of color space based skin color segmentation. These methods propose a learning algorithm based on customized clustering expanse to remove person skin color regions from the stationary hand gesture color descriptions. To improve the performance of classification, two feature synthesis strategies have been proposed in this work.

This paper presents head and hand gestures recognition system for Human Computer Interaction (HCI). Head and Hand gestures are an important modality for human computer interaction. Vision based recognition system can give computers the capability of understanding and responding to the hand and head gestures. The aim of this paper is the proposal of real time vision system for its application within a multimedia interaction environment. This recognition system consists of four modules, i.e. capturing the image, image extraction, pattern matching and command determination. If hand and head gestures are shown in front of the camera, hardware will perform respective action. Gestures are matched with the stored database of gestures using pattern matching. Corresponding to matched gesture, the hardware is moved in left, right, forward and backward directions. An algorithm for optimizing connected component in gesture recognition is proposed, which makes use of segmentation in two images. Connected component algorithm scans an image and group its pixels into component based on pixel connectivity i.e. all pixels in connected component share similar pixel intensity values and are in some way connected with each other. Once all groups have been determined, each pixel is labeled with a color according to component it was assigned to.

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

Hand gestures aid the user in unspoken communication and help interpret sign language. Modern nonverbal communication detection employs gesture recognition systems and allows us to recognize shapes and signals made by hand movements using numerous machine learning techniques. Gestures have a very high value in human-computer interaction and sign language detection systems. To implement the gesture recognition and detection the existing methodologies rely on the application of bodily sensors. Due to this, the effectiveness in communication becomes challenging and cumbersome. Hence targeting on the major fallacies of the existing approaches, the primary aspiration of this research work is to recognize hand gestures in real-time using depth sensors that support natural communication avoiding on body sensor dependency. A novel approach is presented to detect hand gestures using minimalistic training and the efficiency of other algorithms concerning this proposed technique has been compared. Experimental investigations on the proposed methodology have successfully confirmed that, the research exertion has obtained an overall gesture detection accuracy of 90.86% and a high precision of 98.25%.

Hand gesture recognition is becoming an increasingly integral part of our daily lives, enabling seamless communication, enhancing interaction, and revolutionizing multiple industries. To ensure a more precise and efficient system, the key aspect of hand gestures lies in detecting hand patterns and retrieving the hand gestures. However, as the volume of video data increases, extracting the essential hand patterns while excluding unnecessary frames becomes a challenge. Addressing this issue, a novel Harris Hawk Optimization K-Means frame reduction is proposed, inspired by the hunting behavior of Harris Hawks in nature. This proposed approach combines the Harris Hawk Optimization algorithm with the K-Means clustering method. The algorithm simulates the hunting behavior of Harris Hawks and utilizes Euclidean distance as a fitness function to determine the optimal frames. Subsequently, the K-Means clustering method is employed to group similar frames together based on these optimal selections. An average frame is generated and aggregated for each cluster to form a reduced set of frames. These reduced frames are then classified using the modified Mobilenet V2 model, outperforming other state-of-the-art techniques by achieving an exceptional accuracy rate of 99.93%. The experiment results lay the groundwork for incorporating the novel framework of hand gesture recognition into a range of applications, including sign language interpretation, human-computer interaction, and virtual reality systems.

There has been growing interest in the development of new approaches and technologies for bridging the human-computer barrier. Hand gesture recognition is considered as an interaction technique having potential to communicate with machines. Human computer interaction (HCI) was never an easy task and lots of approaches are available to build such systems. Hand gesture recognition (HGR) using wearable data glove provides a solution to build a HCI system, but lags in terms of its computational time and poor interface. Pattern matching is one more solution which uses vision based techniques and provides strong interface to build HCI systems. But again, it requires complex algorithms which takes lots of computational time and hence limits its use in real time HCI applications. In this paper, we presented an eccentric approach for hand gesture recognition which is simple, fast and user independent and can be used to develop real time HCI applications. Based on proposed algorithm we built a system for Indian Sign Language recognition which converts Indian Sign numbers into text. The algorithm first captures the image of single handed gesture of speech/hearing impaired person using simple webcam and then using our proposed algorithm it classifies the gesture into its appropriate class. It uses simple logical conditions for gesture classification which make its use in real time HCI applications.

In this paper we have implemented a wireless vision based mobile robot control through hand gesture recognition based on perceptual color space such as HIS, HSV/HSB, HSL. Vision-based hand gesture recognition is an important problem in the field of human-computer interaction, since hand motions and gestures could potentially be used to interact with computers in more natural ways. The robot control was done purely based on the orientation histograms a simple and fast algorithm on the system which would recognize static hand gestures with HSV color spaces as major parameters. The wireless based mobile robot system using hand gestures is a new innovative user interface that resolves the complications of using numerous remote controls for various applications. Based on one unified set of hand gestures, this system interprets the user hand gestures into pre-defined commands to control the remote robot. The experimental results are very encouraging as the system produces real-time responses and highly accurate recognition towards various gestures under different lighting conditions.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. As the technology advances and communication between humans and machines becomes more complex, HCI systems should also be scaled accordingly in order to accommodate the introduced complexities. In this paper, we propose a methodology to scale hand gestures by forming them with predefined gesture-phonemes, and a convolutional neural network (CNN) based framework to recognize hand gestures by learning only their constituents of gesture-phonemes. The total number of possible hand gestures can be increased exponentially by increasing the number of used gesture-phonemes. For this objective, we introduce a new benchmark dataset named Scaled Hand Gestures Dataset (SHGD) with only gesture-phonemes in its training set and 3-tuples gestures in the test set. In our experimental analysis, we achieve to recognize hand gestures containing one and three gesture-phonemes with an accuracy of 98.47% (in 15 classes) and 94.69% (in 810 classes), respectively. Our dataset, code and pretrained models are publicly available.

Hand motion tracking and gesture recognition are a fundamental technology in the field of proactive computing for a better human computer interaction system. In this paper, we have developed a 3-D hand motion tracking and gesture recognition system via a data glove (namely the KHU-1 data glove consisting of three tri-axis accelerometer sensors, one controller, and one Bluetooth). The KHU-1 data glove is capable of transmitting hand motion signals to a PC through wireless communication via Bluetooth. Also we have implemented a 3-D digital hand model for hand motion tracking and recognition. The implemented 3-D digital hand model is based on the kinematic chain theory utilizing ellipsoids and joints. Finally, we have utilized a rule-based algorithm to recognize simple hand gestures namely scissor, rock, and paper using the 3-D digital hand model and the KHU-1 data glove. Some preliminary experimental results are presented in this paper.

Hand gesture for communication has proven effective for humans, and active research is ongoing in replicating the same success in computer vision systems. Human–computer interaction can be significantly improved from advances in systems that are capable of recognizing different hand gestures. In contrast to many earlier works, which consider the recognition of significantly differentiable hand gestures, and therefore often selecting a few gestures from the American Sign Language (ASL) for recognition, we propose applying deep learning to the problem of hand gesture recognition for the whole 24 hand gestures obtained from the Thomas Moeslund’s gesture recognition database. We show that more biologically inspired and deep neural networks such as convolutional neural network and stacked denoising autoencoder are capable of learning the complex hand gesture classification task with lower error rates. The considered networks are trained and tested on data obtained from the above-mentioned public database; results comparison is then made against earlier works in which only small subsets of the ASL hand gestures are considered for recognition.