Camera-based interactive wall display using hand gesture recognition

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded system. In this work, we propose an HCI system for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially collected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efficiently with a sliding window approach. For the CNN model, several 3-dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver simulator operating in real-time. We make DriverMHG dataset and our source code publicly available 1.1https://www.mmk.ei.tum.de/DriverMHG/

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

Human-Computer Interaction (HCI) grew enormously over the years. Hand gesture recognition technology brought a brand-new era to the artificial intelligence branch of human-computer interaction. Hand Gesture-based communication is one of the most effortless and natural methods. Identifying and interpreting hand gestures from a continuous sequential stream of input data is called gesture recognition. Amateur or aged people find it hard to identify and press the exact alphabet that they need. Overcoming this difficulty stands as the ultimate goal for our proposed system. Typing has taken many forms, first using a keyboard, gradually changing to touch screens, and now too much easier finger motion tracking systems. The gesture is made by the user is detected by the machine via the image processing techniques and the operation unique to the machine is carried out, thereby minimizing the requirement of any hardware input device. This paper proposes a virtually controlled system that uses hand gestures and voice to perform operations. The system will allow the user to manage cursor, keyboard, and writing functions using hand gestures, and also provides the user with an additional feature to convert the user’s voice to text. The proposed system utilizes a webcam as an input device. This application will add a brand-new era of HCI implementation for educational purposes.

Stationary hand gesture recognition can be applied in a variety of domains such as human-computer interaction (HCI), remote control, virtual reality etc. Hand gesture recognition is mainly the study of detection and recognition of various hand gestures like Sign Language hand gestures, by a computer. This work is focused on three main issues in developing a gesture recognition system. These are (i) Threshold (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. It is very demanding to segment hand regions from the static hand gesture color images, due to unreliable light conditions and composite background. Since skin pixels can vary with different explanation condition, to find the range of skin pixels, becomes a hard task in case of color space based skin color segmentation. These methods propose a learning algorithm based on customized clustering expanse to remove person skin color regions from the stationary hand gesture color descriptions. To improve the performance of classification, two feature synthesis strategies have been proposed in this work.

Hand gesture for communication has proven effective for humans, and active research is ongoing in replicating the same success in computer vision systems. Human–computer interaction can be significantly improved from advances in systems that are capable of recognizing different hand gestures. In contrast to many earlier works, which consider the recognition of significantly differentiable hand gestures, and therefore often selecting a few gestures from the American Sign Language (ASL) for recognition, we propose applying deep learning to the problem of hand gesture recognition for the whole 24 hand gestures obtained from the Thomas Moeslund’s gesture recognition database. We show that more biologically inspired and deep neural networks such as convolutional neural network and stacked denoising autoencoder are capable of learning the complex hand gesture classification task with lower error rates. The considered networks are trained and tested on data obtained from the above-mentioned public database; results comparison is then made against earlier works in which only small subsets of the ASL hand gestures are considered for recognition.

The interaction between human and machine can be achieved by implementing human gesture applications which are known as human computer interaction (HCI). It has been developed several years ago, and now we can easily find it in many applications in real world e.g. in virtual reality, games, robotics, sign language recognition, etc. Gesture recognition has the potential to offer a natural way of communication between humans and machines. The most popular methods in human gesture application are hand gesture recognition. Several of hand gesture recognition methods still have the same problem when trying to recognize hand gesture accurately because of the existence of the uncontrolled environment problem, especially complex background. This study offered a method that can handle complex background in hand gesture recognition by analyzing pattern in spatio temporal domain. The system could reduce irregular noise in complex background significantly that affected to recognize the hand gesture. The experiment had been conducted using 180 videos with a total of 15 gestures. The result showed that the proposed method could recognize 15 hand gestures with accuracy of 98.33%.

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture. Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm. The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD). On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures. Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures. Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.

Interacting with mobile devices can be challenging in adverse working environments. Using hand gestures for interaction can overcome severe usability issues that users face when working with mobile devices in industrial settings. This chapter is dedicated to the design, implementation, and evaluation of mobile information systems with hand gesture recognition as means for human computer interaction. The chapter provides a concise theoretical background on human gestural interaction and gesture recognition, guidelines for the design of gesture vocabularies, recommendations for proper implementation, and parameterization of robust and reliable recognition algorithms on energy-efficient 8-bit architectures. It additionally presents an elaborated process for participatory design and evaluation of gesture vocabularies to ensure high usability in the actual context of use. The chapter concludes with a case study that proves the suitability of the proposed framework for the design of efficient and reliable hand gesture-based user interfaces.

Human computer interaction (HCI) is a classic topic, and hand gesture is one of the most natural languages, so the key point in human computer interaction is hand gesture recognition. A skin color model based on Hue is utilized to detect skin color regions like hands. To distinguish the face and hand, a simple and fast hand segmentation algorithm is developed. Moreover, we use the least square method to fit the trajectory of hand gravity motion and the angle and direction of the movement is got by the inverse trigonometric function of the slope of the fitted line. Ten people doing 80 hand gestures in low and high illumination environment are tested. Experimental results show that the accuracy is 94.375% in average and the processing time is 34ms per frame. These demonstrated the good real-time and robustness of the proposed approach.

Hand gestures aid the user in unspoken communication and help interpret sign language. Modern nonverbal communication detection employs gesture recognition systems and allows us to recognize shapes and signals made by hand movements using numerous machine learning techniques. Gestures have a very high value in human-computer interaction and sign language detection systems. To implement the gesture recognition and detection the existing methodologies rely on the application of bodily sensors. Due to this, the effectiveness in communication becomes challenging and cumbersome. Hence targeting on the major fallacies of the existing approaches, the primary aspiration of this research work is to recognize hand gestures in real-time using depth sensors that support natural communication avoiding on body sensor dependency. A novel approach is presented to detect hand gestures using minimalistic training and the efficiency of other algorithms concerning this proposed technique has been compared. Experimental investigations on the proposed methodology have successfully confirmed that, the research exertion has obtained an overall gesture detection accuracy of 90.86% and a high precision of 98.25%.

Sign Language is most accepted and purposeful way of communication for deaf and mute folks of the society. signing uses gestures, head/body movements and facial expressions for communication. it's a strong manner of Non-Verbal communication among humans. each country has its own developed signing. The language which is employed in Bharat is named as “Indian Sign Language (ISL)”. terribly lessresearch work has been carried out during this space as ISL has standardized recently. presently several researchers have primarily focused on gesture recognition that has been recorded underneath static hand gesture. solely few works have been reported for recognition of dynamic hand gesture. several ways are developed to recognize alphabets and numbers of ISL. The major steps concerned in planning such a system area unit gesture acquisition, trailing and segmentation, feature extraction and gesture recognition. This paper presents a survey on varied hand gesture recognition approaches for ISL. General Terms : Hand Gesture Recognition, Human computer interaction, YOLO3

This paper presents head and hand gestures recognition system for Human Computer Interaction (HCI). Head and Hand gestures are an important modality for human computer interaction. Vision based recognition system can give computers the capability of understanding and responding to the hand and head gestures. The aim of this paper is the proposal of real time vision system for its application within a multimedia interaction environment. This recognition system consists of four modules, i.e. capturing the image, image extraction, pattern matching and command determination. If hand and head gestures are shown in front of the camera, hardware will perform respective action. Gestures are matched with the stored database of gestures using pattern matching. Corresponding to matched gesture, the hardware is moved in left, right, forward and backward directions. An algorithm for optimizing connected component in gesture recognition is proposed, which makes use of segmentation in two images. Connected component algorithm scans an image and group its pixels into component based on pixel connectivity i.e. all pixels in connected component share similar pixel intensity values and are in some way connected with each other. Once all groups have been determined, each pixel is labeled with a color according to component it was assigned to.