Hand gesture recognition using Kinect

Hand gesture recognition is a relevant study topic for a reason that sometimes we may not be in position to communicate verbally. There is need to design Hand gesture recognition systems in order to help people adopt to nonverbal communication mainly sign language. However, there is no clue to understand the meaning of gesture through the computers directly. So this calls for definitions that generalize models in a computer. That is why the machine-learning approaches are implemented in recognition systems. There are generally two types of hand gestures recognition systems which researches have concentrated on. These include static and dynamic Hand gesture recognition systems. However, in building Hand gesture recognition systems, various machine learning approaches have been used. For implementing the proposed system, MS Kinect depth sensor was used as a hardware. The Kinect depth sensor is composed of an infrared camera. This is an advantage to the systems that are designed basing on the depth sensing because factors like color, clothing and background have less effect on the performance. So Kinect based depth sensor systems have a high accuracy and performance making them relevant and applicable in our daily lives. In this paper, we propose a static hand gesture recognition system in real time using two machine learning methods namely Support Vector Machine and Artificial Neural Networks. We use of the newly launched Microsoft Kinect sensor for image extraction. The sensor helps us to extract the hand images. We implement the system on a Matlab platform for reasons that Matlab is widely used by researchers in different fields and that can handle complex computations. In the training of the model, we collect a hundred depth-based Histogram of Oriented Gradient features per alphabet from the hand gesture images which we trained, tested and validated using Artificial Neural Networks (ANN) and Support Vector Machine (SVM). From this dataset, we can generate the generalized gesture model for each alphabet image. For the proposed system, the classification with ANN proves a higher performance then SVM.

In this paper, we propose a real-time hand gesture recognition system using a thinning method utilizing a stereo camera. We implement a depth map in the hand detection portion that uses a sum of absolute differences method based on the acquired right-left image to detect the foreground object. We use a convex hull to detect the region of interests (ROI) and calculate the depth of object in ROI to obtain hand images that are more accurate. Then, we remove the background image in ROI and get the foreground image as a hand image. Finally, we use a blob labeling method to obtain the clean hand image, without the noise caused by computing distance from stereo matching. The hand gesture recognition system uses the Zhang and Suen thinning algorithm to obtain the feature point, angle and distance. It recognizes five kinds of hand gestures. The proposed method achieves an average recognition rate of 82.93%.

Hand gesture recognition (HGR) systems using electromyography (EMG) bracelet-type sensors are currently largely used over other HGR technologies. However, bracelets are susceptible to electrode rotation, causing a decrease in HGR performance. In this work, HGR systems with an algorithm for orientation correction are proposed. The proposed orientation correction method is based on the computation of the maximum energy channel using a synchronization gesture. Then, the channels of the EMG are rearranged in a new sequence which starts with the maximum energy channel. This new sequence of channels is used for both training and testing. After the EMG channels are rearranged, this signal passes through the following stages: pre-processing, feature extraction, classification, and post-processing. We implemented user-specific and user-general HGR models based on a common architecture which is robust to rotations of the EMG bracelet. Four experiments were performed, taking into account two different metrics which are the classification and recognition accuracy for both models implemented in this work, where each model was evaluated with and without rotation of the bracelet. The classification accuracy measures how well a model predicted which gesture is contained somewhere in a given EMG, whereas recognition accuracy measures how well a model predicted when it occurred, how long it lasted, and which gesture is contained in a given EMG. The results of the experiments (without and with orientation correction) executed show an increase in performance from 44.5% to 81.2% for classification and from 43.3% to 81.3% for recognition in user-general models, while in user-specific models, the results show an increase in performance from 39.8% to 94.9% for classification and from 38.8% to 94.2% for recognition. The results obtained in this work evidence that the proposed method for orientation correction makes the performance of an HGR robust to rotations of the EMG bracelet.

In today’s ever-evolving healthcare landscape, our objective is to bring technology closer to human experience. In this paper, a vision-based hand gesture recognition (HGR) system is introduced as a human computer interface (HCI) that transforms natural movements into intuitive medium for device and application manipulation. This innovative approach is designed to empower the elderly and enhance patient monitoring by offering a seamless, nonintrusive interface. By bridging compassion with cutting-edge technology, this approach aims to redefine care and create a more connected, responsive healthcare environment. The study begins by reviewing both traditional and contemporary HGR methodologies, with a focus on vision-based systems, which have shown greater potential for practical applications compared to conventional sensor-based systems. Traditional approaches, such as contour analysis, colour segmentation, and template matching, have demonstrated limitations. In contrast, deep learning approaches have gained mass adaptation in vision-based systems for their ability to learn hierarchical features and better handling complex, dynamic scenarios. However, existing models struggle in real-time performance due to computational inefficiencies and environmental variations. This study introduces a deep learning-based detection framework which enables the practical application of HGR. For example, it supports the development of gesture-based virtual assistants for interacting with digital devices and allows elderly residents in care homes to give instructions to remote caregivers. The framework consists of two deep learning models, one for region of interest (ROI) detection and another for classification. The recognition process begins with capturing RGB images, chosen for versatility, affordability, and compatibility, which then passes through YOLOv8 (You Only Look Once version 8) for detection. The YOLOv8 model is trained on a subset, only 4.5%, of the HaGRID dataset, comprises over 550,000 RGB images, to accurately locate hand regions. Once detection is complete, several image processing techniques are applied to overcome typical HGR constraints. These include ROI extraction and gray scaling for faster computation, resizing for model optimization and distance viability, HSV conversion for lighting independence, and histogram equalization for enhanced feature extraction. The processed image is then fed into a convolutional neural network (CNN) for gesture classification. The network is designed with several convolutional layers, followed by dense layers with ReLU activation to introduce non-linearity, and SoftMax activation for classification. To improve the model's robustness, data augmentation techniques such as random flipping and rotation are applied. The HGR system demonstrates a staggering accuracy of 97% for training & validation. The proposed framework effectively addresses the limitations of conventional HGR systems—many of which have been developed and tested in controlled environments—demonstrating its applicability in real-life situations.

Hand gesture is one of the method used in sign language for non-verbal communication. It is most commonly used by deaf & dumb people who have hearing or speech problems to communicate among themselves or with normal people. Various sign language systems has been developed by many makers around the world but they are neither flexible nor cost-effective for the end users. Hence in this paper introduced software which presents a system prototype that is able to automatically recognize sign language to help deaf and dumb people to communicate more effectively with each other or normal people. Pattern recognition and Gesture recognition are the developing fields of research. Being a significant part in nonverbal communication hand gestures are playing key role in our daily life. Hand Gesture recognition system provides us an innovative, natural, user friendly way of communication with the computer which is more familiar to the human beings. By considering in mind the similarities of human hand shape with four fingers and one thumb, the software aims to present a real time system for recognition of hand gesture on basis of detection of some shape based features like orientation, Centre of mass centroid, fingers status, thumb in positions of raised or folded fingers of hand

Hand gesture recognition can be exploited to benefit ubiquitous applications using sensors. Currently, the inherent complexity of human physical activities makes it difficult to accurately recognize gestures with wearable sensors, especially in real time. To this end, a real-time hand gesture recognition system is presented in this paper. In particular, sliding window technology and y-axis threshold are used to detect intended gestures from a continuous data stream and then the segmented data are classified by applying a stacked Long Short-Term Memory (LSTM) model. After noise is removed, six-axis sensor data from wrist-worn devices are fed into the model without requiring feature engineering. We use twelve common hand gestures to evaluate the performance of our model. The experimental results demonstrate the feasibility of our proposed system with an accuracy of 99.8% on average. Our approach allows for an accurate and nonindividual hand gesture recognition. It holds potential to be integrated into a smart watch or other wearable devices for intuitive human computer interaction.

Hand gesture recognition is a quickly developing field with many uses in human-computer interaction, sign language recognition, virtual reality, gaming, and robotics. This paper reviews different ways to model hands, such as vision-based, sensor-based, and data glove-based techniques. It emphasizes the importance of accurate hand modeling and feature extraction for capturing and analyzing gestures. Key features like motion, depth, color, shape, and pixel values and their relevance in gesture recognition are discussed. Challenges faced in hand gesture recognition include lighting variations, complex backgrounds, noise, and real-time performance. Machine learning algorithms are used to classify and recognize gestures based on extracted features. The paper emphasizes the need for further research and advancements to improve hand gesture recognition systems' robustness, accuracy, and usability. This review offers valuable insights into the current state of hand gesture recognition, its applications, and its potential to revolutionize human-computer interaction and enable natural and intuitive interactions between humans and machines. In simpler terms, hand gesture recognition is a way for computers to understand what people are saying with their hands. It has many potential applications, such as allowing people to control computers without touching them or helping people with disabilities communicate. The paper reviews different ways to develop hand gesture recognition systems and discusses the challenges and opportunities in this area.

DeReFNet: Dual-stream Dense Residual Fusion Network for static hand gesture recognition

At the present time, hand gestures recognition system could be used as a more\nexpected and useable approach for human computer interaction. Automatic hand\ngesture recognition system provides us a new tactic for interactive with the\nvirtual environment. In this paper, a face and hand gesture recognition system\nwhich is able to control computer media player is offered. Hand gesture and\nhuman face are the key element to interact with the smart system. We used the\nface recognition scheme for viewer verification and the hand gesture\nrecognition in mechanism of computer media player, for instance, volume\ndown/up, next music and etc. In the proposed technique, first, the hand gesture\nand face location is extracted from the main image by combination of skin and\ncascade detector and then is sent to recognition stage. In recognition stage,\nfirst, the threshold condition is inspected then the extracted face and gesture\nwill be recognized. In the result stage, the proposed technique is applied on\nthe video dataset and the high precision ratio acquired. Additional the\nrecommended hand gesture recognition method is applied on static American Sign\nLanguage (ASL) database and the correctness rate achieved nearby 99.40%. also\nthe planned method could be used in gesture based computer games and virtual\nreality.\n

Communication for mute people has always been very difficult. Mute people use hand gestures, also known as sign language to communicate with normal people. Hands gestures have their assigned meanings which may differ from person to person and hence cannot be understood by normal people. To overcome this difficulty many vision-based hand gesture recognition systems and data glove based hand gesture recognition systems have been proposed. This paper illustrates about two different techniques of vision-based hand gesture recognition and one data glove based technique. The vision — based techniques are static hand gesture recognition technique and real-time hand gesture recognition technique. In both the techniques, MATLAB software is used for processing input images and no dataset is used for decision making which makes this system more accurate than the existing system designs. In data glove based technique, the glove consists of five flex sensors. The change in resistance of flex sensor is used to recognize the hand gesture. All the three techniques are performed on 10 subjects and compared to find the most accurate technique. It was found that both the vision based techniques showed 100% accuracy in bright lighting condition with a white background while the data glove based technique showed an accuracy of 86%. This which clearly shows that mentioned vision based is more stable and reliable compared to the data glove based technique.

To get daily information from the internet has become most people's living habits today. In order to reduce the steps of receiving the information, such as complex mouse or keyboard steps, we propose a system designed for easily getting daily information without mouse and keyboard actions in this paper. We have implemented an approach providing daily information retrieved from the internet. Once the function has been selected by hand gestures, the system will report action information to users by synthesised speech. The system also utilises face recognition to identify each user, bringing up personalised services to each user. In this paper, we use PCA method to recognise faces as also hand gestures, and then a number of hand gestures and system controls are acquired and stored into this system. Experiment results show good performance in face recognition and hand gesture recognition.

Hand gesture recognition (HGR) has gained significant attention due to its potential for various applications. This paper explores the use of deep learning, specifically Convolutional Neural Networks (CNNs), for HGR using the TensorFlow library. We investigate existing research on CNN-based HGR, focusing on image classification tasks. We then provide a brief overview of CNNs and their suitability for image recognition. Subsequently, we describe the typical workflow of a deep learning-based HGR system, including data preprocessing, hand detection, feature extraction with CNNs, and classification. We highlight the advantages of using TensorFlow to build and train CNN models for HGR. Finally, we conclude by summarizing the key findings from related work and mentioning the specific dataset and number of gestures classified in our research. This work contributes to the growing body of research on CNN-based HGR using TensorFlow and emphasizes its potential for developing accurate and efficient HGR systems.

In this project we've discussed a system which uses some dynamic hand gesture recognition technique to manage the media players like VLC media player. This project consist of certain modules which segments the foreground a part of the frame using detection of skin colour and approximate median technique. the popularity of hand gesture is finished by creating a call Tree, that uses certain features extracted from the segmented part. This hand gesture recognition technique introduces a replacement, natural thanks to interact with computers and is beneficial to several folks in our day-to-day life. Hand gestures are associated with the human hands in hand gesture recognition. This project is employed for controlling certain operations (Pause, Play, Volume up, Volume Down, Mute) on video player by mere hand gestures without getting in the rigmarole of pressing buttons or tapping onto the screen. this may be directly associated with our day to life like in presentations. We have considered hand gestures and their directional motion defines a gesture for the applying. during this application image retrieving is finished employing a Webcam. Some functions in VLC media players are used most often and these functions uses some predefined gestures. Shows the defined gestures in keeping with the VLC player control operation. We created a VLC Media Player Controller using Hand Gesture Recognition System to form ‘HUMAN LIFE EASY AND BETTER’. This project is implemented using two steps: (1.) Creation of Hand Gesture Recognition System: this is often done using image processing using OpenCV library. (2.) Controlling VLC Media Player using hand gestures: during this step we controlled the player using shell commands which were recalled using python commands through OS library

Researchers have recently focused their attention on vision-based hand gesture recognition. However, due to several constraints, achieving an effective vision-driven hand gesture recognition system in real time has remained a challenge. This paper aims to uncover the limitations faced in image acquisition through the use of cameras, image segmentation and tracking, feature extraction, and gesture classification stages of vision-driven hand gesture recognition in various camera orientations. This paper looked at research on vision-based hand gesture recognition systems from 2012 to 2022. Its goal is to find areas that are getting better and those that need more work. We used specific keywords to find 108 articles in well-known online databases. In this article, we put together a collection of the most notable research works related to gesture recognition. We suggest different categories for gesture recognition-related research with subcategories to create a valuable resource in this domain. We summarize and analyze the methodologies in tabular form. After comparing similar types of methodologies in the gesture recognition field, we have drawn conclusions based on our findings. Our research also looked at how well the vision-based system recognized hand gestures in terms of recognition accuracy. There is a wide variation in identification accuracy, from 68% to 97%, with the average being 86.6 percent. The limitations considered comprise multiple text and interpretations of gestures and complex non-rigid hand characteristics. In comparison to current research, this paper is unique in that it discusses all types of gesture recognition techniques.

Hand gesture recognition is a challenging task due to the complexity of hand movements and to the variety among the same gesture performed by distinct subjects. Recent technologies, such as Kinect sensor, provide new opportunities, allowing to capture both RGB and depth (RGB-D) images, which offer high discriminant information for efficient hand gesture recognition. In the aspect of feature extraction, the traditional methods process the RGB and depth information independently. In this paper, we propose a robust hand gesture recognition system based on a new feature extraction method, fusing RGB images and depth information simultaneously, by using the quaternion algebra that provide a more robust and holistical representation. In fact, we introduce, for the first time, a novel type of feature extraction method, named quaternion Tchebichef moment invariants. The novelty of the proposed method in this paper lies in the direct derivation of invariants from their orthogonal moments, based on the algebraic properties of the discrete Tchebichef polynomials. The proposed approach based on quaternion algebra is suggested to process the four components holistically, for a robust and efficient hand gesture recognition system. The obtained experimental and theoretical results demonstrate that the present approach is very effective for addressing the problem of hand gesture recognition and have proved its robustness against geometrical distortion, noisy conditions and complex background compared to the state of the art, indicating that it could be highly useful for many computer vision applications.