Principal Component Analysis for Gesture Recognition Using SystemC

A gesture recognition system using stereo vision and arm model fitting

With the escalating role of computers in educational system, human computer interaction, is becoming gradually more important part of it. The general believe is that with the progress in computing speed, communication technologies, and display techniques the existing HCI techniques may become a constraint in the effectual utilization of the existing information flow. The development of user interfaces influences the changes in the Human-Computer Interaction (HCI). Human hand gestures have been a mode of non verbal interaction widely used. The vocabulary of hand gesture communication has many variations. It ranges from simple action of using our finger to point at and using hands to move objects around for more complex expressions for the feelings and communicating with others. Also the hand gestures play a prominent role in teaching considering the explanations and exemplifications being highly dependent on hand gestures. Naturalistic and intuitiveness of the hand gesture has been an immense motivating aspect for the researchers in the field of Human Computer Interaction to put their efforts to research and develop the more promising means of interaction involving human and computers. The pursuance for the Human Computer Interaction research is moved by the central dogma of removing the complex and cumbersome interaction devices and replacing them with more obvious and expressive means of interaction which easily comes to the users with least cognitive burden like hand gestures. This paper designs a simple, natural system for gestural interaction between the user and computer for providing a dynamic user interface. The gesture recognition system uses image processing techniques for detection, segmentation, tracking and recognition of hand gestures for converting it to a meaningful command. This hand gesture recognition system has been proposed, designed and developed with the intensions to make it a substitute for mouse while making dynamic user interface between human and machine. Hence instead of making effort to develop a new vocabulary of hand gesture we have matched control instruction set of mouse to subset of most discriminating hand gestures, so that we get a robust interface. The interface being proposed here can be substantially applied towards different applications like image browser, games etc.

This study presents a multimodal gesture recognition system based on the fusion of visual and strain sensor data, aiming to overcome the limitations of traditional gesture recognition methods in complex environments. A flexible strain sensor made from a CNTs-Graphene-TPU composite material was developed to create a smart glove with high sensitivity and stability, capable of accurately capturing finger flexion signals. Meanwhile, by integrating deep learning algorithms, a hybrid gesture recognition framework was constructed that combines visual and strain sensor data. Experimental results show that the system maintains high recognition accuracy (96.31%) across varying lighting conditions, significantly improving gesture recognition accuracy and system robustness. The system shows promising applications, particularly for gesture interaction and sign language translation in challenging lighting environments.

Gestures are a natural means of communication between humans, and also a natural modality for human-computer interaction. Automatic recognition of gestures using computer vision is an important task in many real-world applications, such as sign language recognition, computer games control, virtual reality, intelligent homes, and assistive environments. In order for a gesture recognition system to be robust and deployable in non-laboratory settings, the system needs to be able to operate in complex scenes, with complicated backgrounds and multiple moving and skin-colored objects. In this paper we propose an approach for improving gesture recognition performance in such complex environments. The key idea is to integrate a face detection module into the gesture recognition system, and use the face location and size to make gesture recognition invariant to scale and translation. Our experiments demonstrate the significant advantages of the proposed method over alternative computer vision methods for gesture recognition.

This work proposes a framework to train and deploy neural network-based gesture recognition algorithms in wearable devices. The approach is demonstrated for a high-density electromyography (HD-EMG) gesture recognition system, where a Siamese convolution neural network (SCNN) learns to associate and dissociate muscle activity patterns from the same or distinct gesture classes. This optimizes learning in low-data environments such as gesture recognition and myoelectric control, where training data must be provided by the end user. Then, using a cosine similarity-based few-shot classifier and inter-session-intra-user transfer of the SCNN’s learning, the proposed model is intended to achieve state-of-the-art results in a framework that is realistically applicable in wearable devices. For an experimented myoelectric interface user, the proposed model achieved 86.49 % accuracy in 6-class gesture recognition, using inter-session-intra-user transfer learning by the SCNN. Few-shot learning demonstrates convergence to this accuracy within 20 shots with the cosine similarity classifier. Across a group of 13 able-bodied participants, 90.66 % and 97.34 % mean and median accuracy were obtained from the 6-way 20-shot learning approach with a majority vote (100 votes) over time-distributed inferences.

With the rapid advancement of computer vision and deep learning technologies, the accuracy and efficiency of real-time gesture recognition have significantly improved. This paper introduces a gesture-controlled robot system based on thermal imaging sensors. By replacing traditional physical button controls, this design significantly enhances the interactivity and operational convenience of human–machine interaction. First, a thermal imaging gesture dataset is collected using Python3.9. Compared to traditional RGB images, thermal imaging can better capture gesture details, especially in low-light conditions, thereby improving the robustness of gesture recognition. Subsequently, a neural network model is constructed and trained using Keras, and the model is then deployed to a microcontroller. This lightweight model design enables the gesture recognition system to operate on resource-constrained embedded devices, achieving real-time performance and high efficiency. In addition, using a standalone thermal sensor for gesture recognition avoids the complexity of multi-sensor fusion schemes, simplifies the system structure, reduces costs, and ensures real-time performance and stability. The final results demonstrate that the proposed design achieves a model test accuracy of 99.05%. In summary, through its gesture recognition capabilities—featuring high accuracy, low latency, non-contact interaction, and low-light adaptability—this design precisely meets the core demands for “convenient, safe, and natural interaction” in rehabilitation, smart homes, and elderly assistive devices, showcasing clear potential for practical scenario implementation.

Gesture recognition plays an important role in human–computer interaction. However, most existing methods are complex and time-consuming, which limit the use of gesture recognition in real-time environments. In this paper, we propose a static gesture recognition system that combines depth information and skeleton data to classify gestures. Through feature fusion, hand digit gestures of 0–9 can be recognized accurately and efficiently. According to the experimental results, the proposed gesture recognition system is effective and robust, which is invariant to complex background, illumination changes, reversal, structural distortion, rotation, etc. We have tested the system both online and offline which proved that our system is satisfactory to real-time requirements, and therefore it can be applied to gesture recognition in real-world human–computer interaction systems.

Gesture recognition is an innovative technology which is fundamentally reshaping the way people live, entertain and work. However, most gesture recognition systems focus on the recognition of simple gestures and ignore the full potential of sequential gestures involving a series of temporally-related simple actions in order. This paper presents SGRS, a battery-free, scalable and non-specific sequential gesture recognition system based on COTS RFID. The key insight is that finegrained phase information extracted from RF signals is capable of perceiving various gestures. In SGRS, we meticulously devise gesture recognition mechanism by incorporating the k-means based vector quantizer and string matching algorithm to enable precise and real-time sequential gesture identification. Moreover, an improved edit distance algorithm is proposed for suppressing individual diversity. We implement SGRS and comprehensively evaluate the performance by recognizing traffic command gestures of Chinese traffic police. Experimental result shows that SGRS achieves an average recognition accuracy of 96.2% with eight sequential gestures and is highly robust to both individual diversity and multipath effect.

Wi-Fi sensing technology based on deep learning has contributed many breakthroughs in gesture recognition tasks. However, most methods concentrate on single domain recognition with high computational complexity while rarely investigating cross-domain recognition with lightweight performance, which cannot meet the requirements of high recognition performance and low computational complexity in an actual gesture recognition system. Inspired by the few-shot learning methods, we propose WiGR, a Wi-Fi-based gesture recognition system. The key structure of WiGR is a lightweight few-shot learning network that introduces some lightweight blocks to achieve lower computational complexity. Moreover, the network can learn a transferable similarity evaluation ability from the training set and apply the learned knowledge to the new domain to address domain shift problems. In addition, we made a channel state information (CSI)-Domain Adaptation (CSIDA) data set that includes channel state information (CSI) traces with various domain factors (i.e., environment, users, and locations) and conducted extensive experiments on two data sets (CSIDA and SignFi). The evaluation results show that WiGR can reach 87.8–94.8% cross-domain accuracy, and the parameters and the calculations are reduced by more than 50%. Extensive experiments demonstrate that WiGR can achieve excellent recognition performance using only a few samples and is thus a lightweight and practical gesture recognition system compared with state-of-the-art methods.

With the development of image processing and computer vision technology, using gesture to communicate with the machine will not only appear in scientific move or just a conceptual product. Gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. With this, we can have a more convenient life. Therefore, our goal is using image processing algorithm to effectively recognize the correct gesture from camera and present a user-friendly human-machine interface. In recent years, gesture recognition has become a popular and important issue. It can be used for robot control, appliances control, gaming control, etc. We present an algorithm which is able to correctly calculate the accurate finger joint position and then evaluate the gesture. This algorithm is divided into two parts: hand position detection and gesture recognition. In gesture detection, we capture the depth image from depth camera to solve the problem caused by illumination and background. In gesture recognition, we use an object recognizing algorithm to make the difficult evaluating finger movement problem corresponds to an easier voting classify problem. Depth camera helps our classifier avoid the illumination problem from incorrectly recognizing object.

Gesture recognition with electrical impedance tomography (EIT) is an enormous potential tool for human-machine interaction because of its low cost, low complexity and high temporal resolution. Although high-precision EIT-based gesture recognition has been achieved in ideal scenarios, ensuring its consistent performance under interference remains challenging. This article presents a novel method to alleviate the effect of confounding factors on EIT gesture recognition. An EIT armband was designed to mitigate the effect of contact impedance variation based on equivalent circuit analysis, and a spatial-temporal fusion network, named the Fold Atrous Spatial Pyramid Pooling-Gated Recurrent Unit (FASPP-GRU), was developed for robust gesture classification. The results showed that the proposed two-layer electrode maintained a stable contact impedance when its contact force with the skin was changed. Although confounding factors caused significant changes in baseline forearm impedance, FASPP-GRU achieved 80% accuracy under the effect of limb position changes and dynamic changes in muscle state over time, which outperforms conventional classifiers. With an 87 μs inference time, the proposed system shows enormous potential in real-time applications.

Device-free (passive) gesture recognition offers an enormous potential to simplify Human-Computer Interaction (HCI) in future smart environments. WIFI-based gesture recognition approaches have attained acclaim amongst others due to the omnipresence, privacy-preservation, and broad coverage of WIFI. However, there is no universal solution built on off-the-shelf devices that can accommodate an expandable set of gestures in a through-the-wall setting. In this work, we propose such a gesture recognition system that can recover information about the actual trajectory of the hand movement allowing an expandable set of gestures. Further, we leverage the rich multipath in a through-the-wall setting to develop a statistical model for the channel variations induced by a hand gesture. This model is used to derive a correspondence between the relative distance moved by the hand and the Time Reversal Resonating Strength (TRRS) decay. Based on this relation and the geometry of the gesture shape, we design feature extraction modules to enable gesture classification. We built a prototype of the proposed system on off-the-shelf WIFI devices and achieved a classification accuracy of 87% on a set of 6 uppercase English alphabets.

Gesture recognition and hand pose tracking are applicable techniques in human---computer interaction fields. Depth data obtained by depth cameras present a very informative explanation of the body or in particular hand pose that it can be used for more accurate gesture recognition systems. The hand detection and feature extraction process are very challenging task in the RGB images that they can be effectively dissolved with simple ways with depth data. However, depth data could be combined with the color information for more reliable recognition. A common hand gesture recognition system requires identifying the hand and its position or direction, extracting some useful features and applying a suitable machine-learning method to detect the performed gesture. This paper presents the novel fusion of the enhanced features for the classification of static signs of the sign language. It begins by explaining how the hand can be separated from the scene by depth data. Then, a combination feature extraction method is introduced for extracting some appropriate features of the images. Finally, an artificial neural network classifier is trained with these fused features and applied to critically analyze various descriptors performance.

Technologies for pattern recognition are used in various fields. One of the most relevant and important directions is the use of pattern recognition technology, such as gesture recognition, in socially significant tasks, to develop automatic sign language interpretation systems in real time. More than 5% of the world’s population—about 430 million people, including 34 million children—are deaf-mute and not always able to use the services of a living sign language interpreter. Almost 80% of people with a disabling hearing loss live in low- and middle-income countries. The development of low-cost systems of automatic sign language interpretation, without the use of expensive sensors and unique cameras, would improve the lives of people with disabilities, contributing to their unhindered integration into society. To this end, in order to find an optimal solution to the problem, this article analyzes suitable methods of gesture recognition in the context of their use in automatic gesture recognition systems, to further determine the most optimal methods. From the analysis, an algorithm based on the palm definition model and linear models for recognizing the shapes of numbers and letters of the Kazakh sign language are proposed. The advantage of the proposed algorithm is that it fully recognizes 41 letters of the 42 in the Kazakh sign alphabet. Until this time, only Russian letters in the Kazakh alphabet have been recognized. In addition, a unified function has been integrated into our system to configure the frame depth map mode, which has improved recognition performance and can be used to create a multimodal database of video data of gesture words for the gesture recognition system.

Hand gesture recognition systems often struggles when a gesture is occluded by the hand itself or by the other hand. To address this problem, multiview gestures may be used while training a gesture recognition system. Unfortunately, there are very few datasets available that contains different views of every single gesture. In this paper, we propose a method to handle occlusions in single-view sign gestures by generating multiple views from a single-view gesture using conditional multiview gesture synthesis. The generated views help to solve the occlusion problem, thereby enhancing the recognition performance. Additionally, we introduce a vision-based Multiview Hand Gesture Recognition (MHGR) framework that utilizes the generated multiview gestures for gesture recognition. Experiments conducted on the HGM-4 dataset demonstrate that the generated images are of high quality, photo-realistic, and significantly improve the recognition accuracy compared to some other existing methods.