Real time hand gesture recognition using AVR microcontroller

A multi-scale descriptor for real time RGB-D hand gesture recognition

This paper outlines a system design and implementation of a 3D input device for graphical applications which uses real time hand tracking and gesture recognition to provide the user with an intuitive interface for tomorrow’s applications. Point Distribution Models (PDMs) have been shown to be successful at tracking deformable objects . This system demonstrates how these ‘smart snakes’ can be used in real time with a real world problem. The system is based upon Open Inventor1 and designed for use with Silicon Graphics Indy Workstations, but provisions have been make for the move to other platforms and applications. We demonstrate how PDMs provide the ideal feature vector for model classification. It is shown how computer vision can provide a low cost, intuitive interface that has few hardware constraints. We also give the reader an insight into the next generation of HCI and Multimedia, providing a 3D scene viewer and VRML browser based upon the handtracker. Further allowances have been made to facilitate the inclusion of the handtracker within third party Inventor applications. All source code, libraries and applications can be downloaded for free from the above web addresses. This paper demonstrates how computer vision and computer graphics can work together providing an interdisciplinary approach to problem solving.

Instant 3D design concept generation and visualization by real-time hand gesture recognition

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

The natural user interface (NUI) is used for the natural motion interface without using device or tool such as mice, keyboards, pens and markers. In this paper, we develop natural user interface framework based on two recognition module. First module is real-time head pose estimation module using random forests and second module is hand gesture recognition module, named Hand gesture Key Emulation Toolkit (HandGKET). Using the head pose estimation module, we can know where the user is looking and what the user’s focus of attention is. Moreover, using the hand gesture recognition module, we can also control the computer using the user’s hand gesture without mouse and keyboard. In proposed framework, the user’s head direction and hand gesture are mapped into mouse and keyboard event, respectively.

A novel polymer MEMS (micro electro mechanical systems) accelerometer with photo-patternable polymer nanocomposite as a piezoresistor is presented in this work. Polymer MEMS Accelerometer with beam thicknesses of 3.3 µm and embedded nanocomposite piezoresistive layer having a gauge factor of 90 were fabricated. The photosensitive nanocomposite samples were prepared and characterized for analyzing the mechanical and electrical properties and thereby ensuring proper process parameters for incorporating the piezoresistive layer into the polymer MEMS accelerometer. The microfabrication process flow and unit processes followed are extremely low cost with process temperatures below 100 °C. This also opens up a new possibility for easy integration of such polymer MEMS with CMOS (complementary metal oxide semiconductor) devices and circuits. The fabricated devices were characterized using laser Doppler vibrometer (LDV) and the devices exhibited a resonant frequency of 10.8 kHz and a response sensitivity of 280 nm g−1 at resonance. The main focus of this paper is on the SU-8/CB nanocomposite piezoresistive MEMS accelerometer technology development which covers the material and the fabrication aspects of these devices. CoventorWare FEA analysis performed using the extracted material properties from the experimental characterization which are in close agreement to performance parameters of the fabricated devices is also discussed. The simulated piezoresistive polymer MEMS devices showed an acceleration sensitivity of 126 nm g−1 and 82 ppm of ΔR/R per 1 g of acceleration.

Micro-electro mechanical system (MEMS) accelerometers are small, inexpensive sensors that have only recently been used in geotechnical centrifuge tests. This is unlike piezoelectric accelerometers, which are by comparison large and expensive but have been used extensively in geotechnical centrifuge tests over the past couple of decades. This paper examines the response of a single-axis ±500g MEMS accelerometer under both static and dynamic conditions in a centrifuge environment. The potential for MEMS accelerometers to be used to measure the depth of objects buried in soil is examined and the achievable resolution is discussed. Unlike piezoelectric accelerometers, which only measure changes in acceleration, MEMS accelerometers can measure both constant and changing accelerations. The merit of this feature is demonstrated through tests in which MEMS and piezoelectric accelerometers are embedded within a dynamically installed model anchor. The MEMS accelerometer is capable of measuring accelerations during both the free-fall phase and the soil embedment phase, whereas the piezoelectric accelerometer is only able to measure the changing accelerations that dominate during the soil embedment phase. Velocity profiles derived from numerical integration of the MEMS accelerations give mudline anchor velocities that agree with independent measurements and anchor embedment depths that agree with direct measurements.

Communication for mute people has always been very difficult. Mute people use hand gestures, also known as sign language to communicate with normal people. Hands gestures have their assigned meanings which may differ from person to person and hence cannot be understood by normal people. To overcome this difficulty many vision-based hand gesture recognition systems and data glove based hand gesture recognition systems have been proposed. This paper illustrates about two different techniques of vision-based hand gesture recognition and one data glove based technique. The vision — based techniques are static hand gesture recognition technique and real-time hand gesture recognition technique. In both the techniques, MATLAB software is used for processing input images and no dataset is used for decision making which makes this system more accurate than the existing system designs. In data glove based technique, the glove consists of five flex sensors. The change in resistance of flex sensor is used to recognize the hand gesture. All the three techniques are performed on 10 subjects and compared to find the most accurate technique. It was found that both the vision based techniques showed 100% accuracy in bright lighting condition with a white background while the data glove based technique showed an accuracy of 86%. This which clearly shows that mentioned vision based is more stable and reliable compared to the data glove based technique.

Real-time hand gesture recognition in FPGA

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

Tracking moving objects is an area increasingly known in computer vision field. It plays a very important role in human-computer interaction. In this context we have developed a hand tracking and gesture recognition system that allows interaction with the machine in an intuitive and natural way. To ensure the tracking we apply the Kalman filter and detect the optimal points of the hand in order to determine the gesture expressed by user.

With recent development in wireless communication and Micro Electro Mechanical Systems (MEMS) technology, it becomes easier to monitor rotating machinery conditions by mounting compact wireless MEMS accelerometers directly on the rotor. This has the potential to provide more accurate dynamic characteristics of the rotating machine and hence achieving high monitoring performance. In this paper, a tiny MEMS accelerometer together with a battery powered microcontroller is mounted on the flywheel to acquire the on-rotor accelerations of a two-stage reciprocating compressor. The measured acceleration data is streamed to a host computer wirelessly via Bluetooth Low Energy (BLE) module. The true tangential acceleration is reconstructed by combining two orthogonal outputs of the sensor, which contain gravitational accelerations. To evaluate the performance of the wireless sensor, three different fault conditions including intercooler leakage, second stage discharge valve leakage and asymmetric stator winding of the motor driver are simulated individually on the compressor test rig. To confirm the wireless sensor performance, an incremental optical encoder was installed on the compressor flywheel to acquire the Instantaneous Angular Speed (IAS) signal for comparison with signals from the wireless sensor. The experimental results show that the running status of the compressor can be remotely monitored, allowing different leakages and motor faults to be diagnosed based on the tangential acceleration reconstructed from a wireless on-rotor MEMS accelerometer.

In today’s ever-evolving healthcare landscape, our objective is to bring technology closer to human experience. In this paper, a vision-based hand gesture recognition (HGR) system is introduced as a human computer interface (HCI) that transforms natural movements into intuitive medium for device and application manipulation. This innovative approach is designed to empower the elderly and enhance patient monitoring by offering a seamless, nonintrusive interface. By bridging compassion with cutting-edge technology, this approach aims to redefine care and create a more connected, responsive healthcare environment. The study begins by reviewing both traditional and contemporary HGR methodologies, with a focus on vision-based systems, which have shown greater potential for practical applications compared to conventional sensor-based systems. Traditional approaches, such as contour analysis, colour segmentation, and template matching, have demonstrated limitations. In contrast, deep learning approaches have gained mass adaptation in vision-based systems for their ability to learn hierarchical features and better handling complex, dynamic scenarios. However, existing models struggle in real-time performance due to computational inefficiencies and environmental variations. This study introduces a deep learning-based detection framework which enables the practical application of HGR. For example, it supports the development of gesture-based virtual assistants for interacting with digital devices and allows elderly residents in care homes to give instructions to remote caregivers. The framework consists of two deep learning models, one for region of interest (ROI) detection and another for classification. The recognition process begins with capturing RGB images, chosen for versatility, affordability, and compatibility, which then passes through YOLOv8 (You Only Look Once version 8) for detection. The YOLOv8 model is trained on a subset, only 4.5%, of the HaGRID dataset, comprises over 550,000 RGB images, to accurately locate hand regions. Once detection is complete, several image processing techniques are applied to overcome typical HGR constraints. These include ROI extraction and gray scaling for faster computation, resizing for model optimization and distance viability, HSV conversion for lighting independence, and histogram equalization for enhanced feature extraction. The processed image is then fed into a convolutional neural network (CNN) for gesture classification. The network is designed with several convolutional layers, followed by dense layers with ReLU activation to introduce non-linearity, and SoftMax activation for classification. To improve the model's robustness, data augmentation techniques such as random flipping and rotation are applied. The HGR system demonstrates a staggering accuracy of 97% for training & validation. The proposed framework effectively addresses the limitations of conventional HGR systems—many of which have been developed and tested in controlled environments—demonstrating its applicability in real-life situations.

Hand gesture recognition can be exploited to benefit ubiquitous applications using sensors. Currently, the inherent complexity of human physical activities makes it difficult to accurately recognize gestures with wearable sensors, especially in real time. To this end, a real-time hand gesture recognition system is presented in this paper. In particular, sliding window technology and y-axis threshold are used to detect intended gestures from a continuous data stream and then the segmented data are classified by applying a stacked Long Short-Term Memory (LSTM) model. After noise is removed, six-axis sensor data from wrist-worn devices are fed into the model without requiring feature engineering. We use twelve common hand gestures to evaluate the performance of our model. The experimental results demonstrate the feasibility of our proposed system with an accuracy of 99.8% on average. Our approach allows for an accurate and nonindividual hand gesture recognition. It holds potential to be integrated into a smart watch or other wearable devices for intuitive human computer interaction.

In this project we've discussed a system which uses some dynamic hand gesture recognition technique to manage the media players like VLC media player. This project consist of certain modules which segments the foreground a part of the frame using detection of skin colour and approximate median technique. the popularity of hand gesture is finished by creating a call Tree, that uses certain features extracted from the segmented part. This hand gesture recognition technique introduces a replacement, natural thanks to interact with computers and is beneficial to several folks in our day-to-day life. Hand gestures are associated with the human hands in hand gesture recognition. This project is employed for controlling certain operations (Pause, Play, Volume up, Volume Down, Mute) on video player by mere hand gestures without getting in the rigmarole of pressing buttons or tapping onto the screen. this may be directly associated with our day to life like in presentations. We have considered hand gestures and their directional motion defines a gesture for the applying. during this application image retrieving is finished employing a Webcam. Some functions in VLC media players are used most often and these functions uses some predefined gestures. Shows the defined gestures in keeping with the VLC player control operation. We created a VLC Media Player Controller using Hand Gesture Recognition System to form ‘HUMAN LIFE EASY AND BETTER’. This project is implemented using two steps: (1.) Creation of Hand Gesture Recognition System: this is often done using image processing using OpenCV library. (2.) Controlling VLC Media Player using hand gestures: during this step we controlled the player using shell commands which were recalled using python commands through OS library