Raspberry Based Hand Gesture Recognition Using Haar Cascade and Local Binary Pattern Histogram

The online version contains supplementary material available at 10.1007/s00521-022-08090-8.

In this article, a novel method for continuous hand gesture detection and recognition is proposed based on a frequency modulated continuous wave (FMCW) radar. Firstly, we adopt the 2-Dimensional Fast Fourier Transform (2D-FFT) to estimate the range and Doppler parameters of the hand gesture raw data, and construct the range-time map (RTM) and Doppler-time map (DTM). Meanwhile, we apply the Multiple Signal Classification (MUSIC) algorithm to calculate the angle and construct the angle-time map (ATM). Secondly, a hand gesture detection method is proposed to segment the continuous hand gestures using a decision threshold. Thirdly, the central time-frequency trajectory of each hand gesture spectrogram is clustered using the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means algorithm, and then the Fusion Dynamic Time Warping (FDTW) algorithm is presented to recognize the hand gestures. Finally, experiments show that the accuracy of the proposed hand gesture detection method can reach 96.17%. The hand gesture average recognition accuracy of the proposed FDTW algorithm is 95.83%, while its time complexity is reduced by more than 50%.

The goal of static hand gesture recognition is to classify the given hand gesture data represented by some features into some prede ned nite number of gesture classes. The main objective of this e ort is to explore the utility of two feature extraction methods, namely, hand contour and complex moments to solve the hand gesture recognition problem by identifying the primary advantages and disadvantages of each method. Arti cial neural network is built for the purpose of classi cation by using the back-propagation learning algorithm. The proposed system presents a recognition algorithm to recognize a set of six speci c static hand gestures, namely: Open, Close, Cut, Paste, Maximize, and Minimize. The hand gesture image is passed through three stages, namely, pre-processing, feature extraction, and classi cation. In the pre-processing stage some operations are applied to extract the hand gesture from its background and prepare the hand gesture image for the feature extraction stage. In the rst method, the hand contour is used as a feature which treats scaling and translation problems (in some cases). The complex moments algorithm is, however,

The video streams of backgrounds are frequently influenced by the background changes (e.g. illumination changes and changes due to adding or removing parts of the background). Further more, the quality of the foreground and the segmented image of hand gesture severely drops. We propose a novel method, which is based on difference background image between consecutive video frames, of using the '3sigma -principle' of normal distribution for hand gesture detection to cope with the problem. The adaptive method of automatic threshold selection based on the method of maximal between-class variance is proposed for hand gesture segmentation to select optimal threshold. Experimentations show that the better images are obtained with complex background, no matter if the proportion of the hand gesture is high or low. Several experimental images are presented to support the validity of the method.

With the escalating role of computers in educational system, human computer interaction, is becoming gradually more important part of it. The general believe is that with the progress in computing speed, communication technologies, and display techniques the existing HCI techniques may become a constraint in the effectual utilization of the existing information flow. The development of user interfaces influences the changes in the Human-Computer Interaction (HCI). Human hand gestures have been a mode of non verbal interaction widely used. The vocabulary of hand gesture communication has many variations. It ranges from simple action of using our finger to point at and using hands to move objects around for more complex expressions for the feelings and communicating with others. Also the hand gestures play a prominent role in teaching considering the explanations and exemplifications being highly dependent on hand gestures. Naturalistic and intuitiveness of the hand gesture has been an immense motivating aspect for the researchers in the field of Human Computer Interaction to put their efforts to research and develop the more promising means of interaction involving human and computers. The pursuance for the Human Computer Interaction research is moved by the central dogma of removing the complex and cumbersome interaction devices and replacing them with more obvious and expressive means of interaction which easily comes to the users with least cognitive burden like hand gestures. This paper designs a simple, natural system for gestural interaction between the user and computer for providing a dynamic user interface. The gesture recognition system uses image processing techniques for detection, segmentation, tracking and recognition of hand gestures for converting it to a meaningful command. This hand gesture recognition system has been proposed, designed and developed with the intensions to make it a substitute for mouse while making dynamic user interface between human and machine. Hence instead of making effort to develop a new vocabulary of hand gesture we have matched control instruction set of mouse to subset of most discriminating hand gestures, so that we get a robust interface. The interface being proposed here can be substantially applied towards different applications like image browser, games etc.

A real-time human-robot interaction framework with robust background invariant hand gesture detection

Stationary hand gesture recognition can be applied in a variety of domains such as human-computer interaction (HCI), remote control, virtual reality etc. Hand gesture recognition is mainly the study of detection and recognition of various hand gestures like Sign Language hand gestures, by a computer. This work is focused on three main issues in developing a gesture recognition system. These are (i) Threshold (ii) illumination normalization (iii) user independent gesture recognition based on fusion of Moments. It is very demanding to segment hand regions from the static hand gesture color images, due to unreliable light conditions and composite background. Since skin pixels can vary with different explanation condition, to find the range of skin pixels, becomes a hard task in case of color space based skin color segmentation. These methods propose a learning algorithm based on customized clustering expanse to remove person skin color regions from the stationary hand gesture color descriptions. To improve the performance of classification, two feature synthesis strategies have been proposed in this work.

Sign language (SL) is the linguistics of speech and hearing-impaired individuals. The hand gesture is the primary model employed in SL by speech and hearing-challenged people to talk with themselves and ordinary persons. At present, hand gesture detection plays a vital part, and it is commonly employed in numerous applications worldwide. Hand gesture detection systems can aid in transmission between machines and humans by aiding these sets of people. Machine learning (ML) is a subdivision of artificial intelligence (AI), which concentrates on the growth of a method. The main challenge in hand gesture detection is that machines do not directly understand human language. A standard medium is required to facilitate communication between humans and machines. Hand gesture recognition (GR) serves as this medium, enabling commands for computer interaction that specifically benefit hearing-impaired and elderly individuals. This study proposes a Gesture Recognition for Hearing Impaired People Using an Ensemble of Deep Learning Models with Improving Beluga Whale Optimization (GRHIP-EDLIBWO) model. The main intention of the GRHIP-EDLIBWO model framework for GR is to assist as a valuable tool for developing accessible communication systems for hearing-impaired individuals. To accomplish that, the GRHIP-EDLIBWO method initially performs image preprocessing using a Sobel filter (SF) to enhance edge detection and extract critical gesture features. For the feature extraction process, the squeeze-and-excitation capsule network (SE-CapsNet) effectively captures spatial hierarchies and complex relationships within gesture patterns. In addition, an ensemble of classification processes, such as bidirectional gated recurrent unit (BiGRU), Variational Autoencoder (VAE), and bidirectional long short-term memory (BiLSTM) technique, is employed. Finally, the improved beluga whale optimization (IBWO) method is implemented for the hyperparameter tuning of the three ensemble models. To achieve a robust classification result with the GRHIP-EDLIBWO approach, extensive simulations are conducted on an Indian SL (ISL) dataset. The performance validation of the GRHIP-EDLIBWO approach portrayed a superior accuracy value of 98.72% over existing models.

Abstract— In this digital era, face recognition system plays a vital role in almost every sector. Face recognition is one of the mostly used biometrics. It can used for security, authentication, identification, and has got many more advantages. Despite of having low accuracy when compared to iris recognition and fingerprint recognition, it is being widely used due to its contactless and non-invasive process. Furthermore, face recognition system can also be used for attendance marking in schools, colleges, offices, etc. This system aims to build a class attendance system which uses the concept of face recognition as existing manual attendance system is time consuming and cumbersome to maintain. And there may be chances of proxy attendance. Thus, the need for this system increases. This system consists of four phases- database creation, face detection, face recognition, attendance updation. Database is created by the images of the students in class. Face detection and recognition is performed using Haar-Cascade classifier and Local Binary Pattern Histogram algorithm respectively. Faces are detected and recognized from live streaming video of the classroom. Keywords—Face Recognition; Face Detection; Haar-Cascade; Opencv :library are used in face recognition system.

Face recognition algorithms enable computational devices to automatically recognize human faces and have been adopted by many big tech companies. These algorithms have shown fairly acceptable performance in criminal identification, healthcare, advertisement, access and security, payments and other different areas. In this paper, we attempt comparatively to identify best algorithms in terms of accuracy when applied on datasets that have considerable facial expression and pose illumination challenges. We selected benchmark datasets and applied preprocessing techniques to suppress noise in the images. We then applied different algorithms for feature extraction and then feed these features to the classifier. Based on the conducted experiments, we observed that the Local Binary Pattern Histogram algorithm outperformed the other two selected algorithms by about 1% against Linear Discriminant Analysis (LDA) and about 9% against Principal Component Analysis (PCA). In addition, LDA outperformed the other two by obtaining about 99.532% of accuracy on JAFFE dataset.

The biometric-based attendance monitoring system replaces the manual method of taking attendance and reduces the time to take attendance and avoids proxy. There are many biometric methods developed based on the fingerprint recognition, DNA matching, face recognition, Iris recognition and voice recognition. Out of all, the methods based on face recognition are proved to be the best methods. In this paper we are presenting the implementation of attendance monitoring system based on the facial recognition. In this method the image database of the classroom is created first and the database is used to train the system. Then in recognition phase the camera is fixed at the focal point in the classroom and it will capture the images, then from the captured images student's faces are recognized & compared with the database and lastly the attendance is marked based on the faces identified. The marked attendance is directly reported to the students/parents and to the faculty handling the respective course. In this method, Haar cascade classifier is used to detect the faces and captured images are compared with the database by using local binary pattern histogram algorithm. The face recognition system is implemented on the Raspberry Pi 3B hardware.

In this digital time, face recognition system is one the challenging area of pattern recognition. Face recognition has wide scope for security, verification, recognizable proof, and has got numerous more points of interest. Face recognition system has more enhanced as compared to biometric based one. It has low exactness when compared to iris recognition and unique mark acknowledgment, hence utilized for participation checking in schools, colleges, workplaces, etc. This framework aims to construct a course participation system which employments the concept of face recognition as the existing manual participation framework. This framework comprises of four stages-database creation, confront discovery, and confront acknowledgment and participation. A standard database has been harnessed.Confront discovery and acknowledgment is performed using Haar-Cascade classifier and Local Binary Pattern Histogram algorithm separately. Faces are recognized and recognized from live spilling video of the classroom and reported individual workforce at the conclusion of the session.

The attendance application is one important system that must be possessed by a school in order to manage student attendance data. Some schools still use manual methods for student attendance by using signatures or calling them one by one. Manual attendance is considered inefficient because the risk of losing attendance sheets is very easy and parents cannot monitor whether their child is attending school or not. This is directly felt by SMA Dua Mei Ciputat with the presence of students who left home but did not attend school. Therefore, this research aims to assist teachers and parents in monitoring students to be more disciplined in coming to school by building a face recognition application for attendance using the haar-cascade classifier method based on a website. The haar-cascade classifier method has the advantages of computation in face detection and the local binary pattern histogram algorithm which has good accuracy in face recognition. The results of this research show that the haar-cascade classifier method can be used to create a face recognition application for attendance with fairly good accuracy, namely 100% with 5 attempts with recognition of one face, 100% with 5 attempts with recognition of two faces, and 20% with 5 attempts with recognition of three faces. With these results, the application is only capable of recognizing two faces simultaneously in the attendance process.

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

A novel hand biometric authentication method based on measurements of the user’s stationary hand gesture of hand sign language is proposed. The measurement of hand gestures could be sequentially acquired by a low-cost video camera. There could possibly be another level of contextual information, associated with these hand signs to be used in biometric authentication. As an analogue, instead of typing a password ‘iloveu’ in text which is relatively vulnerable over a communication network, a signer can encode a biometric password using a sequence of hand signs, ‘i’ , ‘l’ , ‘o’ , ‘v’ , ‘e’ , and ‘u’. Subsequently the features from the hand gesture images are extracted which are integrally fuzzy in nature, to be recognized by a classification model for telling if this signer is who he claimed himself to be, by examining over his hand shape and the postures in doing those signs. It is believed that everybody has certain slight but unique behavioral characteristics in sign language, so are the different hand shape compositions. Simple and efficient image processing algorithms are used in hand sign recognition, including intensity profiling, color histogram and dimensionality analysis, coupled with several popular machine learning algorithms. Computer simulation is conducted for investigating the efficacy of this novel biometric authentication model which shows up to 93.75% recognition accuracy.