A novel approach to frame reduction in robust hand gesture recognition

Dynamic Harris hawks optimizer based on historical information and tournament strategy and its application in numerical optimization of blast furnace ingredients

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

For hand gesture recognition (HGR) sensing based on frequency modulated continuous wave (FMCW) radar, it is challenging to cope with the ambiguity and variability of gesture motions performed by different peoples due to the private habits or position, which inevitably reduce the robustness of HGR sensing. To solve this problem, we propose a novel dual-flow fusion deformable residual network (DFDRN). More detailed gesture information, time frequency maps and spectrum map videos, are fed into DFDRN, which improves the generalization and motion modeling ability of HGR through the extensive feature space. To further improve the robustness of HGR sensing, the deformable block is exploited to replace the convolution. To focus on the important features of dual-flow fusion, the convolutional block attention module (CBAM) is embedded in the backbone of DFDRN. Experimental results show that the accuracy of DFDRN, which can reach 97.5%, is obviously better than the state-of-the-art HGR methods.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded system. In this work, we propose an HCI system for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially collected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efficiently with a sliding window approach. For the CNN model, several 3-dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver simulator operating in real-time. We make DriverMHG dataset and our source code publicly available 1.1https://www.mmk.ei.tum.de/DriverMHG/

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

To naturally interact with virtual environment by hand gesture, this paper presents a robust RGB-D data based recognition method of static and dynamic hand gesture. Firstly, for static hand gesture recognition, starting from the hand gesture contour extraction, the palm center is identified by Distance Transform (DT) algorithm. The fingertips are localized by employing the K-Curvature-Convex Defects Detection algorithm (K-CCD). On the basis, the distances of the pixels on hand gesture contour to palm center and the angle between fingertips are considered as the auxiliary features to construct a multimodal feature vector, and then recognition algorithm is presented to robustly recognize the static hand gestures. Secondly, combining Euclidean distance between hand joints and shoulder center joint with the modulus ratios of skeleton features, this paper generates a unifying feature descriptor for each dynamic hand gesture and proposes an improved dynamic time warping (IDTW) algorithm to obtain recognition results of dynamic hand gestures. Finally, we conduct extensive experiments to test and verify the static and dynamic hand gesture recognition algorithm and realize a low-cost real-time application of natural interaction with virtual environment by hand gestures.

Hand gestures aid the user in unspoken communication and help interpret sign language. Modern nonverbal communication detection employs gesture recognition systems and allows us to recognize shapes and signals made by hand movements using numerous machine learning techniques. Gestures have a very high value in human-computer interaction and sign language detection systems. To implement the gesture recognition and detection the existing methodologies rely on the application of bodily sensors. Due to this, the effectiveness in communication becomes challenging and cumbersome. Hence targeting on the major fallacies of the existing approaches, the primary aspiration of this research work is to recognize hand gestures in real-time using depth sensors that support natural communication avoiding on body sensor dependency. A novel approach is presented to detect hand gestures using minimalistic training and the efficiency of other algorithms concerning this proposed technique has been compared. Experimental investigations on the proposed methodology have successfully confirmed that, the research exertion has obtained an overall gesture detection accuracy of 90.86% and a high precision of 98.25%.

In the recent era of smart world, smarter technologies are gaining focus on human computer interaction (HCI) systems, and traditional ways like remote control, mouse, keyboard, etc. are becoming less popular. This paper presents a framework of simple yet efficient approach for future applications of the new age smart and intelligent technologies that shall enhance the HCI specifically for smart television. Hand gesture recognition (HGR)-based model is proposed for wireless control of smart interactive television (SITV), which includes controlling volume and selecting channels. The proposed framework undergoes three steps: 1) the hand gesture of the person is detected by using shape, colour and skin similarity; 2) the extracted features are classified by using rule-based classification and a gesture code is generated; 3) the classified gesture is interpreted by a novel interpreter. The performance of the proposed framework is evaluated with different people's hand gestures and compared with the techniques of others.

A hybrid model of commercial building cooling load prediction based on the improved NCHHO-FENN algorithm

To address shortcomings of ICP algorithm, which is sensitive to the initial position and easy to fall into local optimum, we propose a Harris Hawks Optimizer (HHO) point cloud registration method combining PCA preprocessing and parallel population initialization. The PCA method is used to align the point clouds with the main axes and perform the main axes correction. Since the six input variables in the population initialization stage of HHO point cloud registration are different and independent, the initial population of HHO algorithm is initialized by the parallel initialization method. The results show that the improved algorithm is able to successfully overwhelm the fault of ICP algorithm, and its global seek performance and optimization accuracy are more advantageous, and the algorithm operates efficiently, with high registration accuracy and robustness.

Human-computer interaction for virtual-real fusion

Proton exchange membrane fuel cell (PEMFC) is considered as propitious solution for an environmentally friendly energy source. A precise model of PEMFC for accurate identification of its polarization curve and in-depth understanding of all its operating characteristics attracted the interest of many researchers. In this paper, novel meta-heuristic optimization methods have been successfully applied to evaluate the unknown parameters of PEMFC models, particularly Harris Hawks’ optimization (HHO) and atom search optimization (ASO) techniques. The proposed optimization algorithms have been tested on three different commercial PEMFC stacks, namely BCS 500-W PEM, 500W SR-12PEM and 250W stack, under various operating conditions. The sum of square errors (SSE) between the results obtained by the application of the estimated parameters and the experimentally measured results of the fuel cell stacks was considered as the objective function of the optimization problem. In order to validate the effectiveness of the proposed methods, the results are compared with that obtained in studies. Moreover, the I/V curves obtained by the application of HHO and ASO showed a clear matching with data sheet curves for all the studied cases. Finally, PEMFC model based on HHO technique surpasses all compared algorithms in terms of the solution accuracy and the convergence speed.

Considerable effort has been put toward the development of intelligent and natural interfaces between users and computer systems. In line with this endeavor, several modes of information (e.g., visual, audio, and pen) that are used either individually or in combination have been proposed. The use of gestures to convey information is an important part of human communication. Hand gesture recognition is widely used in many applications, such as in computer games, machinery control (e.g., crane), and thorough mouse replacement. Computer recognition of hand gestures may provide a natural computer interface that allows people to point at or to rotate a computer-aided design model by rotating their hands. Hand gestures can be classified into two categories: static and dynamic. The use of hand gestures as a natural interface serves as a motivating force for research on gesture taxonomy, its representations, and recognition techniques. This paper summarizes the surveys carried out in human--computer interaction (HCI) studies and focuses on different application domains that use hand gestures for efficient interaction. This exploratory survey aims to provide a progress report on static and dynamic hand gesture recognition (i.e., gesture taxonomies, representations, and recognition techniques) in HCI and to identify future directions on this topic.

Feature selection is a required preprocess stage in most of the data mining tasks. This paper presents an improved Harris hawks optimization (HHO) to find high-quality solutions for global optimization and feature selection tasks. This method is an efficient optimizer inspired by the behaviors of Harris' hawks, which try to catch the rabbits. In some cases, the original version tends to stagnate to the local optimum solutions. Hence, a novel HHO called IHHO is proposed by embedding the salp swarm algorithm (SSA) into the original HHO to improve the search ability of the optimizer and expand the application fields. The update stage in the HHO optimizer, which is performed to update each hawk, is divided into three phases: adjusting population based on SSA to generate SSA-based population, generating hybrid individuals according to SSA-based individual and HHO-based individual, and updating search agent in the light of greedy selection and HHO’s mechanisms. A large group of experiments on many functions is carried out to investigate the efficacy of the proposed optimizer. Based on the overall results, the proposed IHHO can provide a faster convergence speed and maintain a better balance between exploration and exploitation. Moreover, according to the proposed continuous IHHO, a more stable binary IHHO is also constructed as a wrapper-based feature selection (FS) approach. We compare the resulting binary IHHO with other FS methods using well-known benchmark datasets provided by UCI. The experimental results reveal that the proposed IHHO has better accuracy rates over other compared wrapper FS methods. Overall research and analysis confirm the improvement in IHHO because of the suitable exploration capability of SSA.