Hand Gesture Recognition for Real-Time Nano Drone Control on RISC-V

In order to realize high-accuracy recognition of aerobics actions, a highly applicable deep learning model and faster data processing methods are required. Therefore, it is a major difficulty in the field of research on aerobics action recognition. Based on this, this paper studies the application of the convolution neural network (CNN) model combined with the pyramid algorithm in aerobics action recognition. Firstly, the basic architecture of the convolution neural network model based on the pyramid algorithm is proposed. Combined with the application strategy of the common recognition model in aerobics action recognition, the traditional aerobics action capture information is processed. Through the characteristics of different aerobics actions, different accurate recognition is realized, and then, the error of the recognition model is evaluated. Secondly, the composite recognition function of the convolution neural network model in this application is constructed, and the common data layer effect recognition method is used in the optimization recognition. Aiming at the shortcomings of the composite recognition function, the pyramid algorithm is used to improve the convolution neural network recognition model by deep learning optimization. Finally, through the effectiveness comparison experiment, the results show that the convolution neural network model based on the pyramid algorithm is more efficient than the conventional recognition method in aerobics action recognition.

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

The rapid advancement of edge computing and Tiny Machine Learning (TinyML) has created new opportunities for deploying intelligence in resource-constrained environments. With the growing demand for intelligent Internet of Things (IoT) devices that can efficiently process complex data in real-time, there is an urgent need for innovative optimisation techniques that overcome the limitations of IoT devices and enable accurate and efficient computations. This study investigates a novel approach to optimising Convolutional Neural Network (CNN) models for Hand Gesture Recognition (HGR) based on Electrical Impedance Tomography (EIT), which requires complex signal processing, energy efficiency, and real-time processing, by simultaneously reducing input complexity and using advanced model compression techniques. By systematically reducing and halving the input complexity of a 1D CNN from 40 to 20 Boundary Voltages (BVs) and applying an innovative compression method, we achieved remarkable model size reductions of 91.75% and 97.49% for 40 and 20 BVs EIT inputs, respectively. Additionally, the Floating-Point operations (FLOPs) are significantly reduced, by more than 99% in both cases. These reductions have been achieved with a minimal loss of accuracy, maintaining the performance of 97.22% and 94.44% for 40 and 20 BVs inputs, respectively. The most significant result is the 20 BVs compressed model. In fact, at only 8.73 kB and a remarkable 94.44% accuracy, our model demonstrates the potential of intelligent design strategies in creating ultra-lightweight, high-performance CNN-based solutions for resource-constrained devices with near-full performance capabilities specifically for the case of HGR based on EIT inputs.

Gesture recognition has multiple applications in medical and engineering fields. The problem of hand gesture recognition consists of identifying, at any moment, a given gesture performed by the hand. In this work, we propose a new model for hand gesture recognition in real time. The input of this model is the surface electromyography measured by the commercial sensor the Myo armband placed on the forearm. The output is the label of the gesture executed by the user at any time. The proposed model is based on the Λ-nearest neighbor and dynamic time warping algorithms. This model can learn to recognize any gesture of the hand. To evaluate the performance of our model, we measured and compared its accuracy at recognizing 5 classes of gestures to the accuracy of the proprietary system of the Myo armband. As a result of this evaluation, we determined that our model performs better (86% accurate) than the Myo system (83%).

A hidden Markov model (HMM) has been widely used as a cost-effective solution for time series based applications such as hand gesture recognition. A large number of parameters in an HMM often result in a degradation of classification accuracy due to overfitting. Furthermore, the accuracy is often affected by the initialization values of the parameters. Left-right banded (LRB) HMM is one of the representative models which improve the classification accuracy by reducing the number of parameters. However, LRB HMM still suffers from false negative misclassification when it is applied for hand gesture recognition. Misclassification in hand gesture recognition arise from similar trajectories of multiple gestures that are represented by hidden states. This paper proposes a new initialization process that reduces not only state transition parameters but also hidden state observation parameters. To this end, the gesture trajectory is predicted and then the parameters are masked out when their probabilities are negligibly small in the predicted trajectory. Experiment results show that the confusion rate is nearly 14% lower than the LRB HMM while maintaining the classification accuracy.

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

Accurately identifying epileptogenic zone (EZ) using high-frequency oscillations (HFOs) is a challenge that must be mastered to transfer HFOs into clinical use. We analyzed the ability of a convolutional neural network (CNN) model to distinguish EZ and non-EZ HFOs. Nineteen medically intractable epilepsy patients with good surgical outcomes 2 years after surgery were studied. Five-minute interictal intracranial electroencephalogram epochs of slow-wave sleep were selected randomly. Then 5 s segments of ripples (80–200 Hz) and fast ripples (FRs, 200–500 Hz) were detected automatically. The EZs and non-EZs were identified using the surgery resection range. We innovatively converted all epochs into four types of images using two scales: original waveforms, filtered waveforms, wavelet spectrum images, and smoothed pseudo Wigner–Ville distribution (SPWVD) spectrum images. Two scales were fixed and fitted scales. We then used a CNN model to classify the HFOs into EZ and non-EZ categories. As a result, 7,000 epochs of ripples and 2,000 epochs of FRs were randomly selected from the EZ and non-EZ data for analysis. Our CNN model can distinguish EZ and non-EZ HFOs successfully. Except for original ripple waveforms, the results from CNN models that are trained using fixed-scale images are significantly better than those from models trained using fitted-scale images (p < 0.05). Of the four fixed-scale transformations, the CNN based on the adjusted SPWVD (ASPWVD) produced the best accuracies (80.89 ± 1.43% and 77.85 ± 1.61% for ripples and FRs, respectively, p < 0.05). The CNN using ASPWVD transformation images is an effective deep learning method that can be used to classify EZ and non-EZ HFOs.

Multimodal emotion recognition has become a hot topic in human-computer interaction and intelligent healthcare fields. However, combining information from different human different modalities for emotion computation is still challenging. In this paper, we propose a three-dimensional convolutional recurrent neural network model (referred to as 3FACRNN network) based on multimodal fusion and attention mechanism. The 3FACRNN network model consists of a visual network and an EEG network. The visual network is composed of a cascaded convolutional neural network-time convolutional network (CNN-TCN). In the EEG network, the 3D feature building module was added to integrate band information, spatial information and temporal information of the EEG signal, and the band attention and self-attention modules were added to the convolutional recurrent neural network (CRNN). The former explores the effect of different frequency bands on network recognition performance, while the latter is to obtain the intrinsic similarity of different EEG samples. To investigate the effect of different frequency bands on the experiment, we obtained the average attention mask for all subjects in different frequency bands. The distribution of the attention masks across the different frequency bands suggests that signals more relevant to human emotions may be active in the high frequency bands γ (31-50 Hz). Finally, we try to use the multi-task loss function Lc to force the approximation of the intermediate feature vectors of the visual and EEG modalities, with the aim of using the knowledge of the visual modalities to improve the performance of the EEG network model. The mean recognition accuracy and standard deviation of the proposed method on the two multimodal sentiment datasets DEAP and MAHNOB-HCI (arousal, valence) were 96.75 ± 1.75, 96.86 ± 1.33; 97.55 ± 1.51, 98.37 ± 1.07, better than those of the state-of-the-art multimodal recognition approaches. The experimental results show that starting from the multimodal information, the facial video frames and electroencephalogram (EEG) signals of the subjects are used as inputs to the emotion recognition network, which can enhance the stability of the emotion network and improve the recognition accuracy of the emotion network. In addition, in future work, we will try to utilize sparse matrix methods and deep convolutional networks to improve the performance of multimodal emotion networks.

The presented paper proposes a new speech command recognition model for novel engineering applications with limited resources. We built the proposed model with the help of a Convolutional Recurrent Neural Network (CRNN). The use of CRNN instead of Convolutional Neural Network (CNN) helps us to reduce the model parameters and memory requirement as per resource constraints. Furthermore, we insert transmute and curtailment layer between the layers of CRNN. By doing this we further reduce model parameters and float number of operations to half of the CRNN requirement. The proposed model is tested on Google’s speech command dataset. The obtained result shows that the proposed CRNN model requires 1/3 parameters as compared to the CNN model. The number of parameters of the CRNN model is further reduced by 45% and the float numbers of operations between 2% to 12 % in different recognition tasks. The recognition accuracy of the proposed model is 96% on Google’s speech command dataset, and on laboratory recording, its recognition accuracy is 89%.

Hand gestures are a form of non-verbal communication. Apart from traditional input devices, hand gestures are used for interaction with computers too. Communication via hand gestures finds many applications in the real world. Different people have hands with different shapes and orientations, which is termed as nonlinearity. The nonlinearity affects the performance of hand gesture models. A convolutional neural network (CNN) is an approach of neural networks, specifically known as deep learning. CNN is used to recognize and classify images. Sometimes, CNN could not correctly understand the hand gesture due to nonlinearity. Data augmentation helps CNN to understand the nonlinearity and complexity of images better. Data augmentation generates enormous data from lesser data, thus increasing the data adversity. Data augmentation uses various operations like zooming, rotating, shifting, shearing, and scaling to generate more data from the existing data. This article executes a CNN model using augmented data for recognition of static hand gestures. The dataset consists of 10 different hand gestures. The experimented CNN model has been trained using 10000 images and tested using 1000 images. The changes in the output of CNN with and without data augmentation have been highlighted. The CNN model employing data augmentation achieved an accuracy of 98.10%, whereas the CNN model excluding the data augmentation process attained an accuracy of 94.90% only.

Understanding the learning mechanism of convolutional neural networks in spectral analysis

In our life, emotions often have a profound impact on human behavior, especially for drivers, as negative emotions can increase the risk of traffic accidents. As such, it is imperative to accurately discern the emotional states of drivers in order to preemptively address and mitigate any negative emotions that may otherwise manifest and compromise driving behavior. In contrast to many current studies that rely on complex and deep neural network models to achieve high accuracy, this research aims to explore the potential of achieving high recognition accuracy using shallow neural networks through restructuring the structure and dimensions of the data. In this study, we propose an end-to-end convolutional neural network (CNN) model called simply ameliorated CNN (SACNN) to address the issue of low accuracy in cross-subject emotion recognition. We extracted features and converted dimensions of EEG signals from the SEED dataset from the BCMI Laboratory to construct 62-dimensional data, and obtained the optimal model configuration through ablation experiments. To further improve recognition accuracy, we selected the top 10 channels with the highest accuracy by separately training the EEG data of each of the 62 channels. The results showed that the SACNN model achieved an accuracy of 88.16% based on raw cross-subject data, and an accuracy of 91.85% based on EEG channel data from the top 10 channels. In addition, we explored the impact of the position of the BN and dropout layers on the model through experiments, and found that a targeted shallow CNN model performed better than deeper and larger perceptual field CNN models. Furthermore, we discuss herein the future issues and challenges of driver emotion recognition in promising smart city applications.

Purpose: Parkinson’s disease (PD) diagnosis algorithms based on Quantitative Susceptibility Mapping and image algorithms rely on SN labeling. However, the difference between SN labels from different experts (or segmentation algorithms) will have a negative impact on downstream diagnostic tasks, such as the decrease of the algorithm's accuracy or different diagnostic results for the same sample. In this article, we quantify the accuracy of the algorithm on different label sets, and then improve the Convolutional Neural Network Model to obtain a high-precision and high-robust diagnosis algorithm. Methods: The logistic regression model and convolutional neural network (CNN) model were first compared for classification between PD patients and healthy controls (HC), given different sets of substantia nigra (SN) labeling. Then, based on the CNN model with better performance, we further proposed a novel "Gated Pooling" operation and integrated it with deep learning to attain a joint framework for image segmentation and classification. Results: The experimental results show that, with different sets of SN labeling that mimic different experts, the CNN model can maintain the stable classification accuracy around 86.4%, while the conventional logistic regression model yields a large fluctuation ranging from 78.9% to 67.9%. Furthermore, the "Gated Pooling" operation after being integrated for joint image segmentation and classification can improve the diagnosis accuracy to 86.9% consistently, which is statistically better than the baseline. Conclusion: The CNN model, compared with the conventional logistic regression model using radiomics features, has better stability in PD diagnosis. Furthermore, the joint end-to-end CNN model is shown to be suitable for PD diagnosis from the perspectives of accuracy, stability, and convenience in actual use.

BackgroundSurgical process modeling automatically identifies surgical phases, and further improvement in recognition accuracy is expected with deep learning. Surgical tool or time series information has been used to improve the recognition accuracy of a model. However, it is difficult to collect this information continuously intraoperatively. The present study aimed to develop a deep convolution neural network (CNN) model that correctly identifies the surgical phase during laparoscopic cholecystectomy (LC).MethodsWe divided LC into six surgical phases (P1–P6) and one redundant phase (P0). We prepared 115 LC videos and converted them to image frames at 3 fps. Three experienced doctors labeled the surgical phases in all image frames. Our deep CNN model was trained with 106 of the 115 annotation datasets and was evaluated with the remaining datasets. By depending on both the prediction probability and frequency for a certain period, we aimed for highly accurate surgical phase recognition in the operation room.ResultsNine full LC videos were converted into image frames and were fed to our deep CNN model. The average accuracy, precision, and recall were 0.970, 0.855, and 0.863, respectively.ConclusionThe deep learning CNN model in this study successfully identified both the six surgical phases and the redundant phase, P0, which may increase the versatility of the surgical process recognition model for clinical use. We believe that this model can be used in artificial intelligence for medical devices. The degree of recognition accuracy is expected to improve with developments in advanced deep learning algorithms.

Hand gesture recognition (HGR) has gained significant attention due to its potential for various applications. This paper explores the use of deep learning, specifically Convolutional Neural Networks (CNNs), for HGR using the TensorFlow library. We investigate existing research on CNN-based HGR, focusing on image classification tasks. We then provide a brief overview of CNNs and their suitability for image recognition. Subsequently, we describe the typical workflow of a deep learning-based HGR system, including data preprocessing, hand detection, feature extraction with CNNs, and classification. We highlight the advantages of using TensorFlow to build and train CNN models for HGR. Finally, we conclude by summarizing the key findings from related work and mentioning the specific dataset and number of gestures classified in our research. This work contributes to the growing body of research on CNN-based HGR using TensorFlow and emphasizes its potential for developing accurate and efficient HGR systems.