Hand gesture recognition using machine learning-based 4-channel receiving ultrasonic array

Hand gestures are a form of non-verbal communication. Apart from traditional input devices, hand gestures are used for interaction with computers too. Communication via hand gestures finds many applications in the real world. Different people have hands with different shapes and orientations, which is termed as nonlinearity. The nonlinearity affects the performance of hand gesture models. A convolutional neural network (CNN) is an approach of neural networks, specifically known as deep learning. CNN is used to recognize and classify images. Sometimes, CNN could not correctly understand the hand gesture due to nonlinearity. Data augmentation helps CNN to understand the nonlinearity and complexity of images better. Data augmentation generates enormous data from lesser data, thus increasing the data adversity. Data augmentation uses various operations like zooming, rotating, shifting, shearing, and scaling to generate more data from the existing data. This article executes a CNN model using augmented data for recognition of static hand gestures. The dataset consists of 10 different hand gestures. The experimented CNN model has been trained using 10000 images and tested using 1000 images. The changes in the output of CNN with and without data augmentation have been highlighted. The CNN model employing data augmentation achieved an accuracy of 98.10%, whereas the CNN model excluding the data augmentation process attained an accuracy of 94.90% only.

Predicting and accurately classifying intentions for human hand gestures can be used not only for active prosthetic hands, rehabilitation robots and entertainment robots but also for artificial intelligence robots in general. In this paper, first of all, source data of three hand gestures of grasping and three hand gestures of sign language are acquired by using the armband combined with eight sEMG (surface Electromyography) sensors. To classify these hand gestures, basically simple CNN (convolutional neural network) models with raw data, short-time Fourier transform (STFT), wavelet transform (WT), and scale average wavelet transform (SAWT) are applied, and their performances are compared. Finally, it is shown that by using a CNN with SAWT images, the accuracy can be improved up to 94.6% for selected hand gestures with higher accuracy and lower computational burden than conventional multi-channel STFT or WT.

Understanding the learning mechanism of convolutional neural networks in spectral analysis

The purpose of this paper was to investigate the effect of a training state-of-the-art convolution neural network (CNN) for millimeter-wave radar-based hand gesture recognition (MR-HGR). Focusing on the small training dataset problem in MR-HGR, this paper first proposed to transfer the knowledge with the CNN models in computer vision to MR-HGR by fine-tuning the models with radar data samples. Meanwhile, for the different data modality in MR-HGR, a parameterized representation of temporal space-velocity (TSV) spectrogram was proposed as an integrated data modality of the time-evolving hand gesture features in the radar echo signals. The TSV spectrograms representing six common gestures in human–computer interaction (HCI) from nine volunteers were used as the data samples in the experiment. The evaluated models included ResNet with 50, 101, and 152 layers, DenseNet with 121, 161 and 169 layers, as well as light-weight MobileNet V2 and ShuffleNet V2, mostly proposed by many latest publications. In the experiment, not only self-testing (ST), but also more persuasive cross-testing (CT), were implemented to evaluate whether the fine-tuned models generalize to the radar data samples. The CT results show that the best fine-tuned models can reach to an average accuracy higher than 93% with a comparable ST average accuracy almost 100%. Moreover, in order to alleviate the problem caused by private gesture habits, an auxiliary test was performed by augmenting four shots of the gestures with the heaviest misclassifications into the training set. This enriching test is similar with the scenario that a tablet reacts to a new user. The results of two different volunteer in the enriching test shows that the average accuracy of the enriched gesture can be improved from 55.59% and 65.58% to 90.66% and 95.95% respectively. Compared with some baseline work in MR-HGR, the investigation by this paper can be beneficial in promoting MR-HGR in future industry applications and consumer electronic design.

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

Accurately identifying epileptogenic zone (EZ) using high-frequency oscillations (HFOs) is a challenge that must be mastered to transfer HFOs into clinical use. We analyzed the ability of a convolutional neural network (CNN) model to distinguish EZ and non-EZ HFOs. Nineteen medically intractable epilepsy patients with good surgical outcomes 2 years after surgery were studied. Five-minute interictal intracranial electroencephalogram epochs of slow-wave sleep were selected randomly. Then 5 s segments of ripples (80–200 Hz) and fast ripples (FRs, 200–500 Hz) were detected automatically. The EZs and non-EZs were identified using the surgery resection range. We innovatively converted all epochs into four types of images using two scales: original waveforms, filtered waveforms, wavelet spectrum images, and smoothed pseudo Wigner–Ville distribution (SPWVD) spectrum images. Two scales were fixed and fitted scales. We then used a CNN model to classify the HFOs into EZ and non-EZ categories. As a result, 7,000 epochs of ripples and 2,000 epochs of FRs were randomly selected from the EZ and non-EZ data for analysis. Our CNN model can distinguish EZ and non-EZ HFOs successfully. Except for original ripple waveforms, the results from CNN models that are trained using fixed-scale images are significantly better than those from models trained using fitted-scale images (p < 0.05). Of the four fixed-scale transformations, the CNN based on the adjusted SPWVD (ASPWVD) produced the best accuracies (80.89 ± 1.43% and 77.85 ± 1.61% for ripples and FRs, respectively, p < 0.05). The CNN using ASPWVD transformation images is an effective deep learning method that can be used to classify EZ and non-EZ HFOs.

Distinguishing pericarpium citri reticulatae of different origins using terahertz time-domain spectroscopy combined with convolutional neural networks

Convolutional Neural Networks (CNNs) have demonstrated impressive performance in complex machine learning tasks such as classification and regression problems. A reliable neural network structure plays a decisive role in CNN studies. Through comparing and analyzing the structure of neural networks, a model structure for better visualization performance has been discovered, and such a method supports the development of deep learning research. These studies are of particular importance in end-to-end systems for autonomous driving to imitate human driving, where the interpretability of the system is limited. Because of the uncertainty of the ground truth, for the determination of human steering in an image, it is difficult to accurately compare the visualization performance of different CNN models or different visualization methods. For practical applications, however, an objective and quantitative measure for assessing visualization performance is necessary. Therefore, a method to evaluate the visualization performance of CNN models using a driver model instead of human drivers is proposed, to generate a data set which can be used to determine the decisional point (ground truth) in the input image. Then, an exclusive method is also put forth, to quantitatively calculate the relationship between the decisional point (ground truth) and the visualization results produced by CNN models. In this paper, five CNN models as an autonomous steering controller are designed based on PilotNet, and the visualization abilities of each CNN models is compared by three evaluation indicators. By comparing the visualization performance of five different CNN models, it is shown that the proposed method can successfully assess the visualization level of the CNN model.

BACKGROUND AND AIM: Outdoor ultrafine particles (UFP; 100 nm) and black carbon (BC) vary greatly within cities and may have adverse impacts on human health such as cardiovascular mortality and brain tumour incidence. Traditional regression methods for estimating spatial distributions of ambient air pollution depend on extensive, curated geospatial information system databases. This has resulted in a disparity wherein little is known about local air pollution in the data-sparse settings of low- and middle- income countries. We used a hybrid approach to develop new models to estimate within-city spatial variations in outdoor UFP and BC concentrations across Bucaramanga, Colombia. METHODS: We conducted a large-scale mobile monitoring campaign over twenty days in 2019. Land use regression (LUR) models were developed using land use parameters from curated and open-source databases. Convolutional Neural Network (CNN) models were developed using Google Maps satellite view and street view images. Combined models were developed by combining predictions from the LUR and CNN models. After training in the training and validation sets, predictions were generated in the test set and compared to measured values. RESULTS:The combined UFP model (R2=0.54) outperformed the CNN (R2=0.47) and land use regression (LUR) models (R2=0.47) on their own. Similarly, the combined BC model also outperformed the CNN and LUR BC models (R2 = 0.51 vs 0.43 and 0.45 respectively). Spatial variations in model performance were more stable for the CNN and combined models compared to the LUR models suggesting that the combined approach may be less likely to contribute to differential exposure measurement error in epidemiological studies. CONCLUSIONS:Estimates from these models can then be applied to population-based cohorts in order to evaluate population health risks. Additionally, our findings demonstrated that satellite and street-level images can be combined with a traditional LUR modelling approach to improve predictions of within-city spatial variations in outdoor UFP and BC concentrations. KEYWORDS: Ultrafine particles, Black Carbon, Deep learning, Images, Land use regression

PurposeSemantic segmentation is a fundamental part of the surgical application of deep learning. Traditionally, segmentation in vision tasks has been performed using convolutional neural networks (CNNs), but the transformer architecture has recently been introduced and widely investigated. We aimed to investigate the performance of deep learning models in segmentation in robot-assisted radical prostatectomy (RARP) and identify which of the architectures is superior for segmentation in robotic surgery.Materials and MethodsIntraoperative images during RARP were obtained. The dataset was randomly split into training and validation data. Segmentation of the surgical instruments, bladder, prostate, vas and seminal vesicle was performed using three CNN models (DeepLabv3, MANet, and U-Net++) and three transformers (SegFormer, BEiT, and DPT), and their performances were analyzed.ResultsThe overall segmentation performance during RARP varied across different model architectures. For the CNN models, DeepLabV3 achieved a mean Dice score of 0.938, MANet scored 0.944, and U-Net++ reached 0.930. For the transformer architectures, SegFormer attained a mean Dice score of 0.919, BEiT scored 0.916, and DPT achieved 0.940. The performance of CNN models was superior to that of transformer models in segmenting the prostate, vas, and seminal vesicle.ConclusionsDeep learning models provided accurate segmentation of the surgical instruments and anatomical structures observed during RARP. Both CNN and transformer models showed reliable predictions in the segmentation task; however, CNN models may be more suitable than transformer models for organ segmentation and may be more applicable in unusual cases. Further research with large datasets is needed.

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.

Empirical-statistical Downscaling (SD) techniques are valuable tools able to generate high-resolution climate information needed to carry out impact studies. In this regard, Convolutional Neural Networks (CNNs) are promising SD techniques capable of handling large amounts of data and extracting relevant predictor information for each particular site. These characteristics of the CNN represent a major advantage over traditional SD methods, which typically rely on human-guided predictor selection. Notwithstanding, an adequate tuning of the CNN is key for optimising their potential.In southern South America (SSA), CNNs has proven to be skilful in representing daily extreme temperatures and extrapolating into future scenarios. Although the selection of the activation function introduces a source of uncertainty in the future projections.&amp;#160;In this context, this study aims to explore the use of CNNs as a statistical downscaling tool to simulate the wet bulb temperature (Tw) over SSA, a multivariate heat-stress index estimated from temperature and humidity. Tw has been widely used as a heat-stress proxy in different parts of the world, however, its characterisation and modelling in SSA remain as a pending task. To this end, four different CNN architectures regarding the activation function (ReLU or linear), domain size and configuration of the CNN layers were tested. All CNN models were trained during summer days using a cross-validation (CV) scheme in the period 1991-2020 and then evaluated in four unseen summers between 2021 and 2024. For comparison purposes, CNN models were similarly trained and validated to simulate maximum temperature (Tx).&amp;#160;Overall, CNN models well represented all the features evaluated, including the heat-waves that took place in the summers evaluated independently. In particular, CNN models presents a better performance in simulating Tw than Tx with smaller errors in terms of mean and extremes aspects. Regarding the domain size, for both temperatures, the configuration with the smaller domain yields the best results. Also in this latter case, the reduction of the number of filter size in the last layer slightly improves the representation of Tx. When considering the large domain, the differences between the CNNs based on different activation functions increase, and CNN models with linear configuration outperform the ones with ReLu.&amp;#160;The findings of this work reinforces the potential of CNNs for climate downscaling in SSA, especially for its use to simulate multivariate impact indices.

This study leveraged Convolutional Neural Network (CNN) models to estimate canopy height in Southeast Australian forests before and after the 2019–2020 bushfire event, using inputs from Sentinel-1, Sentinel-2, spectral indices, and terrain features and GEDI canopy height (rh99) as target variables. Our research consisted of three primary objectives: (1) an evaluation of GEDI rh99 canopy height in relation to an nDSM derived from airborne LiDAR, (2) the development of two CNN models for pre- and post-fire scenarios, and (3) utilizing the trained CNN models to generate pre- and post-fire canopy height maps and canopy height change maps across the continuous landscape. GEDI rh99 accuracy analysis revealed R 2 = 0.85, RMSE = 7.25 m and nRMSE = 0.23 for pre-fire GEDI and R 2 = 0.78, RMSE = 10.1 m and nRMSE = 0.3 for post-fire GEDI. The pre-fire CNN model achieved R 2 = 0.75, RMSE = 7.29 m and nRMSE = 0.22, and post-fire CNN model achieved R 2 = 0.62, RMSE = 11.67 m, and nRMSE = 0.34 when validated against nDSM. Bias analysis was conducted, revealing that GEDI rh99 underestimated nDSM across all canopy height ranges, and slope has no significant impact on GEDI accuracy. Notably, post-fire GEDI rh99 exhibited significant underestimation in high to extreme fire severity. The CNN models’ prediction displayed overestimation in short forests < 20 m and increasing underestimation in taller forests > 20 m and underestimation in high and extreme fire severity. Subsequent investigation indicated that canopy layer thinning in high fire severity conditions resulted in weak GEDI waveform signals and consequent bias. Additionally, the GEDI rh99 bias was propagated to the CNN models, together with insensitivity of spectral band values to canopy height in dense and burnt forests potentially contributed to models’ bias. Finally, we demonstrated CNN models’ ability to monitor forest recovery by generating canopy height maps spanning East Gippsland from 2019 to 2023.

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

The main goal of this research was to accurately diagnose diseases in pepper plants using convolutional neural network (CNN) models and visual parameters features. Thus, we aimed to build a computing system that can utilise visual cues to identify plant diseases with almost 100% reliability. A total of 2,200 images were trained and tested on a dataset, which was divided into healthy leaves and those with bacterial ailments. The images were preprocessed, resized to 300x300 pixels, and then segmented to extract the relevant traits from them. This CNN model, in a word, is accurate because it detects diseases with 99.65% accuracy. If we want to speak of a high accuracy rate, that's the CNN model. It learns to identify the intricate patterns and features in pepper plants that are associated with diseases. CNN can make revolutionary changes to the identification of plant diseases and help prevent crop damage through timely treatments. Automatic identification of the disease makes life easier for both agriculturists and scientists. The results underscore the importance of image preprocessing, segmentation, and feature extraction in improving the ability of CNN models to locate diseases. Additionally, the performance of the CNN model was compared with other machine learning models such as KNN, Decision Tree, ANN, and Random Forest. It was found that the developed CNN model can predict the response with greater accuracy than the others. Modern high-tech precision farming systems might utilise proposed CNN models for plant disease diagnosis.