IHand: Hand Recognition-Based Text Input Method for Wearable Devices

The widespread creative application and smart devices call for convenient and secure interaction with human users. We propose, design, and implement a smartwatch-based two-factor real-time identification and authentication system named ThumbUp, where smartwatch users can identify and authenticate themselves by some simple hand and finger gestures, such as thumb-up. ThumbUp leverages the signal collected from the Inertial Measurement Unit (IMU) in Commercial Off-The-Shelf (COTS) smart devices and discovers the unique fingerprint pattern produced by each user’s simple hand gestures using a carefully crafted deep learning model. We implement our system and conduct extensive experiments to evaluate its efficacy and efficiency with 65 different users over a period of more than 3 months. It reaches an accuracy of 97% for identification, and EER 0.014 for authentication using only one simple gesture. We also survey the users’ acceptance of our system and discuss how the proficiency of gestures affects authentication accuracy.

In order to solve the problem of poor real-time performance and low accuracy of a single detection target of common driver fatigue driving detection method based on facial features in practical applications, a fatigue driving detection method based on lightweight convolutional neural network is proposed. First, the driver's facial feature point data set is made through MTCNN (multi task convolutional neural network). Then the data set is used to train a lightweight convolutional neural network to detect the accurate feature point position of the eyes and mouth. Finally, the open and close state of the driver's eyes and mouth is judged based on the feature point coordinates. According to the open and closed state of the eyes and mouth of the continuous multi-frame image, the driver is judged to be in the state of fatigue. The experimental results show that the processing time of the single frame image by the algorithm is 23.3 millisecond; the single detection accuracy is up to 99.4%, and the detection accuracy of fatigue driving can reach 95%. The algorithm is better real-time performance and higher accuracy, so it has certain engineering significance and application prospects.

A large number of people with disabilities rely on assistive technologies to communicate with their families, to use social media, and have a social life. Despite a significant increase of novel assitive technologies, robust, non-invasive, and inexpensive solutions should be proposed and optimized in relation to the physical abilities of the users. A reliable and robust identification of intentional visual commands is an important issue in the development of eye-movements based user interfaces. The detection of a command with an eyetracking system can be achieved with a dwell time. Yet, a large number of people can use simple hand gestures as a switch to select a command. We propose a new virtual keyboard based on the detection of ten commands. The keyboard includes all the letters of the Latin script (upper and lower case), punctuation marks, digits, and a delete button. To select a command in the keyboard, the user points the desired item with the gaze, and select it with hand gesture. The system has been evaluated across eight healthy subjects with five predefined hand gestures, and a button for the selection. The results support the conclusion that the performance of a subject, in terms of speed and information transfer rate (ITR), depends on the choice of the hand gesture. The best gesture for each subject provides a mean performance of $8 . 77 \pm 2 .90$ letters per minute, which corresponds to an ITR of $57 . 04 \pm 14 .55$ bits per minute. The results highlight that the hand gesture assigned for the selection of an item is inter-subject dependent.

The development of robots has enabled humans to allow them to carry out simple tasks and activities for easier and faster living. Robotic vehicles are found in our houses and industries and would continue to be integrated into more aspects of human living. In the present age of technology, most robotic systems found in industries and our everyday lives are controlled by a traditional input method or are pre-programmed. This makes them a little tedious for operation by a common man without technical know-how or a certain level of expertise. Due to this, there is a need for a natural mode of communication between humans and robots. This study develops a robotic vehicle system that can be controlled by a user’s simple hand gestures. The system employs the use of an accelerometer to detect hand gestures and then processed by Arduino boards to relay instructions to the robotic vehicle. The development of a hand gesture control system for a robotic vehicle provides a more natural mode of communication for human-robot interaction. This technology would increase the applications of robotic vehicles in various sectors through its implementation.

This study addresses the challenges of low accuracy and slow speed in automatic strawberry classification. We propose an integrated approach that combines image processing, computer vision, and deep learning to classify ripe strawberries comprehensively. A lightweight convolutional neural network (CNN) model is developed to achieve 99.16% accuracy in ripeness level identification. Additionally, a multiple linear regression model, incorporating area, perimeter, length, and width, predicts strawberry weights with an R² of 0.924 and an average prediction error of 2.304%. By integrating ripeness recognition and weight prediction, our method provides a standardized classification system for ripe strawberries. The CNN model ensures high recognition accuracy and real - time grading, while the regression model enhances weight specification accuracy. This approach contributes a scientific and efficient non - destructive classification system, benefiting precision agriculture and strawberry quality control.

Lightweight convolutional neural network model for field wheat ear disease identification

Aim/Purpose: When learning to program, both text-based and visual-based input methods are common. However, it is unclear which method is more appropriate for first-time learners (first learners). Background: The differences in the learning effect between text-based and visual-based input methods for first learners are compared the using a questionnaire and problems to assess first learners’ understanding of programming. In addition, we study the benefits and feasibility of both methods. Methodology: In this research, we used the sandbox game Minecraft and the extended function ComputerCraftEdu (CCEdu). CCEdu provides a Lua programming environments for the two (text and visual) methods inside Minecraft. We conducted a lecture course on both methods for first learners in Japan ranging in age from 6 to about 15 years old. The lecture taught the basics and concepts of programming. Furthermore, we implemented a questionnaire about the attitude of programming before and after the lecture. Contribution: This research is more than a comparison between the visual method and the text method. It compares visual input and text input methods in the same environment. It clearly shows the difference between the programming learning effects of visual input and text input for first learners. In addition, it shows the more suitable input method for introductory education of first learners in programming learning. Findings: The following results are revealed: (1) The visual input method induces a larger change in attitude toward programming; (2) The number of operations and input quantity influence both groups; (3) The overall results suggest that a visual input is advantageous in a programming implementation environment for first learners. Impact on Society: A visual input method is better suited for first learners as it improves the attitude toward programming. Future Research: In the future, we plan to collect and analyze additional data as well as elucidate the correlation between attitudes and understanding of programming.

Earthquake P-wave detection is the first and most critical step in Earthquake Early Warning (EEW), as detection speed determines how much time is available before strong shaking arrives, while detection accuracy governs the reliability of the warning. Conventional P-Wave detection methods such as Short-Term Average over Long-Term Average (STA/LTA) often struggle in complex noise environments and exhibit higher detection latency. Deep learning offers a more robust alternative by learning complex waveform characteristics directly from data, enabling faster and more reliable detections. However, the high computational cost of deep models limits their use on low-cost, resource-constrained edge devices in decentralized EEW networks. Here we present a lightweight Convolutional Neural Network optimized for real-time P- and S-wave detection on edge hardware using only two seconds of seismic input waveforms. Trained on approximately 89K waveform segments recorded by strong-motion sensors across New Zealand, the model achieves an overall accuracy of 97.12% and correctly identifies 98% of P-wave segments. With ~ 38K trainable parameters and sub-7-millisecond inference time on Raspberry Pi 5 hardware, the model demonstrates efficient operation under edge-computing constraints. Its ability to generalize to high-magnitude events highlights its strong potential for integration into distributed EEW systems.

The classification method of steel surface defects with high performance and easy to be embedded in the detection equipment is one of the keys to ensure the quality of hot rolled strip. However, the development of deep convolutional neural networks (CNNs) in many real-world applications is largely hindered by their high computational cost, especially in industrial production, although it has good classification accuracy compared with machine learning-based methods in image recognition. Therefore, in this work, we present a lightweight network FCCNet based on the convolutional neural network to facilitate its application in the detection system. To compensate for the accuracy loss caused by the network downsizing, a knowledge distillation (KD) method using a larger trained network (teacher network) to teach a smaller network (student network) is adopted to improve the performance of our model. As a result, our method achieves a classification accuracy of 99.44%, precision of 99.46%, recall of 99.45%, and an F1 score of 99.45% on the NEU-CLS dataset, using only 0.03 MB parameters. These results show that the FCCNet is lighter than other existing classic CNNs with good performance for surface defects classification of hot-rolled steel strip, and it has the potential to be applied in the actual production line.

Deep convolutional neural networks (CNNs) have seen significant growth in medical image classification applications due to their ability to automate feature extraction, leverage hierarchical learning, and deliver high classification accuracy. However, Deep CNNs require substantial computational power and memory, particularly for large datasets and complex architectures. Additionally, optimising the hyperparameters of deep CNNs, although critical for enhancing model performance, is challenging due to the high computational costs involved, making it difficult without access to high-performance computing resources. To address these limitations, this study presents a fast and efficient model that aims to achieve superior classification performance compared to popular Deep CNNs by developing lightweight CNNs combined with the Nonlinear Lévy chaotic moth flame optimiser (NLCMFO) for automatic hyperparameter optimisation. NLCMFO integrates the Lévy flight, chaotic parameters, and nonlinear control mechanisms to enhance the exploration capabilities of the Moth Flame Optimiser during the search phase while also leveraging the Lévy flight theorem to improve the exploitation phase. To assess the efficiency of the proposed model, empirical analyses were performed using a dataset of 2314 brain tumour detection images (1245 images of brain tumours and 1069 normal brain images). The evaluation results indicate that the CNN_NLCMFO outperformed a non-optimised CNN by 5% (92.40% accuracy) and surpassed established models such as DarkNet19 (96.41%), EfficientNetB0 (96.32%), Xception (96.41%), ResNet101 (92.15%), and InceptionResNetV2 (95.63%) by margins ranging from 1 to 5.25%. The findings demonstrate that the lightweight CNN combined with NLCMFO provides a computationally efficient yet highly accurate solution for medical image classification, addressing the challenges associated with traditional deep CNNs.

Integrated hand-tracking on modern virtual reality (VR) headsets can be readily exploited to deliver mid-air virtual input surfaces for text entry. These virtual input surfaces can closely replicate the experience of typing on a Qwerty keyboard on a physical touchscreen, thereby allowing users to leverage their pre-existing typing skills. However, the lack of passive haptic feedback, unconstrained user motion, and potential tracking inaccuracies or observability issues encountered in this interaction setting typically degrades the accuracy of user articulations. We present a comprehensive exploration of error-tolerant probabilistic hand-based input methods to support effective text input on a mid-air virtual Qwerty keyboard. Over three user studies we examine the performance potential of hand-based text input under both gesture and touch typing paradigms. We demonstrate typical entry rates in the range of 20 to 30 wpm and average peak entry rates of 40 to 45 wpm.

In the present timeline, technology has advanced significantly, and the healthcare sector is constantly expanding. People in rural areas who cannot access corporate hospitals postpone their health check-ups due to the difficulty of traveling long distances. The readily available and expensive nature of diagnostic technologies have prevented many rural residents from seeking clinical examinations, even in cases when they have health issues. Corporate hospitals often make people to wait for long periods of time for health reports and causing delays. These issues highlighted the need for a device which aimed at developing an Arduino based health diagnosis system To resolve this issue, we have introduced a device that offers a comprehensive health diagnosis and monitoring system that uses Arduino technology to store and display vital health information. The system is designed to analyze electrocardiograms (ECG), photoplethysmograms (PPG), body temperature, heart rate and stress levels in real time and then process this information to create meaningful health status assessments and store them in the database for further health assessments. Furthermore, these data could be displayed through web server or an application. Additionally, this system provides voice assistance for the condition of patient when it is critical which helps in early detection of health conditions. Hand gestures are incorporated into this device, which makes it distinctive. Users can control the display of health parameters and easily move through the system's features with simple hand gestures. Users can switch between different types of modes in the system (such as on/off and different operation) by moving their hand in specified patterns. Thus, this makes the system user-friendly so that everyone may use it.

In this study, we aimed to integrate multi-touch gesture with hand gesture and make new input method. Multi-touch gesture and hand gesture have been studied separately, however, these could be integrated, and it would be new interaction way for computer. We made a large scale multi-touch screen by FTIR method that uses infrared ray to detect the touched point, and a hand gesture recognition device with sensors, pressure sensor and bend sensor. In the end, we integrated the input information to interact with computer. User can input coordinate by touching, and simultaneously input action by performing hand gesture. For instance, user can select target by touching, and then make an action on target by clicking the pressure sensor or bending the bend sensor. We demonstrated the input method on GUI application.

The use of hand gestures provides a natural alternative to cumbersome interface devices for Human-Computer Interaction (HCI) systems. However, real-time recognition of dynamic micro hand gestures from video streams is challenging for in-vehicle scenarios since (i) the gestures should be performed naturally without distracting the driver, (ii) micro hand gestures occur within very short time intervals at spatially constrained areas, (iii) the performed gesture should be recognized only once, and (iv) the entire architecture should be designed lightweight as it will be deployed to an embedded system. In this work, we propose an HCI system for dynamic recognition of driver micro hand gestures, which can have a crucial impact in automotive sector especially for safety related issues. For this purpose, we initially collected a dataset named Driver Micro Hand Gestures (DriverMHG), which consists of RGB, depth and infrared modalities. The challenges for dynamic recognition of micro hand gestures have been addressed by proposing a lightweight convolutional neural network (CNN) based architecture which operates online efficiently with a sliding window approach. For the CNN model, several 3-dimensional resource efficient networks are applied and their performances are analyzed. Online recognition of gestures has been performed with 3D-MobileNetV2, which provided the best offline accuracy among the applied networks with similar computational complexities. The final architecture is deployed on a driver simulator operating in real-time. We make DriverMHG dataset and our source code publicly available 1.1https://www.mmk.ei.tum.de/DriverMHG/

The virtual mouse is an important research topic in HCI (Human Computer Interaction) technology, and the most popular method is based on hand gestures. In this paper, a six-degree-of-freedom virtual mouse based on hand gestures is presented using an inexpensive USB webcam. The hand tracking and gesture recognition framework includes the motion, skin color and finger information of the hand. It can even provide computer access for the disabled people who cannot bend their fingers at all with severe disabilities. The experiments show that the system has good interactive input functions in real time.