Fine-Grained and Real-Time Gesture Recognition by Using IMU Sensors

The use of unmanned aerial vehicles (UAV) systems has increased in recent years. Therefore,studies on UAVs have increased today. In this direction, the production of UAV systems with domestic resources has gained importance. In this study, it is desired to develop a domestic and national flight control card and software. In the flight control board designed for the UAV, it is aimed to keep the vehicle in balance in the air. Accurate measurement of platform orientation plays an important role in many applications such as aerospace, robotics, navigation, marine, machine interaction [1]. Inertial Measurement Unit (IMU) sensor was used to accurately measure the orientation of the UAV. IMU sensor is widely used in UAVs due to its light weight and low energy consumption. In this direction, the need for a filter has emerged in the IMU sensor, which is used to accurately measure the orientation of the unmanned aerial vehicle. In this study, a complementary filter was applied on the IMU sensor. Thanks to this filter, it has been observed that the accuracy of the data received from the IMU sensor has increased. Based on the data obtained, a Proportional Integral Derivative (PID) algorithm was developed, and the vehicle was kept in balance. In this study, ARMCortex-M4 based STM32F407VG microcontroller and MPU6050 as IMU sensor were used. Keil-uVision5 compiler is preferred for software. As a result, high accuracy in the orientation detection of unmanned aerial vehicles was obtained by applying a complementary filter on the IMU sensor.

Limb exercises are common in physical therapy to improve range of motion (RoM), strength, and flexibility of the arm/leg. To improve therapy outcomes and reduce cost, motion tracking systems have been used to monitor the user's movements when performing the exercises and provide guidance. Traditional motion tracking systems are based on either cameras or inertial measurement unit (IMU) sensors. Camera-based systems face problems caused by occlusion and lighting. Traditional IMU-based systems require at least two IMU sensors to track the motion of the entire limb, which is not convenient for use. In this paper, we propose a novel limb motion tracking system that uses a single 9-axis IMU sensor that is worn on the distal end joint of the limb (i.e., wrist for the arm or ankle for the leg). Limb motion tracking using a single IMU sensor is a challenging problem because 1) the noisy IMU data will cause drift problem when estimating position from the acceleration data, 2) the single IMU sensor measures the motion of only one joint but the limb motion consists of motion from multiple joints. To solve these problems, we propose a recurrent neural network (RNN) model to estimate the 3D positions of the distal end joint as well as the other joints of the limb (e.g., elbow or knee) from the noisy IMU data in real time. Our proposed approach achieves high accuracy with a median error of 7.2/7.1 cm for the wrist/elbow joint in leave-one-subject-out cross validation when tracking the arm motion, outperforming the state-of-the-art approach by more than 10%. In addition, the proposed model is lightweight, enabling real-time applications on mobile devices.Clinical relevance- This work has great potential to improve limb exercises monitoring and RoM measurement in home-based physical therapy. It is also cost effective and can be made available widely for immediate application.

Human activity recognition is a very active research on pervasive computing and mobile health application. Many human activity systems based on inertial measurement unit (IMU) sensor data were proposed in the past few years. These systems mainly use IMU sensor placed on he torso and limbs to collect data and utilize supervised machine learning algorithms on sensor data. One main issue of these systems is that wearing multiple on-body IMU sensors may bring inconvenience to users' daily life. The other issue of these exiting methods is that an activity recognition model that is trained on a specific subject does not work well when being applied to predict another subject's activities since IMU activity data always carry information that is specific to the human subject who conducts the activities. In our work, inspired by the principle of domain adaption, we proposed a new deep-learning activity recognition model based on an adversarial network which can remove the subject-specific information within the IMU activity data and extract subject-independent features shared by the data collected on different subjects. We also for the first time use data collected from insole based IMU sensors on 8 participants for 5 common activities to build a new real world human activity dataset which can minimize the inconvenience for users to wear. We conducted experiments with our new real-world dataset. Results show that our subject independent activity recognition model outperforms state-of-art supervised learning techniques and eliminates the effects of individual differences between subjects successfully. The average recognition accuracy under the leave-one-out (L1O) condition achieves 99.0% which is higher than the performance of traditional human activity recognition system based on CNNs.

Gait rehabilitation commonly relies on bodyweight unloading mechanisms, such as overhead mechanical support and underwater buoyancy. Lightweight and wireless inertial measurement unit (IMU) sensors provide a cost-effective tool for quantifying body segment motions without the need for video recordings or ground reaction force measures. Identifying the instant when the foot contacts and leaves the ground from IMU data can be challenging, often requiring scrupulous parameter selection and researcher supervision. We aimed to assess the use of machine learning methods for gait event detection based on features from foot segment rotational velocity using foot-worn IMU sensors during bodyweight-supported treadmill walking on land and underwater. Twelve healthy subjects completed on-land treadmill walking with overhead mechanical bodyweight support, and three subjects completed underwater treadmill walking. We placed IMU sensors on the foot and recorded motion capture and ground reaction force data on land and recorded IMU sensor data from wireless foot pressure insoles underwater. To detect gait events based on IMU data features, we used random forest machine learning classification. We achieved high gait event detection accuracy (95–96%) during on-land bodyweight-supported treadmill walking across a range of gait speeds and bodyweight support levels. Due to biomechanical changes during underwater treadmill walking compared to on land, accurate underwater gait event detection required specific underwater training data. Using single-axis IMU data and machine learning classification, we were able to effectively identify gait events during bodyweight-supported treadmill walking on land and underwater. Robust and automated gait event detection methods can enable advances in gait rehabilitation.

This study aims to generate visually useful imagery by preventing cropping while maintaining resolution and minimizing the degradation of stability and distortion to enhance the stability of a video for Augmented Reality applications. The focus is placed on conducting research that balances maintaining execution speed with performance improvements. By processing Inertial Measurement Unit (IMU) sensor data using the Versatile Quaternion-based Filter algorithm and optical flow, our research first applies motion compensation to frames of input video. To address cropping, PCA-flow-based video stabilization is then performed. Furthermore, to mitigate distortion occurring during the full-frame video creation process, neural rendering is applied, resulting in the output of stabilized frames. The anticipated effect of using an IMU sensor is the production of full-frame videos that maintain visual quality while increasing the stability of a video. Our technique contributes to correcting video shakes and has the advantage of generating visually useful imagery at low cost. Thus, we propose a novel hybrid full-frame video stabilization algorithm that produces full-frame videos after motion compensation with an IMU sensor. Evaluating our method against three metrics, the Stability score, Distortion value, and Cropping ratio, results indicated that stabilization was more effectively achieved with robustness to flow inaccuracy when effectively using an IMU sensor. In particular, among the evaluation outcomes, within the “Turn” category, our method exhibited an 18% enhancement in the Stability score and a 3% improvement in the Distortion value compared to the average results of previously proposed full-frame video stabilization-based methods, including PCA flow, neural rendering, and DIFRINT.

In this paper, we proposed an intuitive teleoperation system combining surface-electromyogram (sEMG) with inertia measurement unit (IMU) sensors. Two IMU sensors were worn in the upper arm and forearm to capture arm motion. The results were sent as control command to the remote robotic manipulator in task space. Pattern recognition algorithm was employed to decode sEMG signals collected from the forearm to control a humanoid mechanical hand. As a validation of the proposed system, we evaluated the performance of the system which included a Universal Robot 10 (UR10), a homemade humanoid pro-prosthetic hand (SJT-6) and two homemade armbands. Final experiments showed that the success rate on moving spherical object can be up to 86.7%.

Unmanned Aerial Vehicles (UAVs) play a crucial role in navigation, requiring accurate sensors to determine position, speed, and orientation, especially in unknown environments. Direct navigation systems like the Global Positioning System (GPS) provide positional data, while indirect systems, such as the Inertial Measurement Unit (IMU), integrate accelerometer and gyroscope data to supply speed and orientation information. This study investigates the integration of GPS and IMU sensors using the Unscented Kalman Filter (UKF) to improve localization accuracy on a cost-effective UAV Quadrotor equipped with LiDAR Livox. The research methodology involved collecting raw data from GPS, IMU, and LiDAR sensors during UAV flights. These data were processed using a UKF-based mathematical model to fuse sensor inputs and generate accurate point cloud models. Results show that the UKF fusion method achieved a localization accuracy of 0.403 m, with maximum residuals recorded as 1.332 m for the X axis, 20.421 m for the Y axis

ABSTRACT   In recent years, Taiwan has suffered from large-scale sediment-related disasters frequently caused by climate change induced extreme rainfall events. Fresh wood was usually recruited with large-scale sediment-related disasters and trapped in channels. Once a flash flood occurs, wood accumulated in the river consequently travels, threatening the safety of structures in the river. Furthermore, driftwood forms log jams leading to river blockage and backwater, causing overflooding. Therefore, realizing the initiation of motion mechanism of driftwood helps prevent driftwood disasters.According to previous studies on the initiation of wood motion, driftwood interacts with sediment and results in sliding, rolling, pivoting and floating motion modes dependent on discharge and riverbed conditions. To explore the motion mode of the initiation of wood motion on a mobile bed, this study conducts a flume experiment using driftwood equipped with an IMU (Inertial measurement unit) sensor that measures the posture of driftwood’s motion in the Eular Angle. According to the IMU’s data, the above four motion modes can be interpreted. Meanwhile, using the laser scanner to establish the digital elevation model and explore the correlation between sediment and driftwood motion on different mobile-bed conditions.

Most of the latest context-based applications capture the mobility of a user using Inertial Measurement Unit (IMU) sensors like accelerometer and gyroscope which do not need explicit user-permission for application access. Although these sensors provide highly accurate mobility context information, existing studies have shown that they can lead to undesirable leakage of location information. To evade this breach of location privacy, many of the state-of-the-art studies suggest to impose stringent restrictions over the usage of IMU sensors. However, in this paper, we show that typing and smartphone engagement patterns can act as an alternative modality to sniff the mobility context of a user, even if the IMU sensors are not sampled at all. We develop an adversarial framework, named ConType, which exploits the signatures exposed by typing and smartphone engagement patterns to track the mobility of a user. Rigorous experiments with in-the-wild dataset show that ConType can track the mobility contexts with an average micro-F 1 of 0.87 (±0.09), without using IMU data. Through additional experiments, we also show that ConType can track mobility stealthily with very low power and resource footprints, thus further aggravating the risk.

This paper presents a novel approach for an adaptive Extended Kalman Filter (EKF), which is able to handle bad signal quality caused by shading or loss of Doppler Effect for low cost Global Navigation Satellite System (GNSS) receiver and Inertial Measurement Unit (IMU) sensors fused in a loosely coupled way. It uses the estimated position error as well as the speed to calculate the standard deviation for the measurement uncertainty matrix of the Kalman Filter. The filter is very easy to implement, because some conversions of the measurement, as well as the state variables, are made to reduce the complexity of the Jacobians, which are used in the EKF filter algorithm. The filter implementation is tested within a simulation and with real data and shows significantly better performance, compared to a standard EKF. The developed filter is running in realtime on an embedded device and is able to perform position and attitude estimation of a vehicle with low cost sensors.

Hybrid of discrete wavelet transform and adaptive neuro fuzzy inference system for overall driving behavior recognition

Precise navigation is a vital need for many modern vehicular applications. The global positioning system (GPS) cannot provide continuous navigation information in urban areas. The widely used inertial navigation system (INS) can provide full vehicle state at high rates. However, the accuracy diverges quickly in low cost microelectromechanical systems (MEMS) based INS due to bias, drift, noise, and other errors. These errors can be corrected in a stationary state. But detecting stationary state is a challenging task. A novel stationary state detection technique from the variation of acceleration, heading, and pitch and roll of an attitude heading reference system (AHRS) built from the inertial measurement unit (IMU) sensors is proposed. Besides, the map matching (MM) algorithm detects the intersections where the vehicle is likely to stop. Combining these two results, the stationary state is detected with a smaller timing window of 3 s. A longer timing window of 5 s is used when the stationary state is detected only from the AHRS. The experimental results show that the stationary state is correctly identified and the position error is reduced to 90% and outperforms previously reported work. The proposed algorithm would help to reduce INS errors and enhance the performance of the navigation system.

This paper presents the design and development of a novel wireless pen mouse integrated with gesture recognition capabilities to enhance human-computer interaction (HCI). The proposed system leverages inertial measurement unit (IMU) sensors, specifically the MPU-6050, in conjunction with an ESP32 microcontroller to facilitate accurate motion tracking and real-time gesture recognition. By integrating traditional pen-based input with intuitive hand gesture controls, the device enables a multimodal interaction paradigm, significantly improving user experience in various application domains such as education, digital art, and assistive technology. Machine learning algorithms, including k-Nearest Neighbors (k-NN) and Support Vector Machines (SVM), are employed to classify dynamic gesture patterns with high precision. Experimental evaluations demonstrate superior performance in terms of response time, user satisfaction, and task efficiency compared to conventional input devices such as mice and touchpads. The results validate the efficacy of the proposed device in enabling natural, hands-free interactions, especially for users with physical limitations or accessibility needs. This work contributes to the advancement of smart input technologies by offering a compact, wireless, and usercentric solution for next-generation HCI.

Over the few years civilian and military applications are maneuvering with multi-sensor Unmanned Aerial Vehicles (UAV). The aim of this paper is to improve autonomous UAV positioning by using Inertial measurement unit (IMU) and Global Navigation Satellite System (GNSS) is considered. The positional accuracy of UAV is limited to GNSS signal blockage. The advancements in micro electro mechanical systems (MEMS) helps to design low-cost IMU with meter-level accuracy in positioning. The estimation of attitude, position and velocity measurements are determined from a nine degree of freedom Arduino embedded IMU sensor. The Low-cost IMU sensor has been tested under static condition and then a dynamic movement of sensor is observed under moving base-line conditions on February 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> 2020 at Koneru Lakshmaiah Education Foundation (KLEF). In this paper, covariance filters are implemented to eliminate the gyro bias and accelerometer bias for improving attitude determination of IMU navigation system. The positioning from IMU sensor is derived using standard strap-down inertial navigation system (SINS) formulation. The position error from SINS indicates that IMU sensor is able to perform well with maximum position error of 5 meters and 2.5 meters for static and dynamic environments respectively. The outcome of this work would be useful for integration of navigation sensors using kalman filter based advanced techniques for UAV applications.

A diverse array of assistive technologies have been developed to help Visually Impaired People (VIP) face many basic daily autonomy challenges. Inertial measurement unit sensors, on the other hand, have been used for navigation, guidance, and localization but especially for full body motion tracking due to their low cost and miniaturization, which have allowed the estimation of kinematic parameters and biomechanical analysis for different field of applications. The aim of this work was to present a comprehensive approach of assistive technologies for VIP that include inertial sensors as input, producing results on the comprehension of technical characteristics of the inertial sensors, the methodologies applied, and their specific role in each developed system. The results show that there are just a few inertial sensor-based systems. However, these sensors provide essential information when combined with optical sensors and radio signals for navigation and special application fields. The discussion includes new avenues of research, missing elements, and usability analysis, since a limitation evidenced in the selected articles is the lack of user-centered designs. Finally, regarding application fields, it has been highlighted that a gap exists in the literature regarding aids for rehabilitation and biomechanical analysis of VIP. Most of the findings are focused on navigation and obstacle detection, and this should be considered for future applications.