Dual-Modality Cross-Interaction-Based Hybrid Full-Frame Video Stabilization

The use of unmanned aerial vehicles (UAV) systems has increased in recent years. Therefore,studies on UAVs have increased today. In this direction, the production of UAV systems with domestic resources has gained importance. In this study, it is desired to develop a domestic and national flight control card and software. In the flight control board designed for the UAV, it is aimed to keep the vehicle in balance in the air. Accurate measurement of platform orientation plays an important role in many applications such as aerospace, robotics, navigation, marine, machine interaction [1]. Inertial Measurement Unit (IMU) sensor was used to accurately measure the orientation of the UAV. IMU sensor is widely used in UAVs due to its light weight and low energy consumption. In this direction, the need for a filter has emerged in the IMU sensor, which is used to accurately measure the orientation of the unmanned aerial vehicle. In this study, a complementary filter was applied on the IMU sensor. Thanks to this filter, it has been observed that the accuracy of the data received from the IMU sensor has increased. Based on the data obtained, a Proportional Integral Derivative (PID) algorithm was developed, and the vehicle was kept in balance. In this study, ARMCortex-M4 based STM32F407VG microcontroller and MPU6050 as IMU sensor were used. Keil-uVision5 compiler is preferred for software. As a result, high accuracy in the orientation detection of unmanned aerial vehicles was obtained by applying a complementary filter on the IMU sensor.

Human activity recognition is a very active research on pervasive computing and mobile health application. Many human activity systems based on inertial measurement unit (IMU) sensor data were proposed in the past few years. These systems mainly use IMU sensor placed on he torso and limbs to collect data and utilize supervised machine learning algorithms on sensor data. One main issue of these systems is that wearing multiple on-body IMU sensors may bring inconvenience to users' daily life. The other issue of these exiting methods is that an activity recognition model that is trained on a specific subject does not work well when being applied to predict another subject's activities since IMU activity data always carry information that is specific to the human subject who conducts the activities. In our work, inspired by the principle of domain adaption, we proposed a new deep-learning activity recognition model based on an adversarial network which can remove the subject-specific information within the IMU activity data and extract subject-independent features shared by the data collected on different subjects. We also for the first time use data collected from insole based IMU sensors on 8 participants for 5 common activities to build a new real world human activity dataset which can minimize the inconvenience for users to wear. We conducted experiments with our new real-world dataset. Results show that our subject independent activity recognition model outperforms state-of-art supervised learning techniques and eliminates the effects of individual differences between subjects successfully. The average recognition accuracy under the leave-one-out (L1O) condition achieves 99.0% which is higher than the performance of traditional human activity recognition system based on CNNs.

Gait rehabilitation commonly relies on bodyweight unloading mechanisms, such as overhead mechanical support and underwater buoyancy. Lightweight and wireless inertial measurement unit (IMU) sensors provide a cost-effective tool for quantifying body segment motions without the need for video recordings or ground reaction force measures. Identifying the instant when the foot contacts and leaves the ground from IMU data can be challenging, often requiring scrupulous parameter selection and researcher supervision. We aimed to assess the use of machine learning methods for gait event detection based on features from foot segment rotational velocity using foot-worn IMU sensors during bodyweight-supported treadmill walking on land and underwater. Twelve healthy subjects completed on-land treadmill walking with overhead mechanical bodyweight support, and three subjects completed underwater treadmill walking. We placed IMU sensors on the foot and recorded motion capture and ground reaction force data on land and recorded IMU sensor data from wireless foot pressure insoles underwater. To detect gait events based on IMU data features, we used random forest machine learning classification. We achieved high gait event detection accuracy (95–96%) during on-land bodyweight-supported treadmill walking across a range of gait speeds and bodyweight support levels. Due to biomechanical changes during underwater treadmill walking compared to on land, accurate underwater gait event detection required specific underwater training data. Using single-axis IMU data and machine learning classification, we were able to effectively identify gait events during bodyweight-supported treadmill walking on land and underwater. Robust and automated gait event detection methods can enable advances in gait rehabilitation.

Limb exercises are common in physical therapy to improve range of motion (RoM), strength, and flexibility of the arm/leg. To improve therapy outcomes and reduce cost, motion tracking systems have been used to monitor the user's movements when performing the exercises and provide guidance. Traditional motion tracking systems are based on either cameras or inertial measurement unit (IMU) sensors. Camera-based systems face problems caused by occlusion and lighting. Traditional IMU-based systems require at least two IMU sensors to track the motion of the entire limb, which is not convenient for use. In this paper, we propose a novel limb motion tracking system that uses a single 9-axis IMU sensor that is worn on the distal end joint of the limb (i.e., wrist for the arm or ankle for the leg). Limb motion tracking using a single IMU sensor is a challenging problem because 1) the noisy IMU data will cause drift problem when estimating position from the acceleration data, 2) the single IMU sensor measures the motion of only one joint but the limb motion consists of motion from multiple joints. To solve these problems, we propose a recurrent neural network (RNN) model to estimate the 3D positions of the distal end joint as well as the other joints of the limb (e.g., elbow or knee) from the noisy IMU data in real time. Our proposed approach achieves high accuracy with a median error of 7.2/7.1 cm for the wrist/elbow joint in leave-one-subject-out cross validation when tracking the arm motion, outperforming the state-of-the-art approach by more than 10%. In addition, the proposed model is lightweight, enabling real-time applications on mobile devices.Clinical relevance- This work has great potential to improve limb exercises monitoring and RoM measurement in home-based physical therapy. It is also cost effective and can be made available widely for immediate application.

This paper presents a novel approach for an adaptive Extended Kalman Filter (EKF), which is able to handle bad signal quality caused by shading or loss of Doppler Effect for low cost Global Navigation Satellite System (GNSS) receiver and Inertial Measurement Unit (IMU) sensors fused in a loosely coupled way. It uses the estimated position error as well as the speed to calculate the standard deviation for the measurement uncertainty matrix of the Kalman Filter. The filter is very easy to implement, because some conversions of the measurement, as well as the state variables, are made to reduce the complexity of the Jacobians, which are used in the EKF filter algorithm. The filter implementation is tested within a simulation and with real data and shows significantly better performance, compared to a standard EKF. The developed filter is running in realtime on an embedded device and is able to perform position and attitude estimation of a vehicle with low cost sensors.

In this paper, we proposed an intuitive teleoperation system combining surface-electromyogram (sEMG) with inertia measurement unit (IMU) sensors. Two IMU sensors were worn in the upper arm and forearm to capture arm motion. The results were sent as control command to the remote robotic manipulator in task space. Pattern recognition algorithm was employed to decode sEMG signals collected from the forearm to control a humanoid mechanical hand. As a validation of the proposed system, we evaluated the performance of the system which included a Universal Robot 10 (UR10), a homemade humanoid pro-prosthetic hand (SJT-6) and two homemade armbands. Final experiments showed that the success rate on moving spherical object can be up to 86.7%.

Unmanned Aerial Vehicles (UAVs) play a crucial role in navigation, requiring accurate sensors to determine position, speed, and orientation, especially in unknown environments. Direct navigation systems like the Global Positioning System (GPS) provide positional data, while indirect systems, such as the Inertial Measurement Unit (IMU), integrate accelerometer and gyroscope data to supply speed and orientation information. This study investigates the integration of GPS and IMU sensors using the Unscented Kalman Filter (UKF) to improve localization accuracy on a cost-effective UAV Quadrotor equipped with LiDAR Livox. The research methodology involved collecting raw data from GPS, IMU, and LiDAR sensors during UAV flights. These data were processed using a UKF-based mathematical model to fuse sensor inputs and generate accurate point cloud models. Results show that the UKF fusion method achieved a localization accuracy of 0.403 m, with maximum residuals recorded as 1.332 m for the X axis, 20.421 m for the Y axis

ABSTRACT   In recent years, Taiwan has suffered from large-scale sediment-related disasters frequently caused by climate change induced extreme rainfall events. Fresh wood was usually recruited with large-scale sediment-related disasters and trapped in channels. Once a flash flood occurs, wood accumulated in the river consequently travels, threatening the safety of structures in the river. Furthermore, driftwood forms log jams leading to river blockage and backwater, causing overflooding. Therefore, realizing the initiation of motion mechanism of driftwood helps prevent driftwood disasters.According to previous studies on the initiation of wood motion, driftwood interacts with sediment and results in sliding, rolling, pivoting and floating motion modes dependent on discharge and riverbed conditions. To explore the motion mode of the initiation of wood motion on a mobile bed, this study conducts a flume experiment using driftwood equipped with an IMU (Inertial measurement unit) sensor that measures the posture of driftwood’s motion in the Eular Angle. According to the IMU’s data, the above four motion modes can be interpreted. Meanwhile, using the laser scanner to establish the digital elevation model and explore the correlation between sediment and driftwood motion on different mobile-bed conditions.

Most of the latest context-based applications capture the mobility of a user using Inertial Measurement Unit (IMU) sensors like accelerometer and gyroscope which do not need explicit user-permission for application access. Although these sensors provide highly accurate mobility context information, existing studies have shown that they can lead to undesirable leakage of location information. To evade this breach of location privacy, many of the state-of-the-art studies suggest to impose stringent restrictions over the usage of IMU sensors. However, in this paper, we show that typing and smartphone engagement patterns can act as an alternative modality to sniff the mobility context of a user, even if the IMU sensors are not sampled at all. We develop an adversarial framework, named ConType, which exploits the signatures exposed by typing and smartphone engagement patterns to track the mobility of a user. Rigorous experiments with in-the-wild dataset show that ConType can track the mobility contexts with an average micro-F 1 of 0.87 (±0.09), without using IMU data. Through additional experiments, we also show that ConType can track mobility stealthily with very low power and resource footprints, thus further aggravating the risk.

Precise navigation is a vital need for many modern vehicular applications. The global positioning system (GPS) cannot provide continuous navigation information in urban areas. The widely used inertial navigation system (INS) can provide full vehicle state at high rates. However, the accuracy diverges quickly in low cost microelectromechanical systems (MEMS) based INS due to bias, drift, noise, and other errors. These errors can be corrected in a stationary state. But detecting stationary state is a challenging task. A novel stationary state detection technique from the variation of acceleration, heading, and pitch and roll of an attitude heading reference system (AHRS) built from the inertial measurement unit (IMU) sensors is proposed. Besides, the map matching (MM) algorithm detects the intersections where the vehicle is likely to stop. Combining these two results, the stationary state is detected with a smaller timing window of 3 s. A longer timing window of 5 s is used when the stationary state is detected only from the AHRS. The experimental results show that the stationary state is correctly identified and the position error is reduced to 90% and outperforms previously reported work. The proposed algorithm would help to reduce INS errors and enhance the performance of the navigation system.

Hybrid of discrete wavelet transform and adaptive neuro fuzzy inference system for overall driving behavior recognition

Gesture recognition by using Inertial Measurement Unit (IMU) sensors plays an important role in various Internet of Things (IOT) applications, e.g., smart home, intelligent medical system and so on. Traditional technologies usually utilize machine learning algorithms to train different gestures during the offline phase, then recognize the gesture during the online phase. However, such technologies cannot recognize these gestures without prior training. Even for the same gesture, with different gesture amplitude may result in unsuccessful recognition. Also if we change the person to perform the same gesture, the algorithms fails. In order to overcome these drawbacks, we propose an approach, which will be able to track the human body motion in realtime and also recognize complicated gestures. It utilizes the accelerometer information and proposes comprehensive localization algorithms for each deployed sensor attached on the human body. Then, it takes the correlation and limitation among body parts into account to recognize the gesture. Our experiments results show that, the successful recognition rate of our algorithm is 100%. Furthermore, any part of the human body can be well tracked, the tracking accuracy can reach 0.06m.

A diverse array of assistive technologies have been developed to help Visually Impaired People (VIP) face many basic daily autonomy challenges. Inertial measurement unit sensors, on the other hand, have been used for navigation, guidance, and localization but especially for full body motion tracking due to their low cost and miniaturization, which have allowed the estimation of kinematic parameters and biomechanical analysis for different field of applications. The aim of this work was to present a comprehensive approach of assistive technologies for VIP that include inertial sensors as input, producing results on the comprehension of technical characteristics of the inertial sensors, the methodologies applied, and their specific role in each developed system. The results show that there are just a few inertial sensor-based systems. However, these sensors provide essential information when combined with optical sensors and radio signals for navigation and special application fields. The discussion includes new avenues of research, missing elements, and usability analysis, since a limitation evidenced in the selected articles is the lack of user-centered designs. Finally, regarding application fields, it has been highlighted that a gap exists in the literature regarding aids for rehabilitation and biomechanical analysis of VIP. Most of the findings are focused on navigation and obstacle detection, and this should be considered for future applications.

Over the few years civilian and military applications are maneuvering with multi-sensor Unmanned Aerial Vehicles (UAV). The aim of this paper is to improve autonomous UAV positioning by using Inertial measurement unit (IMU) and Global Navigation Satellite System (GNSS) is considered. The positional accuracy of UAV is limited to GNSS signal blockage. The advancements in micro electro mechanical systems (MEMS) helps to design low-cost IMU with meter-level accuracy in positioning. The estimation of attitude, position and velocity measurements are determined from a nine degree of freedom Arduino embedded IMU sensor. The Low-cost IMU sensor has been tested under static condition and then a dynamic movement of sensor is observed under moving base-line conditions on February 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nd</sup> 2020 at Koneru Lakshmaiah Education Foundation (KLEF). In this paper, covariance filters are implemented to eliminate the gyro bias and accelerometer bias for improving attitude determination of IMU navigation system. The positioning from IMU sensor is derived using standard strap-down inertial navigation system (SINS) formulation. The position error from SINS indicates that IMU sensor is able to perform well with maximum position error of 5 meters and 2.5 meters for static and dynamic environments respectively. The outcome of this work would be useful for integration of navigation sensors using kalman filter based advanced techniques for UAV applications.

The importance of indoor mapping has surged across various applications, encompassing human navigation, indoor exploration, and hazardous site mapping. With satellite positioning systems ineffective indoors, alternative navigational sensors and methods, such as inertial measurement units (IMU) and simultaneous localization and mapping algorithms (SLAM), have been harnessed to furnish precise indoor positioning for mapping endeavors. Leveraging LiDAR-based SLAM, which is sensitive to environmental textures and various pose configurations, addresses these challenges. In order to accomplish reliable and precise indoor 3D mapping, this study combines the Velodyne VLP-16 3D-LiDAR sensor with the Pixhawk Cube Orange, which has a 9 Degrees of Freedom (DOF) IMU sensor. The study uses the LIO-SAM (LiDAR Inertial Odometry and Simultaneous Localization and Mapping) technique, which integrates data from LiDAR and IMU to increase mapping precision. It is carried out at the BAAK Dr. Angka Building, Institut Teknologi Sepuluh Nopember, Surabaya. The experimental results reveal that the 3D indoor mapping exhibits commendable visualization and high geometric accuracy, meeting the LoD 3 criteria. Validation using a roll meter yielded a root mean square error (RMSE) of 0.3552 meters, indicating considerable accuracy. The accuracy was further validated by showing LiDAR point cloud errors ranging from 7.92% to 15.75%. Linear regression analysis confirmed a high correlation (R2 0.9954) between LiDAR point cloud measurements and manual measurements, substantiating the method’s reliability. In order to accomplish reliable and precise indoor 3D mapping, this study combines the Velodyne VLP-16 3D-LiDAR sensor with the Pixhawk Cube Orange, which has a 9 Degrees of Freedom (DOF) IMU sensor. The study uses the LIO-SAM (LiDAR Inertial Odometry and Simultaneous Localization and Mapping) technique, which integrates data from LiDAR and IMU to increase mapping precision. It is carried out at the BAAK Dr. Angka Building, Institut Teknologi Sepuluh Nopember, Surabaya.