Position Tracking During Human Walking Using an Integrated Wearable Sensing System

Autonomous driving relies on reliable localization, but the Global Navigation Satellite System (GNSS) often loses accuracy or becomes unavailable in harsh environments, such as tunnels and dense urban canyons. Visual-Inertial Simultaneous Localization and Mapping (VI-SLAM), which fuses camera and Inertial Measurement Unit (IMU) data, is a key alternative in GNSS-denied scenarios. However most VI-SLAM methods overlook vehicles' inherent periodic motion features, such as zero velocity during stops or gear shifts, and linear motion during cruising. To address this, this paper proposes a visual-inertial localization method integrating these features into a tightly coupled VI-SLAM framework. First, based on IMU data, the Generalized Likelihood Ratio Test is used to detect zero velocity and linear motion. Then, Zero Velocity Update (ZUPT) residuals and Nonholonomic Constraint (NHC) residuals are constructed. These two residuals are jointly optimized with visual residuals, IMU pre-integration residuals, and marginalization residuals within a sliding-window nonlinear backend for Visual-Inertial Odometry. Experiments on self-collected datasets show that compared to the baseline algorithm, both the ZUPT-integrated algorithm and the NHC-integrated algorithm yield higher accuracy, lower error variance, and stronger robustness.

Abstract. With the continuous improvement of the hardware level of the inertial measurement unit (IMU), indoor pedestrian dead reckoning (PDR) using an inertial device has been paid more and more attention. Typical PDR system position estimation is based on acceleration obtained from accelerometers to measure the step count, estimate step length and generate the position with the heading received from angular sensors (magnetometers and gyroscopes). Unfortunately, collected signals are very responsive to the alignment of sensor devices, built-in instrumental errors and distortions from the surrounding environment. In our work, a pedestrian positioning method using step detection based on a shoe-mounted inertial unit is arranged and put to the test, and the final results are analyzed. The extended Kalman filter (EKF) provides estimation of the errors which are acquired by the XSENS IMU sensor biases. The EKF is revised with acceleration and angular rate computations by the ZUPT (zero velocity update) and ZARU (zero angular rate update) algorithms. The step detection associated with these two solutions is the perfect choice to calculate the current position and distance walked and to estimate the IMU sensors' collected errors by using EKF. The test with a shoe-mounted IMU device was performed and analyzed in order to check the performance of the recommended method. The combined PDR final results were compared to GPS/Beidou postprocessing kinematic results (outdoor environment) and to a real route which was prepared and calculated for an indoor environment. After the comparison, the results show that the accuracy of the regular-speed walking under ZUPT and ZARU combination in the case of outdoor positioning did not go beyond 0.19 m (SD) and for indoor positioning accuracy did not exceed 0.22 m (SD). The authors are conscious that built-in drift errors coming from accelerometers and gyroscopes, as well as the final position obtained by XSENS IMU, are only stable for a short time period. Based on this consideration, our future work will be focused on supporting the methods presented with radio technologies (WiFi) or image-based solutions to correct all IMU imperfections.

There has been a proliferation of smartphones, smart watches, and wearable sensors, making them ubiquitous in our daily lives. Mobile sensors have found widespread use due to their ever-decreasing cost, ease of deployment, and ability to provide continuous monitoring as opposed to sensors installed at fixed locations. Various techniques have been proposed for fall detection, gait analysis, activity monitoring, and heart rate and sleep sensing by wearable sensors and mobile phones. Compared to works that use inertial measurement unit (IMU) data or static cameras installed in the environment, there has been relatively less work using egocentric videos, meaning providing the first-person view from wearable cameras. Moreover, most of the existing studies on egocentric videos are based on only one sensor modality, namely the camera. There have been even fewer approaches that combine egocentric video data with IMU data. In this chapter, we will describe three different applications using wearable cameras together with IMU data. First, we will present an overview of a fall detection system using wearable devices, e.g., smartphones and tablets, equipped with cameras and accelerometers. Since the portable device is worn by the subject, monitoring is not limited to confined areas, and extends to wherever the subject may travel, as opposed to static sensors installed in certain rooms. Second, we will present an autonomous and robust method for counting footsteps, and tracking and calculating stride length by using both accelerometer and camera data from smartphones or Google™ glass. To provide higher precision, instead of using a preset stride length, the proposed method calculates the distance traveled with each step by using the camera data. This method is compared with the commercially available accelerometer-based step counter apps. The results show that the proposed method provides a significant increase in accuracy, and has the lowest average error rate both in number of steps taken and the distance traveled. Finally, we will provide an overview of a robust and autonomous method to detect activities with more details and context by using accelerometer and egocentric video data obtained from a smartphone.

There are several large datasets available that are captured from motion tracking systems which could be useful to train wearable human activity recognition (HAR) systems, if only their spatial data could be mapped into the equivalent inertial measurement unit (IMU) data that would be sensed on the body. In this paper, we describe a mapping from 3D Vicon motion tracking data to data collected from a BlueSense on-body IMU. We characterise the error incurred in order to discern the extent to which it is possible to generate useful training data for a wearable activity recognition system from data collected with a motion capture system. We analyse this by mapping Vicon motion tracking data to rotational velocity and linear acceleration at the head, and compare this to actual gyroscope and accelerometer data collected by an IMU mounted on the head. In a 15 min dataset comprising three static activities—sitting, standing and lying down—we find that 95% of the reconstructed gyroscope data is within an error of [−7.25;+7.46] \(deg \cdot s^{-1}\), while 95% of the reconstructed accelerometer data was contained within [−96.1;+72.9] \(m \cdot G\). However, when we introduce more movement by including data collected while walking this increases to [−19.0;+18.2] \(deg \cdot s^{-1}\) for the gyroscope and [−208;+186] \(m \cdot G\) for the accelerometer. We conclude that generating accurate IMU data from motion capture datasets is possible and could be useful in providing larger volumes of data for activity recognition tasks and in helping enable advanced, data-hungry techniques such as deep learning to be employed on a larger scale within the domain of human activity recognition.

Attitude estimation arises in many applications, and is crucial for the aerospace and aeronautic industry. A classical approach to perform the attitude estimation is via Inertial Measurement Unit (IMU) data and a set of measured observation vectors, both in the object and inertial reference frame. The attitude estimation can be then formulated as a constrained least-squares problem. As a possible alternative, attitude estimation can be performed using IMU data and a set of measured positions, hence removing the need for observation vectors. Attitude estimation based on IMU and position measurements is already undergoing industrial research. However, the underlying estimation problem does not necessarily admit a unique solution, and can therefore degenerate depending on the object trajectories. This paper theoretically investigates this approach to attitude estimation, and provides a simple, explicit relationship between the standard deviation of the attitude estimation and the object trajectories, which is independent of the choice of parametrization of the special orthogonal group SO(3).

For indoor pedestrian navigation with a shoe-mounted inertial measurement unit (IMU), the zero velocity update (ZUPT) technique is implemented to constrain the sensors' error. ZUPT uses the fact that a stance phase appears in each step at zero velocity to correct IMU errors periodically. This paper introduces three main contributions we have achieved based on ZUPT. Since correct stance phase detection is critical for the success of applying ZUPT, we have developed a new approach to detect the stance phase of different gait styles, including walking, running and stair climbing. As the extension of ZUPT, we have proposed a new concept called constant velocity update (CUPT) to correct IMU errors on a moving platform with constant velocity, such as elevators or escalators where ZUPT is infeasible. A closed-loop step-wise smoothing algorithm has also been developed to eliminate discontinuities in the trajectory caused by sharp corrections. Experimental results demonstrate the effectiveness of the proposed algorithms.

Assessment of varus thrust using inertial measurement units.

Inertial Measurement Units (IMUs) are widely utilized in shoulder rehabilitation due to their portability and cost-effectiveness, but their reliance on spatial motion data restricts their use in comprehensive musculoskeletal analyses. To overcome this limitation, we propose SWCTNet (Sliding Window CNN + Channel-Time Attention Transformer Network), an advanced neural network specifically tailored for multichannel temporal tasks. SWCTNet integrates IMU and surface electromyography (sEMG) data through sliding window convolution and channel-time attention mechanisms, enabling the efficient extraction of temporal features. This model enables the prediction of muscle activation patterns and kinematics using exclusively IMU data. The experimental results demonstrate that the SWCTNet model achieves recognition accuracies ranging from 87.93% to 91.03% on public temporal datasets and an impressive 98% on self-collected datasets. Additionally, SWCTNet exhibits remarkable precision and stability in generative tasks: the normalized DTW distance was 0.12 for the normal group and 0.25 for the patient group when using the self-collected dataset. This study positions SWCTNet as an advanced tool for extracting musculoskeletal features from IMU data, paving the way for innovative applications in real-time monitoring and personalized rehabilitation at home. This approach demonstrates significant potential for long-term musculoskeletal function monitoring in non-clinical or home settings, advancing the capabilities of IMU-based wearable devices.

IntroductionDigital home rehabilitation systems require accurate segmentation methods to provide appropriate feedback on repetition counting and exercise technique. Current segmentation methods are not suitable for clinical use; they are not highly accurate or require multiple sensors, which creates usability problems. We propose a model for accurately segmenting inertial measurement unit data for shoulder rehabilitation exercises. This study aims to use inertial measurement unit data to train and test a machine learning segmentation model for single- and multiple-inertial measurement unit systems and to identify the optimal single-sensor location.MethodsA focus group of specialist physiotherapists selected the exercises, which were performed by participants wearing inertial measurement units on the wrist, arm and scapula. We applied a novel machine learning based segmentation technique involving a convolutional classifier and Finite State Machine to the inertial measurement unit data. An accuracy score was calculated for each possible single- or multiple-sensor system.ResultsThe wrist inertial measurement unit was chosen as the optimal single-sensor location for future system development (mean overall accuracy 0.871). Flexion and abduction based exercises mostly could be segmented with high accuracy, but scapular movement exercises had poor accuracy.ConclusionA wrist-worn single inertial measurement unit system can accurately segment shoulder exercise repetitions; however, accuracy varies depending on characteristics of the exercise.

Gait rehabilitation commonly relies on bodyweight unloading mechanisms, such as overhead mechanical support and underwater buoyancy. Lightweight and wireless inertial measurement unit (IMU) sensors provide a cost-effective tool for quantifying body segment motions without the need for video recordings or ground reaction force measures. Identifying the instant when the foot contacts and leaves the ground from IMU data can be challenging, often requiring scrupulous parameter selection and researcher supervision. We aimed to assess the use of machine learning methods for gait event detection based on features from foot segment rotational velocity using foot-worn IMU sensors during bodyweight-supported treadmill walking on land and underwater. Twelve healthy subjects completed on-land treadmill walking with overhead mechanical bodyweight support, and three subjects completed underwater treadmill walking. We placed IMU sensors on the foot and recorded motion capture and ground reaction force data on land and recorded IMU sensor data from wireless foot pressure insoles underwater. To detect gait events based on IMU data features, we used random forest machine learning classification. We achieved high gait event detection accuracy (95–96%) during on-land bodyweight-supported treadmill walking across a range of gait speeds and bodyweight support levels. Due to biomechanical changes during underwater treadmill walking compared to on land, accurate underwater gait event detection required specific underwater training data. Using single-axis IMU data and machine learning classification, we were able to effectively identify gait events during bodyweight-supported treadmill walking on land and underwater. Robust and automated gait event detection methods can enable advances in gait rehabilitation.

Terrestrial mobile gravimetry using temperature-stabilized Inertial Measurement Units (IMUs) is emerging as an efficient technique for acquiring high-accuracy gravity data in dynamic environments. This study explores the integration of the iCORUS-2 IMU system with GNSS along the Cebeci-Elmadağ route in Ankara, Türkiye. The iTempStab technology was incorporated into the iCORUS-2 system to stabilize temperature variations to enhance the accuracy of gravity measurements. Notably, this research represents the results of vehicle-based terrestrial mobile gravimetry using a thermally stabilized IMU. An Extended Kalman Filter (EKF) was applied to integrate IMU and GNSS data. The advantage of terrestrial mobile gravimetry over airborne methods is the ability to make static updates for vertical gravity, zero gravity gradient and zero velocity measurements, which can be introduced into the EKF as measurement updates. The method was tested by analyzing the statistical impact of these updates at varying intervals: without any gravity measurement updates, and by applying updates at approximately 2 km, 3 km, 4 km, 5 km, 6 km, and 7 km intervals. Results indicated that without updates, a root mean square error (rmse) value between external terrestrial gravity data and mobile gravimetry estimates is 1.362 mGal. However, with updates at approximately 2 km intervals, the rmse value significantly decreased to 0.948 mGal, representing a %30.4 reduction in rmse and achieving sub-mGal accuracy. The inclusion of temperature stabilization and frequent gravity measurement updates demonstrated a marked improvement in data accuracy and consistency. These advancements not only validate the potential of mobile gravimetry for geophysical surveys but also broaden its applications in fields requiring precise gravity data acquisition.

Wearable inertial motion capture, a new type of motion capture technology, mainly estimates the human posture in 3-D space through multisensor data fusion. The available method for sensor fusion is usually aided by magnetometers to remove the drift error in yaw angle estimation, which in turn limits their application in the presence of a complex magnetic field environment. In this article, an extended Kalman filter (EKF) data fusion method is proposed to fuse the 9-axis sensor data. Meanwhile, the heuristic drift reduction (HDR) method is used to calibrate the accumulated error of a heading angle. In addition, the position in 3-D space is estimated by the foot-mounted zero-velocity-update (ZUPT) technique. Combining 3-D attitude and position, a biomechanical model of the human body is established to track the motion of a real human body. The EKF algorithm and position estimation methods are benchmarked against the golden standard, optical motion capture system, for various indoor experiments. In addition, various outdoor experiments are also conducted to verify the reliability of the proposed method. The results show that the proposed algorithm outperforms the available attitude estimation model in motion tracking and is feasible for 3-D human motion capture.

Human activity recognition is a very active research on pervasive computing and mobile health application. Many human activity systems based on inertial measurement unit (IMU) sensor data were proposed in the past few years. These systems mainly use IMU sensor placed on he torso and limbs to collect data and utilize supervised machine learning algorithms on sensor data. One main issue of these systems is that wearing multiple on-body IMU sensors may bring inconvenience to users' daily life. The other issue of these exiting methods is that an activity recognition model that is trained on a specific subject does not work well when being applied to predict another subject's activities since IMU activity data always carry information that is specific to the human subject who conducts the activities. In our work, inspired by the principle of domain adaption, we proposed a new deep-learning activity recognition model based on an adversarial network which can remove the subject-specific information within the IMU activity data and extract subject-independent features shared by the data collected on different subjects. We also for the first time use data collected from insole based IMU sensors on 8 participants for 5 common activities to build a new real world human activity dataset which can minimize the inconvenience for users to wear. We conducted experiments with our new real-world dataset. Results show that our subject independent activity recognition model outperforms state-of-art supervised learning techniques and eliminates the effects of individual differences between subjects successfully. The average recognition accuracy under the leave-one-out (L1O) condition achieves 99.0% which is higher than the performance of traditional human activity recognition system based on CNNs.

Clinicians lack objective means for monitoring if their knee osteoarthritis patients are improving outside of the clinic (e.g., at home). Previous human activity recognition (HAR) models using wearable sensor data have only used data from healthy people and such models are typically imprecise for people who have medical conditions affecting movement. HAR models designed for people with knee osteoarthritis have classified rehabilitation exercises but not the clinically relevant activities of transitioning from a chair, negotiating stairs and walking, which are commonly monitored for improvement during therapy for this condition. Therefore, it is unknown if a HAR model trained on data from people who have knee osteoarthritis can be accurate in classifying these three clinically relevant activities. Therefore, we collected inertial measurement unit (IMU) data from 18 participants with knee osteoarthritis and trained convolutional neural network models to identify chair, stairs and walking activities, and phases. The model accuracy was 85% at the first level of classification (activity), 89–97% at the second (direction of movement) and 60–67% at the third level (phase). This study is the first proof-of-concept that an accurate HAR system can be developed using IMU data from people with knee osteoarthritis to classify activities and phases of activities.

Objective: Older adults’ falls are a critical public health problem. The majority of free-living fall risk assessment methods have investigated fall predictive power of step-related digital biomarkers extracted from wearable inertial measurement unit (IMU) data. Alternatively, the examination of characteristics and frequency of naturally-occurring compensatory balance reactions (CBRs) may provide valuable information on older adults’ propensity for falls. To address this, models to automatically detect naturally-occurring CBRs are needed. However, compared to steps, CBRs are rare events. Therefore, prolonged collection of criterion standard data (along with IMU data) is required to validate model’s performance in free-living conditions. Methods: By investigating 11 fallers’ and older non-fallers’ free-living criterion standard data, 8 naturally-occurring CBRs, i.e., 7 trips (self-reported using a wrist-mounted voice-recorder) and 1 hit/bump (verified using egocentric vision data) were localized in the corresponding trunk-mounted IMU data. Random forest models were trained on independent/unseen datasets curated from multiple sources, including in-lab data captured using a perturbation treadmill. Subsequently, the models’ translation/generalization to older adults’ out-of-lab data were assessed. Results: A subset of models differentiated between naturally-occurring CBRs and free-living activities with high sensitivity (100%) and specificity (≥99%). Conclusions: The findings suggest that accurate detection of naturally-occurring CBRs is feasible. Clinical/Translational Impact- As a multi-institutional validation study to detect older adults’ naturally-occurring CBRs, suitability for larger-scale free-living studies to investigate falls etiology, and/or assess the effectiveness of perturbation training programs is discussed.