RNN-Aided Human Velocity Estimation from a Single IMU †

As the global navigation satellite system (GNSS) faces the problems of signals blocked by various objects and absorbed by many materials in indoor environments, many indoor positioning methods based on smartphones' inertial measurement unit (IMU) become increasingly applicable. Pedestrian dead reckoning (PDR) as one of the most universal methods, several algorithms with high accuracy have been proposed based on it. However, many algorithms are only applicable to scenes where pedestrians are in the “going forward” state. When the pedestrian's motions include complex patterns such as sidestepping and walking backward, PDR will not work normally as the estimated result is far from the real trajectory. In this paper, we design a deep learning model assisted PDR algorithm to recognize the pedestrian's complex motion patterns (going forward, walking backward, left sidestepping, right sidestepping) from the smartphones' IMU. We firstly employ one combined model of trained convolution neural network (CNN) and long short-term memory (LSTM) neural network to determine the correction angle for different motion patterns. Then, we estimate the pedestrian's motion direction by adding the correction angle to the attitude estimated by the inertial navigation system (INS) mechanization. Finally, the position can be computed by the estimated stride length and the corrected motion direction. Compared with different algorithms, the experimental results have proved that our algorithm can effectively process data in complex motion patterns.

The NewSpace landscape for microlaunchers is highly competitive. The cost is one of the main drivers in this competition. Several microlauncher companies have decided to outsource the GNC sub-system to external providers, procuring low-cost commercial-of-the-shelf solutions. Lately, Astos has been involved in several activities related with GNC for (micro-)launchers. Astos’ work has been focused on providing GNC development tools and flexible and easily reconfigurable GNC software that can be applied to a wide range of launch vehicles. Navigation is one of the main drivers for launcher orbital injection accuracy and it is also one of the main drivers for cost in the GNC sub-system. Usually, this cost is driven by the Inertial Navigation System (INS) or Inertial Measurement Unit (IMU), in which expensive technologies are employed such as ring-laser gyroscopes. In traditional inertial propagation algorithms, errors are accumulated due to integration resulting in a steady divergence of the position and attitude estimate. Therefore, the performance of the navigation mainly depends on the performance of the IMU. This makes an investment into a highly accurate inertial sensors, especially the gyroscope, necessary to meet the navigation requirements. Lately, the use of hybrid navigation solutions that fuse the inertial measurements with Global Navigation Satellite Systems (GNSS) measurements has diminished, in some degree, the reliance of the navigation algorithm on the IMU performance. Although this is true for translational states, as position and velocity, the same cannot be said with the same degree of validity for attitude determination and, more specifically, for roll angle estimation. One possible way to reduce the cost of inertial sensors is by using microelectromechanical systems (MEMS). The advantages of using this technology, apart from the large reduction in cost, are the significant saves in mass and power required to operate these sensors. The disadvantage is clearly the lower performance, especially in high-vibration environments such as encountered in launcher vehicles. Lately, efforts to improve the performance of MEMS navigation solutions were driven by the demand of industries such as the wearables and unmanned aerial systems. This led to the development of solutions that utilize multiple low-cost inertial measurement units. By that, the hardware cost is not gravely increased in exchange for higher navigation accuracy, reliability, and redundancy. In this paper, a navigation algorithm is proposed that incorporates the measurements from multiple low-grade MEMS IMUs. The algorithm implements parallel Extended Kalman Filters (EKF), one for each IMU, combining the inertial measurements with the position, velocity, and time solutions provided by a single GNSS receiver. The algorithm has a loosely coupled architecture and the inertial propagation is regularly reset using the fused estimates of the vehicle’s state and IMU biases (closed-loop implementation). The navigation solution, providing estimates of position, velocity, and attitude, is then a combination of the individual EKF results through weighted means based on the covariance of the individual estimates. This formulation utilizing parallel filters is called in the literature as Federated Kalman Filter architecture. An analysis will be presented relating the performance of the navigation architecture with the number of sensors used and their distribution on the launch vehicle’s body. The assessment is performed with relation to a baseline navigation architecture also implementing a loosely coupled, closed-loop, hybrid structure but employing a single low-grade IMU. The analysis will determine the most promising sensor constellation, based on navigation performance, cost and implementation complexity. The selected multiple IMU solution is then compared with two different navigation architectures: Strapdown inertial propagation utilizing a single high-grade IMU; and hybrid navigation scheme implementing a loosely coupled, closed-loop architecture with a medium grade IMU. The performance evaluation is conducted utilizing simulated measurements retrieved from high-fidelity 6 degrees-of-freedom non-linear simulator of a representative microlauncher scenario. The simulators’ dynamics, kinematics, environment, and sensor modelling are provided by the ASTOS tool. Astos intends to further study the feasibility and performance of the proposed Inertial/GNSS hybrid navigation architecture with multiple IMUs. In the roadmap for raising the algorithm’s technology readiness level, more advanced simulations are foreseen, with improved sensor error modelling and previously unmodelled dynamical effects such as flexibility and sloshing. Additionally, processor- and hardware-in-the-loop tests with real sensor data are planned.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

The use of wearable foot-based inertial measurement units (IMUs) incorporated in a navigation system can address the problem of single-person location tracking in situations and environments where GPS signals may be unavailable or inconsistent; e.g., in buildings. This Pedestrian Dead Reckoning (PDR) approach, which enables standalone personal tracking, is an idea that has been attempted many times with varying degrees of success. There are two main approaches to solve this difficult problem: the first is to use inertial measurement units (IMUs) in a filter to apply stride-length, heading correction, and zero-velocity updates to a Kalman filter to get a position solution, while the other is to use Machine Learning techniques to better identify and classify a person’s movement to achieve an accurate position. This paper continues on the path of the former method by investigating the feasibility of PDR using a pair of low cost IMUs along with a pair of relative position and attitude magnetic sensors connected to an individual’s feet. Laverne, et al., [1] have shown that incorporating a ranging measurement between two IMU-based foot sensors significantly increases the accuracy of the pedestrian dead reckoning (PDR) solution. This work will be utilizing a solution more desirable for wearable PDR applications by incorporating smaller, low cost, commercial grade IMUs. To compensate for this degradation in quality, the ranging sensor is replaced with two small magnetic sensors that provide a 6 degree-of-freedom relative pose estimate between the sensors. In addition to a ranging measurement, this paper will introduce the use of a relative rotation measurement update to assist with drift reduction.

The acceleration of a vehicle is important information in vehicle states. The vehicle acceleration is measured by an inertial measurement unit (IMU). However, gravity affects the IMU when there is a transition in vehicle attitude; thus, the IMU produces an incorrect signal output. Therefore, vehicle attitude information is essential for obtaining correct acceleration information. This paper proposes a convolutional neural network (CNN) for attitude estimation. Using sequential data of a vehicle’s chassis sensor signal, the roll and pitch angles of a vehicle can be estimated without using a high-cost sensor such as a global positioning system or a six-dimensional IMU. This paper also proposes a dual-extended Kalman filter (DEKF), which can accurately estimate acceleration/angular velocity based on the estimated roll/pitch information. The proposed method is validated by real-car experiment data and CarSim, a vehicle simulator. It accurately estimates the attitude estimation with limited sensors, and the exact acceleration/angular velocity is estimated considering the roll and pitch angle with de-noising effect. In addition, the DEKF can improve the modeling accuracy and can estimate the roll and pitch rates.

This paper provides a hybrid approach between Wi-Fi and pedestrian dead reckoning (PDR) for the iPhones in indoor environments, and addresses the following two problems. First, since Apple Inc. no longer provides public information about currently-connected Wi-Fi (e.g., service set identifier, received signal strength and channel), Wi-Fi based pedestrian tracking apps that run on the iPhones are more restricted compared to other ones on the Android platforms. Second, even the PDR approach provides such a great way for self-localization, it suffers from accumulated errors of inertial sensors embedded in the smartphones. We propose a conversion function from a Wi-Fi status value to a proximity for the localization purpose. Then, a mobile iPhone collects mobility information from inertial measurements unit (IMU), inputs to the PDR, and combines with Wi-Fi proximity in order to perform accurate self-localization and tracking. Moreover, we improve the PDR by reducing drifting effects caused by the IMU biases. Experiment results show that the proposed scheme is effective and has low complexity, while bringing the benefits from smartphone IMU.

Visual–inertial estimation of velocity for multicopters based on vision motion constraint

Substations are critical infrastructures in modern power systems, requiring accurate and reliable personnel positioning to ensure operational safety and inspection efficiency. Pedestrian dead reckoning (PDR) technology has emerged as a preferred solution for real-time seamless positioning due to its autonomous, anti-jamming and passive characteristics. However, in practical applications, the PDR algorithm faces challenges such as the inability to provide absolute positioning due to unknown starting points and cumulative errors over time. To address these challenges, this paper proposes a head-mounted pedestrian positioning system (HPPS) integrating fisheye camera/real-time kinematic (RTK)/PDR based on factor graph optimization (FGO). First, a skyward-facing fisheye camera identifies non-line-of-sight (NLOS) signals caused by architectural obstructions, improving the absolute positioning accuracy of RTK. Next, an adaptive pedestrian gait detection and step length estimation algorithm based on head-mounted inertial measurement units (IMU) is developed to enhance PDR robustness. Finally, the FGO framework integrates the inputs of the fisheye camera, RTK, and PDR to achieve seamless indoor-outdoor positioning. Experimental results demonstrate that the proposed system achieves horizontal root mean square error (RMSE) values of less than 0.42 m and 0.35 m in two distinct outdoor substation environments and less than 1.36 m in indoor-outdoor scenarios. This work highlights the integration of precise positioning technologies within the Internet of Things (IoT) framework, advancing smart grid applications and enabling effective location tracking in complex environments.

A new approach is proposed to identify gait events in non-laboratory environments with a single inertial measurement unit (IMU) embedded inside shoe. The aim of our work is to develop a useful clinical tool for monitoring individuals walking disability and detect specific pathological gait patterns. Temporal parameters of gait are determined by classification of accelerations and angular velocities. Wavelets denoising of IMU signals allows for an important amount of information that is exploited in different manners for event identification. It was found that wavelet denoising enhanced specific turning points which could effectively identify gait events. The method is verified by comparing the results of video-based motion capture system and force plates as conventional standards. This portable gait-monitoring system allows for versatile application beyond gait laboratory.

Automatic lecture recording is an appealing alternative approach to manually recording lectures in the process of online course making as it can to a large extent save labor cost. The key of the automatic recording system is lecturer tracking, and the existing automatic tracking methods tend to lose the target in the case of lecturer’s rapid movement. This article proposes a lecturer tracking system based on MobileNet-SSD face detection and Pedestrian Dead Reckoning (PDR) technology to solve this problem. First, the particle filter algorithm is used to fuse the PDR information with the rotation angle information of the Pan-Tilt camera, which can improve the accuracy of detection under the tracking process. In addition, to improve face detection performance on the edge side, we utilize the OpenVINO toolkit to optimize the inference speed of the Convolutional Neural Networks (CNNs) before deploying the model. Further, when the lecturer is beyond the camera’s field of view, the PDR auxiliary module is enabled to capture the object automatically. We built the entire lecture recording system from scratch and performed the experiments in the real lectures. The experimental results show that our system outperforms the systems without a PDR module in terms of the accuracy and robustness.

A Map aided Pedestrian Dead Reckoning (PDR) algorithm is proposed to mitigate the drift errors and step detection limitations of pedestrian dead reckoning algorithm with handheld sensors in indoor and outdoor spaces. Specific to this context is the changing lever-arm between the handheld device and the pedestrian center of mass that introduces a misalignment between the inertial sensors and the walking directions. To address these challenges, an adaptive routing graph is built based on possible pedestrian's motions, which depend on personal mobility profile and surroundings. An adaptive decision process is also developed to fuse map data with GNSS positions and PDR outputs in a particle filter. The performance is assessed with 1km walk experiments. Main contributions are (1) the calibration of the PDR step length model using both GNSS and map data during straight line travels with miss/over-detected steps modeled by the particle filter; (2) the estimation of angular misalignment between the walking and the handheld unit pointing directions in geometrically constrained areas; (3) a dynamic choice of opportune periods and measurements to calibrate the PDR outputs and improve the positioning process.

Abstract. With the continuous improvement of the hardware level of the inertial measurement unit (IMU), indoor pedestrian dead reckoning (PDR) using an inertial device has been paid more and more attention. Typical PDR system position estimation is based on acceleration obtained from accelerometers to measure the step count, estimate step length and generate the position with the heading received from angular sensors (magnetometers and gyroscopes). Unfortunately, collected signals are very responsive to the alignment of sensor devices, built-in instrumental errors and distortions from the surrounding environment. In our work, a pedestrian positioning method using step detection based on a shoe-mounted inertial unit is arranged and put to the test, and the final results are analyzed. The extended Kalman filter (EKF) provides estimation of the errors which are acquired by the XSENS IMU sensor biases. The EKF is revised with acceleration and angular rate computations by the ZUPT (zero velocity update) and ZARU (zero angular rate update) algorithms. The step detection associated with these two solutions is the perfect choice to calculate the current position and distance walked and to estimate the IMU sensors' collected errors by using EKF. The test with a shoe-mounted IMU device was performed and analyzed in order to check the performance of the recommended method. The combined PDR final results were compared to GPS/Beidou postprocessing kinematic results (outdoor environment) and to a real route which was prepared and calculated for an indoor environment. After the comparison, the results show that the accuracy of the regular-speed walking under ZUPT and ZARU combination in the case of outdoor positioning did not go beyond 0.19 m (SD) and for indoor positioning accuracy did not exceed 0.22 m (SD). The authors are conscious that built-in drift errors coming from accelerometers and gyroscopes, as well as the final position obtained by XSENS IMU, are only stable for a short time period. Based on this consideration, our future work will be focused on supporting the methods presented with radio technologies (WiFi) or image-based solutions to correct all IMU imperfections.

With the wide application of artificial intelligence technology in intelligent education, intelligent office and other fields, handwritten optical character recognition (OCR) as an important content of intelligent education research has been widely concerned. Traditional character recognition technology is mainly for high-quality printed images, but there are some problems in English handwriting, such as irregular handwriting, overlapping letters, word smearing, etc., which can not achieve the recognition accuracy of printed images. In particular, the lack of database construction for handwriting images with scrawl and daub increases the difficulty of model training, so handwriting recognition is a very challenging task. In view of the existing problems mentioned above and the limitations of traditional handwriting recognition methods, this paper studies the English handwriting recognition method based on deep learning technology, and realizes the English handwriting image detection and recognition system on this basis. The main tasks of this paper are as follows: 1. The database of English handwriting image is established, including neat, scrawl, neat and smeared and scrawl and smeared. Label information is assigned to each type of data, and SVM classifier is trained to classify new samples using this database. In order to reduce the interference of smear area and background information on the character recognition model, the image is preprocessed with smear block removal, gray scale and tilt correction, and the processing results are used as the input of the subsequent detection and recognition model. 2. A convolutional cyclic neural network based on cross-channel and cross-space attention mechanism is proposed. The convolutional neural network is used to extract local feature information and cross-channel and cross-space attention mechanisms are introduced to reduce the loss of key features. At the same time, bidirectional recurrent neural network is used to capture the context dependence of characters and obtain the word order information of English script. 3. An English handwritten recognition method based on Semantic Reasoning Networks (SRN) and CCSACRNN (SNCN) is proposed.

Text classification is a task to assign text documents according to its content to one or more classes automatically. Recently character-level models using deep neural networks have been developed to do classification text. Moreover, in some cases, character-level models have outperformed word-level models and traditional models, especially on user-generated dataset. The topologies that have been used for the character-level models are convolutional neural networks (CNN) and bidirectional recurrent neural networks (Bi-RNN), with its variants; long short-term memory (LSTM) and gated recurrent units (GRU). In this paper, CNN, Bi-RNN, and the combination of both are tested with character-level features and word-level features for text classification on English and Indonesian social media datasets. On small size datasets, word-level model outperformed character-level models. However, on dataset with millions of data, character-level model outperformed word-level model. Further analysis on the evaluation of word-level and character-level models is also discussed in this paper.

Inter-segment joint angles can be obtained from inertial measurement units (IMUs); however, accurate 3D joint motion measurement, which requires sensor fusion and signal processing, sensor alignment with segments and joint axis calibration, can be challenging to achieve. Can an artificial neural network modeling framework be used for direct, real-time conversion of IMU data to joint angles during walking and running, and how does sensor number, location on the body and gait speed impact prediction accuracy? Thirty healthy adult participants performed gait experiments in which kinematics data were obtained from self-placed IMUs and video motion analysis, the reference standard for joint kinematics. Data were collected during walking at 0.5 m/s, 1.0 m/s and 1.5 m/s, as well as during running at 2.0 m/s and 3.0 m/s. A generative adversarial network was trained and used to predict lower limb joint angles at all gait speeds using IMU data only, and prediction accuracy assessed using all combinations of sensors. Joint angle prediction accuracy was strongly dependent on the number and location of sensors, as well as walking and running speed. A single IMU could be used to predict sagittal plane joint angles at either the hip, knee or ankle during walking with RMS errors below 4.0°, though highest 3D joint motion accuracy was obtained with two or three IMUs for a given joint. This study reports a modeling framework for direct conversion of IMU data to joint angles without signal processing or joint calibration. The findings suggest that combinations of up to four IMUs reproduce hip, knee and ankle joint kinematics simultaneously during walking and running with highest accuracy. The findings may be useful in maximizing accuracy of IMU-based motion measurements of the lower limb joints in applications such as remote monitoring of movement, sports training, and in rehabilitation.