A Real-Time Gait Recognition and Trajectory Prediction Scheme for Exoskeleton During Continuous Multilocomotion Tasks

The estimation of human gait phase has recently been used to parameterize continuous lower limb exoskeleton controllers. By deriving the gait phase estimation directly from the user's biological signals (i.e., joint/segment angles), the user can be granted an avenue to control their progression through the gait cycle. The inherent challenge for this strategy is the high accuracy required, as the estimated gait phase determines the timing of the device's assistive and resistive torques. Poor device timing can result in desynchronization of the device with the user. To address this challenge, this paper presents a novel method for using the information from the contralateral leg to improve the estimation accuracy. Simulation results showed a 75.2% reduction in estimation error relative to the state of the art method for able-bodied individuals. Additionally, an average of 47.6% reduction in estimation error was observed among data from three hemiparetic stroke patients. Across both able bodied and hemiparetic subjects, an estimation RMSE of 2.28% was shown. The evaluation of this technique displays its promise for the control of rehabilitative robotics.Clinical Relevance-This improvement to lower limb exoskeleton control aims to encourage greater user participation in robot assisted gait training for stroke rehabilitation.

Gait phase is an important state variable that helps exoskeletons provide assistance to users. Exoskeleton systems need to determine the assistance time and calculate the reference assistance force based on the gait phase. A large number of studies have used Finite State Machines (FSMs) or Adaptive Oscillators (AOs) to estimate gait phase. However, FSMs require manual calibration of gait events, and AOs can only estimate the phase of stable gait in real time. Both are difficult to adapt to irregular changes in gait. Deep learning algorithms can learn the nonlinear characteristics of kinematic data to estimate joint torque, gait phase, etc. The control algorithm based on Long Short-Term Memory (LSTM) Neural Network can automatically detect gait events according to the joint angle collected by Inertial Measurement Unit (IMU), effectively responding to dynamic changes in gait. This paper designed a Light-Weight Long Short-Term Memory (LWLSTM) Neural Network to estimate gait phase precisely and reduce training time of LSTM model. The experimental results showed that the Root Mean Square Error (RMSE) of LWLSTM was 8. 0 0 3 3%, quite lower than Convolutional Neural Network (CNN) and AO.

User gait phase estimation plays a key role for the seamless control of the lower-limb robotic assistive devices (e.g., exoskeletons or prostheses) during ambulation. To achieve this, several studies have attempted to estimate the gait phase using a thigh or shank angle. However, their estimation resulted in some deviation from the actual walking and varied across the walking speeds. In this study, we investigated the different setups using for the machine learning approach to obtain more accurate and consistent gait phase estimation for the robotic transfemoral prosthesis over different walking speeds. Considering the transfemoral prosthetic application, we proposed two different sensor setups: i) the angular positions and velocities of both thigh and torso (S1) and ii) the angular positions and velocities of both thigh and torso, and heel force data (S2). The proposed setups and method are experimentally evaluated with three healthy young subjects at four different walking speeds: 0.5, 1.0, 1.5, and 2.0 m/s. Both results showed robust and accurate gait phase estimation with respect to the ground truth (loss value of S1: 4.54e-03 Vs. S2: 4.70e-03). S1 had the advantage of a simple equipment setup using only two IMUs, while S2 had the advantage of estimating more accurate heel-strikes than S1 by using additional heel force data. The choice between the two sensor setups can depend on the researchers' preference in consideration of the device setup or the focus of the interest.

Continuous gait phase plays an important role in wearable robot control. This study focuses on the online estimation of continuous gait phase based on robotic transtibial prosthesis signals. First, we adopt the prosthetic foot deformation information to detect the heel strike as the start timing (reset 0 rad) of one gait cycle. Then we conduct the gait phase estimation based on adaptive oscillators using the prosthetic shank angle signal as input. Three transtibial amputees were recruited in this study and they walked on the treadmill at different speeds (slow, normal and fast) and on different ramps ($10^{\circ}, 5^{\circ}, 0^{\circ}$, -5° and -10°) in the experiment. The root-meansquare error between online estimation result and ground truth gait phase is calculated. The maximum and minimum errors are 0.147 rad and 0.058 rad, and the corresponding ratios in one gait cycle are 2.34% and 0.92%. This study achieves good performance and provides an effective method to estimate the continuous gait phase, which will instruct robotic transtibial prosthesis control.

To improve human-machine cooperation and enhance the accuracy of gait recognition in wearable lower limb exoskeletons, an enhancement method of gait recognition based on adaptive feature selection and an optimized machine learning algorithm was proposed. In this study, surface electromyography (sEMG) signals of rectus femoris, medialis femoris, lateralis femoris, semitendinosus and biceps femoris were recorded during level-ground walking. Then, time domain, frequency domain, time-frequency domain and nonlinear features were extracted. The integrated value of electromyography, variance, root mean square and wavelength were selected as the time domain features, and the mean power frequency and median frequency were selected as the frequency domain features. Wavelet packet energy was selected as the time-frequency domain feature. Nonlinear features including approximate entropy, sample entropy and fuzzy entropy of sEMG were extracted. To reduce feature dimension and improve the calculation efficiency, adaptive feature selection was performed by particle swarm optimization combined with sigmoid function. Then, the feature matrix was determined as the input for a random forest classifier to recognize different gait phases. An adaptive optimization mechanism based on Bayesian optimization was applied to random forest. Compared with random forest, the overall performance of the optimized model was improved. Its accuracy was increased by 3.57%. The feature selection and adaptive optimization mechanisms in gait recognition not only enhance the accuracy of random forest algorithms applied to sEMG for gait prediction, but also facilitate the flexibility and consistency required for lower limb exoskeleton gait control.

In this paper, a system solution is designed to improve the service efficiency of intersection. For the purpose, a method of trajectory prediction for vehicle is put forward, which is based on the basis of realize multi-targets tracking. The characteristics of traffic video, structural of road profile and movement of vehicles have been analyzed. We designed the multi-targets tracking algorithm based on particle filter for traffic video sequence, in order to improve the accuracy and rapidity, the heuristic method for multi-targets data correlation is proposed. Based on the results of tracking, we analyze the rule of movement parameters of vehicle as the path has changed, according to the features of structured road the vehicle running status parameters is simplified, and a Hidden Markov Model (HMM) has trained, which is used for vehicle trajectory prediction. In the end, Analysis the basic reason of traffic conflict and the dynamic model of traffic conflict zone is designed. Experimental results demonstrate that, the method of tracking based on particle filter has good tracking performance in traffic video; the vehicle trajectory can be predicted rapidly in the required time, and traffic conflict zone is produced accurately.

A typical man–machine coupling system could provide the wearer a coordinated and assisted movement by the lower limb exoskeleton. The process of cooperative movement relies on the accurate perception of the wearer’s human movement information and the accurate planning and control of the joint movement of the lower limb exoskeleton. In this paper, a neural network and a Long-Short Term Memory (LSTM) machine learning model method is proposed to predict the actual movement trajectory of the human body’s lower limbs. Then a wearable joint angle measurement device was designed for gait trajectory prediction, which can be used for predictive control through machine learning methods. The experimental results show that the LSTM model can accurately predict the gait trajectory with an average mean square error. This method has practical significance for prediction the trajectory of the lower limb exoskeleton.

Trajectory prediction is a prerequisite for missile and high-speed vehicle guidance interception. To address the trajectory problem for the near space hypersonic unpowered gliding vehicle. Firstly, the trajectory prediction mechanism is analyzed. Based on the existing NSHV trajectory prediction method, the key techniques of trajectory prediction are discussed from the perspectives of prediction parameter selection and prediction parameter description. Then, a trajectory prediction method based on Hough transform is proposed. The core of the method is to select the longitude latitude, and high direction positions as prediction parameters, and use the Hough transform to fit the prior basis functions. In the latitude and longitude direction, the rationality of predicting parameters is discussed from two aspects: position extreme point and lateral maneuverability. In the height direction, for the NSHV oscillation drop characteristic, two Hough transforms are used to separate the periodic term from the linear term. In addition, we give the approximate interval of the parameter values of the method, and design a parameter adaptive scheme. Finally, trajectory prediction is performed on two different control patterns. The error of the prediction time of 100s is within 25km, and the results show the effectiveness of the proposed method.

To improve the human-machine cooperativity of a wearable lower limb exoskeleton, a gait recognition method based on surface electromyography (sEMG) was proposed. sEMG of rectus femoris, vastus medialis, vastus lateralis, semitendinosus and biceps femoris were acquired. Then, time domain, frequency domain, time-frequency domain and nonlinear features were extracted. The integrated value of electromyography, variance, root mean square and wavelength were selected as the time domain features and the frequency domain feature includes mean power frequency. Wavelet packet energy was selected as the time-frequency domain feature. Nonlinear features including approximate entropy, sample entropy and fuzzy entropy of sEMG were extracted. Classification accuracy of different feature matrices and different muscle groups were constructed and verified. The optimal multi-dimensional fusion feature matrix was determined. Introducing the Bayesian optimization algorithm, the Bayesian optimized Random Forest classification model was constructed to identify different gait phases. Comparing with Random Forest, the accuracy of the optimized Random Forest was improved by 5.89%. Applying Random Forest algorithm with Bayesian optimization to gait prediction based on sEMG, the followership and consistency of gait control in lower limb exoskeleton can be improved. This template explains and demonstrates how to prepare your camera-ready paper for Trans Tech Publications. The best is to read these instructions and follow the outline of this text.

Machine learning for autonomous vehicle's trajectory prediction: A comprehensive survey, challenges, and future research directions

Connected and Automated Vehicles (CAV) have been rapidly developed, which, inevitably, renders that human-driven and autonomous vehicles share the road. Thus, trajectory prediction is an important research topic, which helps each CAV to efficiently follow a Human-Driven Vehicle (HV). In a wider scope, trajectory prediction, also, helps to improve the throughput of traffic flow and enhance its stability. To realize the trajectory prediction of Connected and Automated Vehicles to Human-Driven Vehicles, a car-following model, which is based on trajectory data, was established. Adding deep neural networks and an Attention mechanism, this paper established a data-driven car-following model, based on CNN-BiLSTM-Attention for CAV, to predict trajectory, by referring to the modeling idea of the traditional car-following model. The trajectory data in the next-generation-simulation (NGSIM) datasets that match the car-following characteristics were selected. In addition, noise-reduction pre-processing of the trajectory data was performed, to make it match the actual car-following situation. Experiments, for selecting the optimal structure of the model and the method of trajectory prediction, were carried out. The data-driven car-following models, such as LSTM, GRU, and CNN-BiLSTM, were selected for comparative analysis of trajectory prediction. The results show that the CNN-BiLSTM-Attention model has the smallest MAE and MSE as well as the largest R2. The CNN-BiLSTM-Attention model has the highest accuracy in vehicle-trajectory prediction. The model can, effectively, realize vehicle-trajectory prediction and provide a theoretical basis for vehicle-trajectory-based velocity guidance of Human-Driven Vehicles. In the future, the model can, also, provide the theoretical basis for Connected and Automated Vehicles, to make car-following decisions in mixed traffic flow.

A novel fusion strategy for locomotion activity recognition based on multimodal signals

A recently designed gait phase detection (GPD) system, with the ability to detect all seven phases of gait in healthy adults, was modified for GPD in children with cerebral palsy (CP). A shank-attached gyroscope sent angular velocity to a rule-based algorithm in LabVIEW to identify the distinct characteristics of the signal. Seven typically developing children (TD) and five children with CP were asked to walk on treadmill at their self-selected speed while using this system. Using only shank angular velocity, all seven phases of gait (Loading Response, Mid-Stance, Terminal Stance, Pre-Swing, Initial Swing, Mid-Swing and Terminal Swing) were reliably detected in real time. System performance was validated against two established GPD methods: (1) force-sensing resistors (GPD-FSR) (for typically developing children) and (2) motion capture (GPD-MoCap) (for both typically developing children and children with CP). The system detected over 99% of the phases identified by GPD-FSR and GPD-MoCap. Absolute values of average gait phase onset detection deviations relative to GPD-MoCap were less than 100 ms for both TD children and children with CP. The newly designed system, with minimized sensor setup and low processing burden, is cosmetic and economical, making it a viable solution for real-time stand-alone and portable applications such as triggering functional electrical stimulation (FES) in rehabilitation systems. This paper verifies the applicability of the GPD system to identify specific gait events for triggering FES to enhance gait in children with CP.

Pedestrian trajectory prediction is fundamental to a wide range of scientific research work and industrial applications. Most of the current advanced trajectory prediction methods incorporate context information such as pedestrian neighbourhood, labelled static obstacles, and the background scene into the trajectory prediction process. In contrast to these methods which require rich contexts, the method in our paper focuses on predicting a pedestrian's future trajectory using his/her observed part of the trajectory only. Our method, which we refer to as LVTA, is a Location-Velocity-Temporal Attention LSTM model where two temporal attention mechanisms are applied to the hidden state vectors from the location and velocity LSTM layers. In addition, a location-velocity attention layer embedded inside a tweak module is used to improve the predicted location and velocity coordinates before they are passed to the next time step. Extensive experiments conducted on three large benchmark datasets and comparison with eleven existing trajectory prediction methods demonstrate that LVTA achieves competitive prediction performance. Specifically, LVTA attains 9.19 pixels Average Displacement Error (ADE) and 17.28 pixels Final Displacement Error (FDE) for the Central Station dataset, and 0.46 metres ADE and 0.92 metres FDE for the ETH&UCY datasets. Furthermore, evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Data-driven trajectory prediction with weather uncertainties: A Bayesian deep learning approach