Probabilistic Locomotion Mode Recognition with Wearable Sensors

Adaptive Bayesian inference system for recognition of walking activities and prediction of gait events using wearable sensors

Joint moment measurements represent an objective biomechemical parameter in joint health assessment. Inverse dynamics based on 3D motion capture data is the current 'gold standard' to estimate joint moments. Recently, machine learning combined with data measured by wearable technologies such electromyography (EMG), inertial measurement units (IMU), and electrogoniometers (GON) has been used to enable fast, easy, and low-cost measurements of joint moments. This study investigates the ability of various deep neural networks to predict lower limb joint moments merely from IMU sensors. The performance of five different deep neural networks (InceptionTimePlus, eXplainable convolutional neural network (XCM), XCMplus, Recurrent neural network (RNNplus), and Time Series Transformer (TSTPlus)) were tested to predict hip, knee, ankle, and subtalar moments using acceleration and gyroscope measurements of four IMU sensors at the trunk, thigh, shank, and foot. Multiple locomotion modes were considered including level-ground walking, treadmill walking, stair ascent, stair descent, ramp ascent, and ramp descent. We show that XCM can accurately predict lower limb joint moments using data of only four IMUs with RMSE of 0.046 ± 0.013 Nm/kg compared to 0.064 ± 0.003 Nm/kg on average for the other architectures. We found that hip, knee, and ankle joint moments predictions had a comparable RMSE with an average of 0.069 Nm/kg, while subtalar joint moments had the lowest RMSE of 0.033 Nm/kg. The real-time feedback that can be derived from the proposed method can be highly valuable for sports scientists and physiotherapists to gain insights into biomechanics, technique, and form to develop personalized training and rehabilitation programs.

This paper presents a new, simple, and efficient strategy for locomotion classification using a force myography (FMG) system developed in-house. The FMG signals were acquired from eight calibrated force sensors. These sensors were wrapped around the thigh and captured volumetric changes in thigh muscles during locomotion. The data were collected from eight able-bodied subjects and two unilateral transfemoral amputees while walking in different terrains, i.e., level walk, ramp ascent, ramp descent, stair ascent, and stair descent (SD). The repeatability of FMG signals within all the five locomotion modes was expressed in terms of variance ratio (VR). For all locomotion modes, except SD, the VR ranges from 0.1 to 0.3, showing high repeatability within the locomotion modes for able-bodied and amputee subjects. Furthermore, these locomotion modes are characterized by computationally simplified time-domain features of FMG signals and classified using linear discriminant analysis (LDA). The subject-dependent LDA model gave an average classification accuracy of 99.5% ( ${\text{SD}}\,= \,{\text{0.5}}$ ) and 96.1% ( ${\text{SD}}\,= \,{\text{1.9}}$ ) for able-bodied and amputee subjects, respectively. These results show the potential of the FMG system in a real-time classification of locomotion modes for lower-limb prosthetic control.

Detecting human movement intentions is fundamental to neural control of robotic exoskeletons, as it is essential for achieving seamless transitions between different locomotion modes. In this study, we enhanced a muscle synergy-inspired method of locomotion mode identification by fusing the electromyography data with two types of data from wearable sensors (inertial measurement units), namely linear acceleration and angular velocity. From the finite state machine perspective, the enhanced method was used to systematically identify 2 static modes, 7 dynamic modes, and 27 transitions among them. In addition to the five broadly studied modes (level ground walking, ramps ascent/descent, stairs ascent/descent), we identified the transition between different walking speeds and modes of ramp walking at different inclination angles. Seven combinations of sensor fusion were conducted, on experimental data from 8 able-bodied adult subjects, and their classification accuracy and prediction time were compared. Prediction based on a fusion of electromyography and gyroscope (angular velocity) data predicted transitions earlier and with higher accuracy. All transitions and modes were identified with a total average classification accuracy of 94.5% with fused sensor data. For nearly all transitions, we were able to predict the next locomotion mode 300-500ms prior to the step into that mode.

COVID-19 has affected daily life in unprecedented ways, with dramatic changes in mental health, sleep time and level of physical activity. These changes have been especially relevant in the elderly population, with important health-related consequences. In this work, two different sensor technologies were used to quantify the energy expenditure of ageing adults. To this end, a technological platform based on Raspberry Pi 4, as an elaboration unit, was designed and implemented. It integrates an ambient sensor node, a wearable sensor node and a coordinator node that uses the information provided by the two sensor technologies in a combined manner. Ambient and wearable sensors are used for the real-time recognition of four human postures (standing, sitting, bending and lying down), walking activity and for energy expenditure quantification. An important first aim of this work was to realize a platform with a high level of user acceptability. In fact, through the use of two unobtrusive sensors and a low-cost processing unit, the solution is easily accessible and usable in the domestic environment; moreover, it is versatile since it can be used by end-users who accept being monitored by a specific sensor. Another added value of the platform is the ability to abstract from sensing technologies, as the use of human posture and walking activity for energy expenditure quantification enables the integration of a wide set of devices, provided that they can reproduce the same set of features. The obtained results showed the ability of the proposed platform to automatically quantify energy expenditure, both with each sensing technology and with the combined version. Specifically, for posture and walking activity classification, an average accuracy of 93.8% and 93.3% was obtained, respectively, with the wearable and ambient sensor, whereas an improvement of approximately 4% was reached using data fusion. Consequently, the estimated energy expenditure quantification always had a relative error of less than 3.2% for each end-user involved in the experimentation stage, classifying the high level information (postures and walking activities) with the combined version of the platform, justifying the proposed overall architecture from a hardware and software point of view.

We describe the design and control of a wearable robotic device powered by pneumatic artificial muscle actuators for use in ankle–foot rehabilitation. The design is inspired by the biological musculoskeletal system of the human foot and lower leg, mimicking the morphology and the functionality of the biological muscle–tendon–ligament structure. A key feature of the device is its soft structure that provides active assistance without restricting natural degrees of freedom at the ankle joint. Four pneumatic artificial muscles assist dorsiflexion and plantarflexion as well as inversion and eversion. The prototype is also equipped with various embedded sensors for gait pattern analysis. For the subject tested, the prototype is capable of generating an ankle range of motion of 27° (14° dorsiflexion and 13° plantarflexion). The controllability of the system is experimentally demonstrated using a linear time-invariant (LTI) controller. The controller is found using an identified LTI model of the system, resulting from the interaction of the soft orthotic device with a human leg, and model-based classical control design techniques. The suitability of the proposed control strategy is demonstrated with several angle-reference following experiments.

Behavior-based user authentication has arisen as a viable method for strengthening cybersecurity in an age of pervasive wearable and mobile technologies. This research introduces an innovative approach for ongoing user authentication via behavioral biometrics obtained from wearable sensors. We present a hybrid deep learning network called SE-DeepConvNet, which integrates a squeeze-and-excitation (SE) method to proficiently simulate and authenticate user behavior characteristics. Our methodology utilizes data collected by wearable sensors, such as accelerometers, gyroscopes, and magnetometers, to obtain a thorough behavioral appearance. The suggested network design integrates convolutional neural networks for spatial feature extraction, while the SE blocks improve feature identification by flexibly recalibrating channel-wise feature responses. Experiments performed on two datasets, HMOG and USC-HAD, indicate the efficacy of our technique across different tasks. In the HMOG dataset, SE-DeepConvNet attains a minimal equal error rate (EER) of 0.38% and a maximum accuracy of 99.78% for the Read_Walk activity. Our model presents outstanding authentication (0% EER, 100% accuracy) for various walking activities in the USC-HAD dataset, encompassing intricate situations such as ascending and descending stairs. These findings markedly exceed existing deep learning techniques, demonstrating the promise of our technology for secure and inconspicuous continuous authentication in wearable devices. The suggested approach demonstrates the potential for use in individual device security, access management, and ongoing uniqueness verification in sensitive settings.

In recent years, intelligent wearable sensor devices have developed rapidly and can be seen everywhere in daily life. With the rapid development of electronic components and the continuous improvement of their performance, intelligent wearable intelligent products have gradually become possible and have shown explosive growth. In addition, intelligent wearable electronic devices have many advantages that traditional devices do not have. With the popularity of fitness wearable devices, intelligent wearable devices can also be used for real-time heart rate and dynamic electrocardiogram (ECG) monitoring during marathon sports. It can effectively prevent sudden death. During marathon training and other health services, it is very important to use intelligent wearable sensor devices to monitor heart function. This paper puts forward a heart function monitoring system for marathon training based on intelligent wearable sensor, expounds the origin of marathon sports and the importance of heart function monitoring for marathon athletes during training. This paper discusses the technology and construction method of heart rate monitoring system based on intelligent wearable sensor device. At the same time, relevant experiments are carried out to verify the relevant performance of the intelligent wearable sensor device in the algorithm. The results show that the R wave detection accuracy of wearable devices based on traditional algorithms is usually between 92% and 93%. The R wave detection accuracy of the intelligent wearable sensor device improved by the algorithm in this paper has been improved to more than 97%, and the R wave detection accuracy of the algorithm in this paper is much higher than that of the traditional algorithm. This also reflects the effectiveness of the intelligent wearable sensor device of the algorithm during the training of marathon athletes.

A wearable electrochemical sensor utilizing multifunctional hydrogel for antifouling ascorbic acid quantification in sweat.

To adapt to different locomotion modes or terrains, real-time human intents recognition is an essential skill to the control of lower-limb exoskeletons timely and precisely. In this paper, we propose a real-time on-board training and recognition method to identify locomotion-related activities for an active pelvis orthosis using two IMUs integrated into it. The designed on-board intent recognition system with a BPNN based algorithm realizes distinguish among six locomotion modes including standing, level ground walking, ramp ascending, ramp descending, stair ascending and stair descending, and deliver the recognition results for future control strategies. Experiments are conducted on one healthy subject including on-board training and online recognition parts. The overall recognition accuracy is 97.79% with the cost time of one recognition decision is about 0.9ms, which is sufficient short compared with the sample interval of 10ms. The experimental results validate the great performance of the proposed real-time on-board training and recognition method for future control of the lower-limb exoskeletons assisting in various locomotion modes or terrains.

Exo-Skeleton is a wearable robotic device which was emerged in later 1960s that has a multitude of applications ranging from weightlifting to wearer’s stability improvement. This paper makes a novel approach in reviewing the various exo-skeleton models that are available at the present. The idea of this paper is to study and compare the models in terms of medical applicability like gait rehabilitation, physiotherapy, human strength augmentation, various control strategies of exo-skeleton, ergonomic study, and need for exo-skeleton system for enhancing the life of humans. Since exo-skeletons are wearable devices, it requires precise controlling of the actuators and repeatability, and accuracy plays a vital role. The paper elucidated with a detailed analysis, reviewed and summarized the core essence of 300 research papers and patents in the field of exo-skeleton with the aforesaid aspects of application and drive system with their control methodologies from 2007 to 2022 and shown a trend with year-wise data from which it is clear that the future of industrial work will become a collaborative activity involving the exo-skeleton system with human labours. Human – robot interaction is vital and must for enabling an integration which can be achieved with the amelioration in the control and actuation techniques for precise control. With the furtherance in the technology the exo-skeletons can be purpose built that can be of rigid or flexible structures with active, passive, or quasi-passive controls based on the user needs. Joint handed operation of Exo-skeleton with the alternative treatment methodologies will yield a plethora of benefits in the near future in comparison with the present conventional treatment modes that could result in reduction of surgeries.

This paper proposes a hierarchical support vector machine recognition algorithm based on a finite state machine (FSM-HSVM) to accurately and reliably recognize the locomotion mode recognition of an exoskeleton robot. As input signals, this method utilizes the angle information of the hip joint and knee joint collected by inertial sensing units (IMUs) on the thighs and shanks of the exoskeleton and the plantar pressure information collected by force sensitive resistors (FSRs) are used as input signals. This method establishes a framework for mode transition by combining the finite state machine (FSM) with the common locomotion modes. The hierarchical support vector machine (HSVM) recognition model is then tightly integrated with the mode transition framework to recognize five typical locomotion modes and eight locomotion mode transitions in real-time. The algorithm not only reduces the abrupt change in the recognition of locomotion mode, but also significantly improves the recognition efficiency. To evaluate recognition performance, separate experiments are conducted on six subjects. According to the results, the average accuracy of all motion modes is 97.106% ± 0.955%, and the average recognition delay rate is only 25.017% ± 6.074%. This method has the benefits of a small calculation amount and high recognition efficiency, and it can be applied extensively in the field of robotics.

Flexible wearable sensors show great potential for applications in wearable devices, remote health monitoring, artificial intelligence, soft robotics, and artificial skin due to their stretchability, bendability, thinness and portability, and excellent electrical properties. Hydrogels have tunable mechanical properties, excellent biocompatibility, and flexibility, making them attractive candidates for wearable flexible sensors. Among them, tremendous efforts have focused on the advancement of chitosan-based hydrogels (CS-Gels) to realize multifunctional wearable sensing by modifying hydrogel networks with additives/nanofillers/functional groups. Recently, remarkable progress has been made in flexible wearable sensors. Herein, this review summarizes recent advances in CS-Gels wearable sensors for applications such as human motion monitoring, health monitoring, human-machine interface and soft robotics. Representative synthesis methods and strategies for CS-Gels are briefly described, the problems and deficiencies of CS-Gels for wearable sensors are discussed. Finally, the possible opportunities and challenges for the future development of CS-Gels flexible wearable devices are proposed.

Monitoring gait quality in daily activities through wearable sensors has the potential to improve medical assessment in Parkinson’s Disease (PD). In this study, four gait partitioning methods, two based on thresholds and two based on a machine learning approach, considering the four-phase model, were compared. The methods were tested on 26 PD patients, both in OFF and ON levodopa conditions, and 11 healthy subjects, during walking tasks. All subjects were equipped with inertial sensors placed on feet. Force resistive sensors were used to assess reference time sequence of gait phases. Goodness Index (G) was evaluated to assess accuracy in gait phases estimation. A novel synthetic index called Gait Phase Quality Index (GPQI) was proposed for gait quality assessment. Results revealed optimum performance (G < 0.25) for three tested methods and good performance (0.25 < G < 0.70) for one threshold method. The GPQI resulted significantly higher in PD patients than in healthy subjects, showing a moderate correlation with clinical scales score. Furthermore, in patients with severe gait impairment, GPQI was found higher in OFF than in ON state. Our results unveil the possibility of monitoring gait quality in PD through real-time gait partitioning based on wearable sensors.

Real-time, effective and affordable nutrition and dietetics wearable technology and sensors are emerging field with immense opportunities and benefits to the global nutrition challenge. As a powerful public health nutrition game changer, it requires more research and development support in tackling food security and nutrition safety challenges and evidence in local and global priorities needs worldwide. Such revolution real time, home, work and hospital-based rapid, accurate and cost-effective self-detection and diagnosis of direct or indirect causes or diet deficiency or excess are much needed for generating evidence-based information and knowledge for individual and vulnerable group nutritional and dietary mitigation and lifestyle adaptation through wearable sensors and technology. Importantly, they foster national decision making nutrition policy and guidelines, programs and interventions can best practices in self-management, healthier lifestyle and increasing life expectancy, productivity and wellness. However, urgent political commitment and financial investment is needed in building and sustaining dietary and nutrition wearable and implantable technologies and devices of research and development. These can enhance evidence-based, coherent and coordinated nutrition and dietary programs and strategies to a targeted group or illness, vital in addressing malnutrition and under-nutrition public health burden amongst African children. Moreover, enhancing balanced dietary and nutrition promotion and awareness, education and best practices culture in improving sustained lifestyle adaptations in people living with HIV (PLWHIV) and diabetes or overweight or stroke patients. Diseases, building wearable consumers’ health and fitness prognosis, prospective digital nutrition, dietetic data and database or nutrition informatics platforms. These provide a paradigm shift in engaging participatory communication amongst public consumers, dietetic and nutritionist professionals in improving quality interventions, management and outcomes. Furthermore, fostering and sustaining new ideas and solutions including the quest for food and nutrition enlightenment, knowledge and resilience. However, assessment and understanding of nutritional and dietary needs, and potential opportunities in functional health benefits and resource development in personalized accessibility and availability of needed resources to encourage positive behavior, diet and nutrition changes is needed in monitoring current and future healthy generations. Key words: Nutrition, dietetic, public health, wearable sensors, digital technology, Africa, low and middle income countries (LMICs).