Study of Robot Localization and Control Based on Motion Capture in Indoor Environment

Robot odor source localization in indoor environments based on gradient adaptive extremum seeking search

Markerless motion capture (MMC) shows promise for examining human movement across many domains because of its nonintrusive nature and negligible per-subject setup time. However, published MMC systems typically require specific hardware. This validation study compared lower-body joint kinematics from Ergomechanic, a hardware-agnostic pose model-based MMC system, to an established marker-based motion capture (MBMC) system. Static trial data from eighteen people were used to register MMC keypoints within a widely used musculoskeletal model. The registered model was used to calculate joint kinematics for static pose, squatting, and walking trials. A novel perturbation analysis estimated the contributions to differences in MBMC and MMC approaches to measurement disparities. Very good (0.87-1.0) correlations between the systems were calculated for ankle, knee, and hip flexion-extension angles. Good (0.70-0.86) correlations were found for hip external-internal and abduction-adduction. Pelvis and lumbar spine angles had a wider range of correlation results (-0.06 to 0.95), likely due to the few MMC keypoints in these body regions. Relative contributions from the perturbation analysis were (i) 75% from variations in MMC data relative to MBMC; (ii) 8% because MMC keypoints (26) < MBMC markers (67); and (iii) 3% from differences in musculoskeletal model scaling. These results validate Ergomechanic for leg kinematics during standing, walking, and squatting. Further, they suggest system improvements for pelvis and torso kinematics and provide new insights into the sources of known differences between MMC and MBMC measurements.

We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply single-person capture approaches to the multi-person case. We therefore propose a combined image segmentation and tracking approach to overcome these difficulties. A new probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Thereafter, a single-person markerless motion and surface capture approach can be applied to each individual, either one-by-one or in parallel, even under strong occlusions. We demonstrate the performance of our approach on several challenging multi-person motions, including dance and martial arts, and also provide a reference dataset for multi-person motion capture with ground truth.

Marker-less motion capture has been extensively studied in recent years as a means of evaluating productivity, safety, and workplace design for manual operations on-site. These technologies are ideal for circumstances in which traditional motion capture systems are ineffective due to the need for a laboratory setting and movement-inhibiting markers or sensors. However, many marker-less motion capture systems rely on RGB-D sensors that have limited range and susceptibility to interference from sunlight and ferromagnetic radiation, making them unsuitable for modeling worker actions on construction sites. To address this issue, we propose a marker-less motion capture approach utilizing optical images and depth data obtained from stereo vision cameras. Multiple camera lenses and triangulation algorithms generate depths maps similar to those produced by RGB-D sensors, while still utilizing an optical recording process unhindered by potentially harsh construction site conditions. These data are adapted for existing kinematic modeling systems (i.e. iPiMocap Studio) for 3-D pose estimation. The experiments show that the proposed approach can provide data precision comparable to that of RGB-D-based systems with fewer operational constraints; thus, motion data can be collected where previously developed methods fail due to environmental or maneuverability restrictions. With the proposed approach, kinematic modeling of human movements can be carried out on construction sites without inhibiting the mobility of the recorded subject.

Spatiotemporal parameters can characterize the gait patterns of individuals, allowing assessment of their health status and detection of clinically meaningful changes in their gait. Video-based markerless motion capture is a user-friendly, inexpensive, and widely applicable technology that could reduce the barriers to measuring spatiotemporal gait parameters in clinical and more diverse settings. Two studies were performed to determine whether gait parameters measured using markerless motion capture demonstrate concurrent validity with those measured using marker-based motion capture and a pressure-sensitive gait mat. For the first study, thirty healthy young adults performed treadmill gait at self-selected speeds while marker-based motion capture and synchronized video data were recorded simultaneously. For the second study, twenty-five healthy young adults performed over-ground gait at self-selected speeds while footfalls were recorded using a gait mat and synchronized video data were recorded simultaneously. Kinematic heel-strike and toe-off gait events were used to identify the same gait cycles between systems. Nine spatiotemporal gait parameters were measured by each system and directly compared between systems. Measurements were compared using Bland-Altman methods, mean differences, Pearson correlation coefficients, and intraclass correlation coefficients. The results indicate that markerless measurements of spatiotemporal gait parameters have good to excellent agreement with marker-based motion capture and gait mat systems, except for stance time and double limb support time relative to both systems and stride width relative to the gait mat. These findings indicate that markerless motion capture can adequately measure spatiotemporal gait parameters of healthy young adults during treadmill and over-ground gait.

The resource-intensive nature of traditional marker-based motion capture systems limits opportunities for quantitative motion analysis. However, the advancement of markerless motion capture technology yields tremendous promise for accessible kinematic analyses beyond conventional research settings. This work compared the lower limb kinematics measured by a neural network-driven markerless motion capture system to those from a standard marker-based motion capture system during squat, hip hinge, and reverse lunge tasks. Fourteen adults performed three repetitions of each movement while being recorded simultaneously by two iPads and 17 infrared optical motion capture cameras. The mean peak cross-correlation values indicated high agreement for knee and hip flexion (&gt;0.95) and poor agreement for hip adduction, knee varus, and internal rotation (&lt;0.49) for all tasks; agreement for hip internal rotation and ankle flexion was task dependent (0.27–0.97). The average root mean square error indicated joint-specific offsets between systems, as values ranged from 6.43° to 12.32° for the knee, 11.25° to 17.35° for the hip, and 21.51° to 25.67° for the ankle. These findings suggest that, while the markerless system demonstrates the ability to capture gross motor patterns in the sagittal plane, further refinement of the underlying models is needed to improve the validity of the system’s measurements.

For indoor mobile robots, many localization systems based on wireless sensor network have been reported. Received signal strength indicator is often used for distance measurement. However, the value of received signal strength indicator always has large fluctuation because radio signal is easily influenced by environmental factors. This will bring adverse effect on the distance measurement and deteriorate the performance of robot localization. In this article, the measured data are dealt with weighted recursive filter, which can depress the measurement noise effectively. In the linearization procedure, the least square method often causes additional error because it seriously relies on anchor nodes. Therefore, a minimum residual localization algorithm based on particle swarm optimization is proposed for a mobile robot running in indoor environment. With continuous optimization and update of particle swarm, the position that gets the best solution of objective function can be adopted as the final estimated position. Experiment results show that the proposed algorithm, compared with traditional algorithms, can attain better localization accuracy and is closer to Cramer–Rao lower bound.

The purpose of this paper is to provide a new method for solving the problem of the 3D source localization in indoor environments with weak airflow by the experimental study on the multi-robot method. For this purpose, we developed a 3D MRO (mobile robot olfaction) system consisting of three mobile robots. The gas sensor carried with each robot has a moving range of 0.5 m to 1.5 m on the Z-axis. A total of 60 experiments were conducted to validate and compare the standard whale optimization algorithm (SWOA) and the standard particle swarm optimization (SPSO) methods in a 7.65 m×4.1 m experimental area with two source heights (0.75 m and 1.05 m). For two source heights, the success rate and localization step of the SPSO method are 56.7% (17/30) and 31.4 steps, respectively. In addition, the success rate and localization step of the SWOA method are 80% (24/30) and 32.9 steps, respectively. These results show that the SWOA method has strong application potential in indoor environments with weak airflow.

Accurate assessment of jumping is important for sports performance and rehabilitation. While laboratory-based motion capture is considered gold-standard, markerless motion capture (MMC) systems offer an accessible alternative for field and clinical settings. However, their validity and reliability vary, warranting a comprehensive synthesis of current literature. Six databases (CINAHL, Embase, MEDLINE, Scopus, SPORTDiscus, Web of Science) were searched following PRISMA for studies comparing MMC derived kinematics, kinetics or performance with gold-standard measures during vertical or horizontal jump tasks. Twenty studies met inclusion. The meta-analysis for sagittal-plane pooled root mean square error (RMSE) values were 5.3° for hip (95% CI 2.9–7.6°), 4.4° for knee (2.9–5.9°) and 4.9° for ankle (3.9–5.9°). Frontal-plane RMSE values were 3.0° for hip (2.5–3.5°) and 7.5° for ankle (4.1–10.9°). Pooled jump-height bias was −2.9 cm (−8.1 to 2.3 cm). Between-study heterogeneity ranged from low to substantial, with wider prediction intervals for sagittal hip, frontal ankle and jump height. This review suggests that current MMC systems can achieve moderate accuracy for lower limb jump biomechanics, but performance varies widely between systems and contexts. Practitioners should consider system-specific evidence to make informed decisions regarding the suitability of a given system for their needs.

Markerless human body motion capture promises to remove markers from capture studios, thus simplifying its diverse application fields, from life science to virtual reality. This comprehensive review examines recent advances in real-time markerless motion capture systems from 2020 to 2024, focusing on real-time multi-view, multi-person tracking solutions. Recent advancements, particularly driven by neural network-based pose estimation, have enabled real-time tracking with minimal latency, achieving at least 25 frames per second. Through systematic analysis, we evaluate these methods based on three key metrics: accuracy in pose reconstruction, end-to-end latency, and computational efficiency. Special attention is given to how architectural decisions impact system scalability regarding the number of camera viewpoints and tracked individuals. While current methods show promise for applications like sports analysis and virtual reality, challenges remain in achieving optimal performance across all metrics. Through systematic analysis of leading real-time pipelines, we identify key technical advances and persistent challenges. This synthesis provides critical insights for researchers and practitioners working to develop more robust markerless motion capture systems, while outlining important directions for future research.

Biomechanical analysis is a powerful tool in the evaluation of movement dysfunction in orthopaedic and neurologic populations. Three-dimensional (3D) motion capture systems are widely used, accurate systems, but are costly and not available in many clinical settings. The Microsoft Kinect™ has the potential to be used as an alternative low-cost motion analysis tool. The purpose of this study was to assess concurrent validity of the Kinect™ with Brekel Kinect software in comparison to Vicon Nexus during sagittal plane gait kinematics. Twenty healthy adults (nine male, 11 female) were tracked while walking and jogging at three velocities on a treadmill. Concurrent hip and knee peak flexion and extension and stride timing measurements were compared between Vicon and Kinect™. Although Kinect measurements were representative of normal gait, the Kinect™ generally under-estimated joint flexion and over-estimated extension. Kinect™ and Vicon hip angular displacement correlation was very low and error was large. Kinect™ knee measurements were somewhat better than hip, but were not consistent enough for clinical assessment. Correlation between Kinect™ and Vicon stride timing was high and error was fairly small. Variability in Kinect™ measurements was smallest at the slowest velocity. The Kinect™ has basic motion capture capabilities and with some minor adjustments will be an acceptable tool to measure stride timing, but sophisticated advances in software and hardware are necessary to improve Kinect™ sensitivity before it can be implemented for clinical use.

Global localization problem is one of the classical and important problems in mobile robot. In this paper, we present an approach to solve robot global localization in indoor environments with grid map. It combines Hough Scan Matching (HSM) and grid localization method to get the initial knowledge of robot's pose quickly. For pose tracking, a scan matching technique called Iterative Closest Point (ICP) is used to amend the robot motion model, this can drastically decreases the uncertainty about the robot's pose in prediction step. Then accurate proposal distribution taking into account recent observation is introduced into particle filters to recover the best estimate of robot trajectories, which seriously reduces number of particles for pose tracking. The proposed approach can globally localize mobile robot fast and accurately. Experiment results carried out with robot data in indoor environments demonstrates the effectiveness of the proposed approach.

Our work provides a fast approach-path-independent framework for the problem of place recognition and robot localization in indoor environments. The approach-path independence is achieved by using highly viewpoint-invariant 3D junction features extracted from stereo pairs of images; these are based on stereo reconstructions of the JUDOCA junctions extracted from the individual images of a stereo pair. The speed in place-recognition and robot-localization is achieved by using a novel cylindrical data structure - we refer to it as the Feature Cylinder - for representing either all of the 3D junction features found in a hallway system during the learning phase of the robot or a set of locale signatures derived from the data. For the case when all data is placed on the Feature Cylinder, we can use the 3D-POLY polynomial-time in a hypothesize-and-verify approach to place recognition. On the other hand, in the locale signature based approach, we can use the same data structure for constant-time place recognition.

Gait analysis using three-dimensional motion capture systems (3D motion capture) provides a combination of kinematic and kinetic measurements for quantifying and characterizing the motion and loads, respectively, of lower extremity joints during human movement. However, their high cost and limited accessibility impact their utility. Wearable inertial motion sensors offer a cost-effective alternative to measure simple temporospatial variables, but more complex kinematic variables require machine learning interfaces. We hypothesize that kinematic measures about the knee collected using motion capture can be replicated by coupling raw data collected from inertial measurement units (IMUs) to machine learning algorithms. Data from 40 healthy participants performing fixed walking, stair climbing, and sit-to-stand tasks were collected using both 3D motion capture and IMUs. Sequence to sequence convolutional neural networks were trained to map IMU data to three motion capture kinematic outputs: right knee angle, right knee angular velocity, and right hip angle. Model performance was assessed using mean absolute error. The convolutional neural network models exhibited high accuracy in replicating motion capture-derived kinematic variables. Mean absolute error values for right knee angle ranged from 4.30 ± 1.55 to 5.79 ± 2.93 degrees, for right knee angular velocity from 7.82 ± 3.01 to 22.16 ± 9.52 degrees per second, and for right hip angle from 4.82 ± 2.29 to 8.63 ± 4.73 degrees. Task-specific variations in accuracy were observed. The findings highlight the potential of leveraging raw data from wearable inertial sensors and machine learning algorithms to reproduce gait lab-quality kinematic data outside the laboratory settings for the study of knee function following joint injury, surgery, or the progression of joint disease.

This study evaluated if a lower-extremity exoskeleton affects error between marker-based ($M^{+}$) and markerless ($\mathbf{M}^{\boldsymbol{-}}$) motion capture in children with crouch gait. Two participants (P1: female, P2: male, ages 6 and 11 years old, cerebral palsy and spina bifida, both with assistive devices) walked with (Exo) and without (NoExo) an exoskeleton spanning the knee and ankle. Mean absolute difference (MAD) between $\mathrm{M}^{+}$ and $\mathrm{M}^{-}$ of gait cycle normalized hip, knee, and ankle angles were analyzed. Key outcomes of an ongoing clinical trial including measures of sagittal knee angle and gait speed were also compared. $\mathrm{M}^{-}$ correlated with $\mathrm{M}^{+}$ kinematics, but MAD ranged 1.57-17.0 deg. Except for sagittal (of P1) and frontal (of P2) hip angles, the Exo increased MAD (range 0.74-8.7 deg). $\mathrm{M}^{-}$ underestimated knee flexion at initial ground contact, peak knee extension in stance, and peak knee extension in swing. No significant difference in MAD (Exo vs NoExo) was observed except for total joint excursion which had distinct trends for P 1 and P 2. For peak knee extension, a primary endpoint of crouch gait severity, the Exo increased MAD beyond minimum clinically meaningful differences (5 deg). Gait speed was consistent, with MAD $<0.03 ~\mathrm{m} / \mathrm{s}$ for all observations (NoExo vs Exo $\mathbf{p = 0. 2 2}$). While $\mathbf{M}^{-}$ has potential to ease gait assessment, early results warrant caution for use with wearable devices, highlighting the need for kinematic estimation algorithms to accommodate diverse end users in clinical and research settings.