Robust orientation estimate via inertial guided visual sample consensus

The aging population has become a burning issue for all modern societies around the world recently. There are two important issues existing now to be solved. One is how to continuously monitor the movements of those people having suffered a stroke in natural living environment for providing more valuable feedback to guide clinical interventions. The other one is how to guide those old people effectively when they are at home or inside other buildings and to make their life easier and convenient. Therefore, human motion tracking and navigation have been active research fields with the increasing number of elderly people. However, motion capture has been extremely challenging to go beyond laboratory environments and obtain accurate measurements of human physical activity especially in free-living environments, and navigation in free-living environments also poses some problems such as the denied GPS signal and the moving objects commonly presented in free-living environments. This thesis seeks to develop new technologies to enable accurate motion tracking and positioning in free-living environments. This thesis comprises three specific goals using our developed IMU board and the camera from the imaging source company: (1) to develop a robust and real-time orientation algorithm using only the measurements from IMU; (2) to develop a robust distance estimation in static free-living environments to estimate people’s position and navigate people in static free-living environments and simultaneously the scale ambiguity problem, usually appearing in the monocular camera tracking, is solved by integrating the data from the visual and inertial sensors; (3) in case of moving objects viewed by the camera existing in free-living environments, to firstly design a robust scene segmentation algorithm and then respectively estimate the motion of the vIMU system and moving objects. To achieve real-time orientation tracking, an Adaptive-Gain Orientation Filter (AGOF) is proposed in this thesis based on the basic theory of deterministic approach and frequency-based approach using only measurements from the newly developed MARG (Magnet, Angular Rate, and Gravity) sensors. To further obtain robust positioning, an adaptive frame-rate vision-aided IMU system is proposed to develop and implement fast vIMU ego-motion estimation algorithms, where the orientation is estimated in real time from MARG sensors in the first step and then used to estimate the position based on the data from visual and inertial sensors. In case of the moving objects viewed by the camera existing in free-living environments, a robust scene segmentation algorithm is firstly proposed to obtain position estimation

Multiple Model Adaptive Complementary Filter for Attitude Estimation

This study addresses challenges in sensor fusion for accurate and robust joint orientation estimation in human movement analysis using wearable inertial measurement units (IMUs). A magnetometer-free refined Kalman filter (KF) approach is presented and validated to address various indoor environmental constraints and challenges posed by human movement. These include variability in motion and dynamics, as well as magnetic disturbances caused by ferromagnetic materials or electronic interferences. Our proposed approach utilizes a Kalman-filter-based framework that analyzes the accelerometer’s alignment with the Earth’s frame to estimate orientation and correct gyroscope readings, eliminating reliance on magnetometer inputs. The algorithm was tested on both controlled robotic movements and real-world upper-limb-motion-monitoring scenarios. First, a comparative analysis was conducted on the double-stage Kalman filter (DSKF) and complementary filter using the collected robot motion encoder data. The results demonstrated superior performance in orientation estimation, particularly in yaw measurements, where the proposed method significantly improved accuracy. It achieved a lower root mean square error (RMSE = ) and mean absolute error (MAE = ), outperforming both the DSKF and complementary filter approaches. Additionally, the study’s findings were validated against a standard motion capture system, revealing error metrics within generally acceptable ranges ( of the joint range of motion [ROM]) and strong correlation coefficients (). However, some deviations were observed during complex motion cycle intervals, highlighting opportunities for further refinement. These findings suggest that the proposed approach presents a promising alternative for human joint orientation estimation in industrial settings with magnetic distortions.

A Drift-Free 3D Orientation and Displacement estimation method (DFOD) based on a single inertial measurement unit (IMU) is proposed and validated. Typically, body segment orientation and displacement methods rely on a constant- or zero-velocity point to correct for drift. Therefore, they are not easily applicable to more proximal segments than the foot. DFOD uses an alternative single sensor drift reduction strategy based on the quasi-cyclical nature of many human movements. DFOD assumes that the quasi-cyclical movement occurs in a quasi-2D plane and with an approximately constant cycle average velocity. DFOD is independent of a constant- or zero-velocity point, a biomechanical model, Kalman filtering or a magnetometer. DFOD reduces orientation drift by assuming a cyclical movement, and by defining a functional coordinate system with two functional axes. These axes are based on the mean acceleration and rotation axes over multiple complete gait cycles. Using this drift-free orientation estimate, the displacement of the sensor is computed by again assuming a cyclical movement. Drift in displacement is reduced by subtracting the mean value over five gait cycle from the free acceleration, velocity, and displacement. Estimated 3D sensor orientation and displacement for an IMU on the lower leg were validated with an optical motion capture system (OMCS) in four runners during constant velocity treadmill running. Root mean square errors for sensor orientation differences between DFOD and OMCS were 3.1 ± 0.4° (sagittal plane), 5.3 ± 1.1° (frontal plane), and 5.0 ± 2.1° (transversal plane). Sensor displacement differences had a root mean square error of 1.6 ± 0.2 cm (forward axis), 1.7 ± 0.6 cm (mediolateral axis), and 1.6 ± 0.2 cm (vertical axis). Hence, DFOD is a promising 3D drift-free orientation and displacement estimation method based on a single IMU in quasi-cyclical movements with many advantages over current methods.

This paper presents a new method for 3D position and orientation estimation based on the magnetic field generated by a single coil and the fusion of low power magnetometers and inertial sensors. The fixed spatial relationship between the sensors is exploited to solve the ambiguity problem that occurs when only a single coil is employed. In this context a new positioning algorithm is developed and verified by simulations. Using a single coil instead of three coils as common in conventional magnetic field based systems brings a benefit both regarding power consumption and construction size. Moreover, as the proposed system requires no line of sight between source and sensors, it is ideally suited for portable, location-independent applications such as the mobile assessment of a persons' head to torso posture and movement e.g. for the purpose of examining causes for chronic pain in the cervical spine area under conditions of daily life.

VEHICLE STATE ESTIMATION USING VISION AND INERTIAL MEASUREMENTS

Augmented reality (AR) technologies enhance users knowledge of their immediate surroundings by presenting contextualized and spatially relevant information. This augmentation is enabled by the automatic estimation of the device's pose with respect to the environment. Many simple AR applications for smartphones are well served by the pose estimate provided by GPS and inertial sensors, while other demand an higher accuracy which can be obtained with vision-based methods. The application presented in this paper, which aims to augment pictures of mountainous landscapes with geo-referenced data, belongs to the latter category. Our application is based on a novel approach for image-to-world registration which jointly relies on inertial and visual sensors. In a nutshell, first GPS and inertial sensors are used to compute a rough estimate of the device position and pose, then visual data are employed to refine it. Specifically, a learning-based edge detection algorithm is used to extract mountain profiles from the picture of interest. Then, a novel registration algorithm based on a robust optimization framework is employed to align the computed profiles to synthetic ones obtained from Digital Elevation Models. Our experiments, conducted on a novel dataset of manually aligned pictures which is made publicly available, demonstrate that the proposed registration method guarantees an increased accuracy with respect to competing methods and it is computationally efficient when implemented on a smartphone.

Local dominant orientation estimation is one of the most important operations in almost all automatic fingerprint authentication systems. Robust orientation and anisotropy estimation improves the system's reliability in handling low-quality fingerprints, which is crucial for the system's massive application such as securing multimedia. This paper analyzes the robustness of the orientation and anisotropy estimation methods and the effect of the modulus normalization on the estimation performance. A two-stage averaging framework with block-wise modulus handling is introduced to inherit the merits of the both linear and normalized averaging methods. We further propose to set the modulus of an orientation vector to be its anisotropy estimate instead of unity so that the orientation inconsistency of gradients is included in the second stage of averaging. These two measures improve the robustness of the fingerprint local dominant orientation estimation and lead to an anisotropy estimate that reflects the characteristics of fingerprint more effectively. In addition, the proposed approach is computationally efficient for online fingerprint authentication. Extensive experiments using both synthetic images and real fingerprints verify the feasibility of the proposed approach and demonstrate its robustness to noise and low-quality fingerprints.

Tracking of surgical instruments is an essential step towards the modernization of the surgical workflow by a comprehensive surgical landscape guidance system (COMPASS). Real-time tracking of a laparoscopic camera used in minimally-invasive surgery is required for applications in surgical workflow documentation, machine learning, image-localization, and intra-operative visualization. In our approach, an inertial measurement unit (IMU) assists the tool tracking in situations when no line-of-sight is available for infrared (IR) based tracking of the laparoscopic camera. The novelty of this approach lies in the localization method adjusted for the laparoscopic visceral surgery, particularly when the line-of-sight is lost. It is based on IMU tracking and the positioning of the trocar entry point. The trocar entry point is the remote center of motion (RCM), reducing degrees of freedom. We developed a method to tackle localization and a real-time tool for position and orientation estimation. The main error sources are given and evaluated in a test scenario. It reveals that for small changes in penetration length (e.g., pivoting), the IMU’s accuracy determines the error.

Mobile robots can be easily equipped with numerous sensors which can aid in the tasks of localization and ego-motion estimation. Two such examples are Inertial Measurement Units (IMU), which provide a gravity vector via pitch and roll angular velocities, and wide-angle or panoramic imaging devices. As the number of powerful devices on a single robot increases, an important problem arises in how to fuse the information coming from multiple sources to obtain an accurate and efficient motion estimate. The IMU provides real-time readings which can be employed in orientation estimation, while in principle an Omnidirectional camera provides enough information to estimate the full rigid motion (up to translational scale). However, in addition to being computationally overwhelming, such an estimation is traditionally based on the sensitive search for feature correspondences between image frames. In this paper we present a novel algorithm that exploits information from an IMU to reduce the five parameter motion search to a three-parameter estimation. For this task we formulate a generalized Hough transform which processes image features directly to avoid searching for correspondences. The Hough space is computed rapidly by re-treating the transform as a convolution of spherical images.

Consumer electronics mobile devices, such as smartphones or tablets, are rapidly growing in computing power and are equipped with an increasing number of sensors. This enables to use a present-day mobile device as a viable platform for computation-intensive, real-time applications in navigation and guidance. In this paper, we present a study on the performance of the orientation estimation based on the data acquired by the accelerometer, magnetometer, and gyroscope in a mobile device. Reliable orientation estimation based on the readouts from inertial sensors may be used in more complex systems, e.g., to correct the orientation error of a visual odometry system. We present a rigorous derivation of the mathematical estimation model, and we thoroughly evaluate the performance of the orientation estimation mechanism available in the Android OS, and the proposed alternative solutions on an unique dataset gathered using an actual smartphone. From the experimental results, we draw the conclusions as to the best performing algorithm, and then we evaluate its execution time on Android-based devices to demonstrate the possibility of real-time usage. The Android code for the proposed orientation estimation system is made publicly available for scientific and commercial applications.

Magnetic and inertial sensors have been widely used to estimate the orientation of human segments due to their low cost, compact size and light weight. However, the accuracy of the estimated orientation is easily affected by external factors, especially when the sensor is used in an environment with magnetic disturbances. In this paper, we propose an adaptive method to improve the accuracy of orientation estimations in the presence of magnetic disturbances. The method is based on existing gradient descent algorithms, and it is performed prior to sensor fusion algorithms. The proposed method includes stationary state detection and magnetic disturbance severity determination. The stationary state detection makes this method immune to magnetic disturbances in stationary state, while the magnetic disturbance severity determination helps to determine the credibility of magnetometer data under dynamic conditions, so as to mitigate the negative effect of the magnetic disturbances. The proposed method was validated through experiments performed on a customized three-axis instrumented gimbal with known orientations. The error of the proposed method and the original gradient descent algorithms were calculated and compared. Experimental results demonstrate that in stationary state, the proposed method is completely immune to magnetic disturbances, and in dynamic conditions, the error caused by magnetic disturbance is reduced by 51.2% compared with original MIMU gradient descent algorithm.

This paper presents a method to increase the reliability of Unmanned Aerial Vehicle (UAV) sensor Fault Detection and Identification (FDI) in a multi-UAV context. Differential Global Positioning System (DGPS) and inertial sensors are used for sensor FDI in each UAV. The method uses additional position estimations that augment individual UAV FDI system. These additional estimations are obtained using images from the same planar scene taken from two different UAVs. Since accuracy and noise level of the estimation depends on several factors, dynamic replanning of the multi-UAV team can be used to obtain a better estimation in case of faults caused by slow growing errors of absolute position estimation that cannot be detected by using local FDI in the UAVs. Experimental results with data from two real UAVs are also presented.

Visual and inertial sensors, in combination, are able to provide accurate motion estimates and are well suited for use in many robot navigation tasks. However, correct data fusion, and hence overall performance, depends on careful calibration of the rigid body transform between the sensors. Obtaining this calibration information is typically difficult and time-consuming, and normally requires additional equipment. In this paper we describe an algorithm, based on the unscented Kalman filter, for self-calibration of the transform between a camera and an inertial measurement unit (IMU). Our formulation rests on a differential geometric analysis of the observability of the camera—IMU system; this analysis shows that the sensor-to-sensor transform, the IMU gyroscope and accelerometer biases, the local gravity vector, and the metric scene structure can be recovered from camera and IMU measurements alone. While calibrating the transform we simultaneously localize the IMU and build a map of the surroundings, all without additional hardware or prior knowledge about the environment in which a robot is operating. We present results from simulation studies and from experiments with a monocular camera and a low-cost IMU, which demonstrate accurate estimation of both the calibration parameters and the local scene structure.

BackgroundA majority of employees in the industrial world spend most of their working time in a seated position. Monitoring sitting postures can provide insights into the underlying causes of occupational discomforts such as low back pain.ObjectiveThis study focuses on the technologies and algorithms used to classify sitting postures on a chair with respect to spine and limb movements, using sensors and wearables such as inertial measurement units, pressure or piezoresistive sensors, accelerometers or gyroscopes, combined with machine learning approaches.MethodsA total of three electronic literature databases were surveyed to identify studies classifying sitting postures in adults. Quality appraisal was performed to extract critical details and assess biases in the shortlisted papers.ResultsA total of 14 papers were shortlisted from 952 papers obtained after a systematic search. The majority of the studies used pressure sensors to measure sitting postures, whereas neural networks were the most frequently used approaches for classification tasks in this context. Only 2 studies were performed in a free-living environment. Most studies presented ethical and methodological shortcomings. Moreover, the findings indicate that the strategic placement of sensors can lead to better performance and lower costs.ConclusionsThe included studies differed in various aspects of design and analysis. The majority of studies were rated as medium quality according to our assessment. Our study suggests that future work for posture classification can benefit from using inertial measurement unit sensors, since they make it possible to differentiate among spine movements and similar postures, considering transitional movements between postures, and using three-dimensional cameras to annotate the data for ground truth. Finally, comparing such studies is challenging, as there are no standard definitions of sitting postures that could be used for classification. In addition, this study identifies five basic sitting postures along with different combinations of limb and spine movements to help guide future research efforts.