- Research Article
23
- 10.1016/j.robot.2018.06.009
Robust RGB-D visual odometry based on edges and points
- Jun 26, 2018
- Robotics and Autonomous Systems
- Erliang Yao + 4 more +4
Robust RGB-D visual odometry based on edges and points
PurposeTo realize stable and precise localization in the dynamic environments, the authors propose a fast and robust visual odometry (VO) approach with a low-cost Inertial Measurement Unit (IMU) in this study.Design/methodology/approachThe proposed VO incorporates the direct method with the indirect method to track the features and to optimize the camera pose. It initializes the positions of tracked pixels with the IMU information. Besides, the tracked pixels are refined by minimizing the photometric errors. Due to the small convergence radius of the indirect method, the dynamic pixels are rejected. Subsequently, the camera pose is optimized by minimizing the reprojection errors. The frames with little dynamic information are selected to create keyframes. Finally, the local bundle adjustment is performed to refine the poses of the keyframes and the positions of 3-D points.FindingsThe proposed VO approach is evaluated experimentally in dynamic environments with various motion types, suggesting that the proposed approach achieves more accurate and stable location than the conventional approach. Moreover, the proposed VO approach works well in the environments with the motion blur.Originality/valueThe proposed approach fuses the indirect method and the direct method with the IMU information, which improves the localization in dynamic environments significantly.
Robust RGB-D visual odometry based on edges and points
Robust RGB-D visual odometry based on edges and points
Robust robot localization with visually adaptive consensus filters in dynamic corridor environments
Robust robot localization with visually adaptive consensus filters in dynamic corridor environments
Development of RGB-D simultaneous localization and mapping in dynamic environments based on motion removal and dense map reconstruction
The state-of-art simultaneous localization and mapping (SLAM) methods have been demonstrated of satisfactory results in static environments. However, most of the existing methods cannot be directly applied in dynamic environments to account for moving objects. To solve the practical problem of SLAM in dynamic environments, an RGB-D SLAM method is developed based on moving objects removal and dense map reconstruction. Specifically, deep learning-based instance segmentation is performed to obtain the semantic information which is furthermore combined with multiview geometry theory to detect possible moving objects. Kalman filter and feature fusion algorithms are further used for moving objects tracking to improve the detection accuracy. Then the camera pose is estimated using the static feature points after motion removal to realize reliable localization in dynamic environments. Out of concern for the infeasibility of navigation and perception with sparse feature maps, each-single-view of dense map is constructed using the RGB image and the depth map collected by the RGB-D camera after motion removal. The dense environmental map is finally reconstructed through the registration of point cloud of selected key image frames. Experiments are conducted on public datasets and real scene data to evaluate the performance of the proposed method. Compared with the state-of-art method ORB-SLAM2, the absolute trajectory error and the relative pose error obtained by the proposed method are significantly reduced by at least 91%, and the localization accuracy reaches 0.001 m.
Read moreDevelopment of an Online Adaptive Parameter Tuning vSLAM Algorithm for UAVs in GPS-Denied Environments
In recent years, unmanned aerial vehicles (UAVs) have been applied in many fields owing to their mature flight control technology and easy-to-operate characteristics. No doubt, these UAV-related applications rely heavily on location information provided by the positioning system. Most UAVs nowadays use a global navigation satellite system (GNSS) to obtain location information. However, this outside-in 3rd party positioning system is particularly susceptible to environmental interference and cannot be used in indoor environments, which limits the application diversity of UAVs. To deal with this problem, in this paper, a stereo-based visual simultaneous localization and mapping technology (vSLAM) is applied. The presented vSLAM algorithm fuses onboard inertial measurement unit (IMU) information to further solve the navigation problem in an unknown environment without the use of a GNSS signal and provides reliable localization information. The overall visual positioning system is based on the stereo parallel tracking and mapping architecture (S-PTAM). However, experiments found that the feature-matching threshold has a significant impact on positioning accuracy. Selection of the threshold is based on the Hamming distance without any physical meaning, which makes the threshold quite difficult to set manually. Therefore, this work develops an online adaptive matching threshold according to the keyframe poses. Experiments show that the developed adaptive matching threshold improves positioning accuracy. Since the attitude calculation of the IMU is carried out based on the Mahony complementary filter, the difference between the measured acceleration and the gravity is used as the metric to online tune the gain value dynamically, which can improve the accuracy of attitude estimation under aggressive motions. Moreover, a static state detection algorithm based on the moving window method and measured acceleration is proposed as well to accurately calculate the conversion mechanism between the vSLAM system and the IMU information; this initialization mechanism can help IMU provide a better initial guess for the bundle adjustment algorithm (BA) in the tracking thread. Finally, a performance evaluation of the proposed algorithm is conducted by the popular EuRoC dataset. All the experimental results show that the developed online adaptive parameter tuning algorithm can effectively improve the vSLAM accuracy and robustness.
Read moreToward a Better Robotic Hand Prosthesis Control: Using EMG and IMU Features for a Subject Independent Multi Joint Regression Model
Ahstract- The interest on wearable prosthetic devices has boost the research for a robust framework to help injured subjects to regain their lost functionality. A great number of solutions exploit physiological human signals, such as Electromyography (EMG), to naturally control the prosthesis, reproducing what happens in the human limbs. In this paper, we propose for the first time a way to integrate EMG signals with Inertial Measurement Unit (IMU) information, as a way to improve subject-independent models for controlling robotic hands. EMG data are very sensitive to both physical and physiological variations, and this is particularly true between different subjects. The introduction of IMUs aims at enriching the subject-independent model, making it more robust with information not strictly dependent from the physiological characteristics of the subject. We compare three different models: the first based on EMG solely, the second merging data from EMG and the 2 best IMUs available, and the third using EMG and IMUs information corresponding to the same 3 electrodes. The three techniques are tested on two different movements executed by 35 healthy subjects, by using a leave-one-out approach. The framework is able to estimate online the bending angles of the joints involved in the motion, obtaining an accuracy up to 0.8634. The resulting joint angles are used to actuate a robotic hand in a simulated environment.
Read moreA Mobile Robots PSO-based for Odor Source Localization in Dynamic Advection-Diffusion Environment
This paper presents a problem of odor source localization in a dynamic environment, which means the odor distribution is changing over time. Odor source localization is an interesting application in dynamic problems. Modified particle swarm optimization is a well-known algorithm, which can continuously track a changing optimum over time. PSO can be improved or adapted by incorporating the change detection and responding mechanisms for solving dynamic problems. Charged PSO which is another extension of the PSO has also been applied to solve dynamic problems. We adopted two types of modified concepts of PSO for a new algorithm in order to control autonomous vehicles in more realistic environment where a speed limitation of the robot behavior and collision avoidance mechanism should be taken into consideration as well as the effect of noise and threshold value for the odor sensor response, also positioning error of GPS sensor of robot. Simulations illustrate that the new approach can solve such dynamic problems in advection-diffusion odor model environment
Read moreJoint optimization based on direct sparse stereo visual-inertial odometry
This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.
Read moreResearch on SLAM Based on the Fusion of Stereo Vision and Inertial Measurement Unit
<div class="section abstract"><div class="htmlview paragraph">With the continuous improvement of positioning technology and industry demand, the shortcomings of each sensor are constantly amplified. Only relying on a single sensor, the demand of high-precision positioning and mapping for intelligent vehicles is difficult to be satisfied. The accuracy of system positioning and mapping is reduced due to the loss of feature points in pure visual SLAM as the environmental characteristics are not obvious or the texture is not abundant. IMU is a sensor with low cost and high update frequency, which can correct the running trajectory in real time and reduce the error of environmental factors on visual sensor data. Therefore, a method based on ORB_SLAM2 algorithm and VINS-Fusion algorithm, the stereo camera information and inertial measurement unit information are extracted and fused in robot operating system is proposed. ORB_SLAM2 algorithm is adopted as the main body and the feature point matching of ORB_SLAM2 algorithm is employed to enhance the mapping effect. While the tightly-coupled approach to visual information and inertial measurement unit information in VINS-Fusion algorithm is integrated into ORB_SLAM2 algorithm to achieve higher-precision synchronous positioning and mapping, and improve the robustness of the system. The tracked vehicle is tested in an appropriate scenario. On the basis of the testing results, it can be proved that the positioning and mapping effect of SLAM system fused with inertial measurement unit information is significantly improved compared with pure visual SLAM system. It can also be attested indirectly that the ORB_SLAM2 algorithm tightly-coupled with inertial measurement unit can effectively improve the global consistency of the localization trajectory, furthermore improve the accuracy and robustness of SLAM system.</div></div>
Read moreHeading Estimation for Agricultural Vehicles with Multi-antenna RTK/GNSS, Tactical-Grade and Low-Cost MEMS IMUs
Nowadays, Autonomous Vehicles (AVs) are employed for various tasks, including spraying, harvesting, and planting.For AVs to navigate autonomously, accurate heading knowledge of the vehicle is essential.Sensors such as the Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) are used on AVs to produce heading information.Dual-frequency and Real-Time Kinematic (RTK) capable systems with multiple antennas are used to increase the heading accuracy of GNSS.For IMUs, heading accuracy is directly related to the quality of the sensors, so high accuracy is achieved with expensive IMUs.However, with the development of micro-electro-mechanical system (MEMS) technology, studies are also being carried out on lowcost IMU solutions.This study tested the heading performances of three different sensors: a lowcost RTK/GNSS with multiple antennas, a tactical-grade IMU, and a low-cost MEMS IMU.An Unmanned Ground Agricultural Vehicle (UGAV) designed for spraying was driven on a line, and the sensors' data mounted on the UGAV were collected.Heading accuracy was also examined according to the distance between the RTK/GNSS system antennas.As a result of the analysis, the average errors of RTK/GNSS, tactical-grade IMU, and low-cost IMU are 0.58, 0.60, and 4.24 degrees, respectively.
Read moreTowards Constant-Time Robot Localization in Large Dynamic Environments
Global localization is the problem in which a mobile robot has to estimate the self-position with respect to an a priori map of landmarks using the perception and the motion measurements without using any knowledge of the initial self-position. Recently, random sample consensus (RANSAC), a robust multi-hypothesis estimator, has been successfully applied to offline global localization in static environments. However, online global localization in dynamic environments is still a difficult problem, due to incrementally arriving measurements as well as many outlier measurements. To realize a real time algorithm for such an online process, we have developed an incremental version of RANSAC algorithm by extending an efficient preemptive RANSAC scheme, in order to find inlier hypotheses of self-positions out of a large number of outlier hypotheses contaminated by the outlier measurements
Read moreEvent-based feature tracking in a visual inertial odometry framework
Introduction: Event cameras report pixel-wise brightness changes at high temporal resolutions, allowing for high speed tracking of features in visual inertial odometry (VIO) estimation, but require a paradigm shift, as common practices from the past decades using conventional cameras, such as feature detection and tracking, do not translate directly. One method for feature detection and tracking is the Eventbased Kanade-Lucas-Tomasi tracker (EKLT), an hybrid approach that combines frames with events to provide a high speed tracking of features. Despite the high temporal resolution of the events, the local nature of the registration of features imposes conservative limits to the camera motion speed.Methods: Our proposed approach expands on EKLT by relying on the concurrent use of the event-based feature tracker with a visual inertial odometry system performing pose estimation, leveraging frames, events and Inertial Measurement Unit (IMU) information to improve tracking. The problem of temporally combining high-rate IMU information with asynchronous event cameras is solved by means of an asynchronous probabilistic filter, in particular an Unscented Kalman Filter (UKF). The proposed method of feature tracking based on EKLT takes into account the state estimation of the pose estimator running in parallel and provides this information to the feature tracker, resulting in a synergy that can improve not only the feature tracking, but also the pose estimation. This approach can be seen as a feedback, where the state estimation of the filter is fed back into the tracker, which then produces visual information for the filter, creating a “closed loop”.Results: The method is tested on rotational motions only, and comparisons between a conventional (not event-based) approach and the proposed approach are made, using synthetic and real datasets. Results support that the use of events for the task improve performance.Discussion: To the best of our knowledge, this is the first work proposing the fusion of visual with inertial information using events cameras by means of an UKF, as well as the use of EKLT in the context of pose estimation. Furthermore, our closed loop approach proved to be an improvement over the base EKLT, resulting in better feature tracking and pose estimation. The inertial information, despite prone to drifting over time, allows keeping track of the features that would otherwise be lost. Then, feature tracking synergically helps estimating and minimizing the drift.
Read moreTowards effective localization in dynamic environments
Localization in dynamic environments is still a challenging problem in robotics - especially if rapid and large changes occur irregularly. Inspired by SLAM algorithms, our Bayesian approach to this so-called dynamic localization problem divides it into a localization problem and a mapping problem, respectively. To tackle the localization problem we use a particle filter, coupled with a distance filter and a scan matching method, which achieves a more robust localization against dynamic obstacles. For the mapping problem we use an extended sensor model which results in an effective and precise map update effect. We compare our approach against other localization methods and evaluate the impact the map update effect has on the localization in dynamic environments.
Read moreEFRNet-VL: An end-to-end feature refinement network for monocular visual localization in dynamic environments
EFRNet-VL: An end-to-end feature refinement network for monocular visual localization in dynamic environments
Inertial measurement unit-camera calibration based on incomplete inertial sensor information
This paper is concerned with the problem of estimating the relative orientation between an inertial measurement unit (IMU) and a camera. Unlike most existing IMU-camera calibrations, the main challenge in this paper is that the information output from the IMU is incomplete. For example, only two tilt information can be read from the gravity sensor of a smart phone. Despite incomplete inertial information, there are strong restrictions between the IMU and camera coordinate systems. This paper addresses the incomplete information based IMU-camera calibration problem by exploiting the intrinsic restrictions among the coordinate transformations. First, the IMU transformation between two poses is formulated with the unknown IMU information. Then the defective IMU information is restored using the complementary visual information. Finally, the Levenberg-Marquardt (LM) algorithm is applied to estimate the optimal calibration result in noisy environments. Experiments on both synthetic and real data show the validity and robustness of our algorithm.
Read moreDesign and implementation of an inertial navigation system for pedestrians based on a low-cost MEMS IMU
Inertial navigation systems for pedestrians are infrastructure-less and can achieve sub-meter accuracy in the short/medium period. However, when low-cost inertial measurement units (IMU) are employed for their implementation, they suffer from a slowly growing drift between the true pedestrian position and the corresponding estimated position. In this paper we illustrate a novel solution to mitigate such a drift by: a) using only accelerometer and gyroscope measurements (no magnetometers required); b) including the sensor error model parameters in the state vector of an extended Kalman filter; c) adopting a novel soft heuristic for foot stance detection and for zero-velocity updates. Experimental results evidence that our inertial-only navigation system can achieve similar or better performance with respect to pedestrian dead-reckoning systems presented in related studies, although the adopted IMU is less accurate than more expensive counterparts.
Read more