RGB-D Inertial Odometry for a Resource-Restricted Robot in Dynamic Environments

Robust robot localization with visually adaptive consensus filters in dynamic corridor environments

Accurate localization and reliable mapping is essential for autonomous navigation of robots. As one of the core technologies for autonomous navigation, Simultaneous Localization and Mapping (SLAM) has attracted widespread attention in recent decades. Based on vision or LiDAR sensors, great efforts have been devoted to achieving real-time SLAM that can support a robot’s state estimation. However, most of the mature SLAM methods generally work under the assumption that the environment is static, while in dynamic environments they will yield degenerate performance or even fail. In this paper, first we quantitatively evaluate the performance of the state-of-the-art LiDAR-based SLAMs taking into account different pattens of moving objects in the environment. Through semi-physical simulation, we observed that the shape, size, and distribution of moving objects all can impact the performance of SLAM significantly, and obtained instructive investigation results by quantitative comparison between LOAM and LeGO-LOAM. Secondly, based on the above investigation, a novel approach named EMO to eliminating the moving objects for SLAM fusing LiDAR and mmW-radar is proposed, towards improving the accuracy and robustness of state estimation. The method fully uses the advantages of different characteristics of two sensors to realize the fusion of sensor information with two different resolutions. The moving objects can be efficiently detected based on Doppler effect by radar, accurately segmented and localized by LiDAR, then filtered out from the point clouds through data association and accurate synchronized in time and space. Finally, the point clouds representing the static environment are used as the input of SLAM. The proposed approach is evaluated through experiments using both semi-physical simulation and real-world datasets. The results demonstrate the effectiveness of the method at improving SLAM performance in accuracy (decrease by 30% at least in absolute position error) and robustness in dynamic environments.

The state-of-art simultaneous localization and mapping (SLAM) methods have been demonstrated of satisfactory results in static environments. However, most of the existing methods cannot be directly applied in dynamic environments to account for moving objects. To solve the practical problem of SLAM in dynamic environments, an RGB-D SLAM method is developed based on moving objects removal and dense map reconstruction. Specifically, deep learning-based instance segmentation is performed to obtain the semantic information which is furthermore combined with multiview geometry theory to detect possible moving objects. Kalman filter and feature fusion algorithms are further used for moving objects tracking to improve the detection accuracy. Then the camera pose is estimated using the static feature points after motion removal to realize reliable localization in dynamic environments. Out of concern for the infeasibility of navigation and perception with sparse feature maps, each-single-view of dense map is constructed using the RGB image and the depth map collected by the RGB-D camera after motion removal. The dense environmental map is finally reconstructed through the registration of point cloud of selected key image frames. Experiments are conducted on public datasets and real scene data to evaluate the performance of the proposed method. Compared with the state-of-art method ORB-SLAM2, the absolute trajectory error and the relative pose error obtained by the proposed method are significantly reduced by at least 91%, and the localization accuracy reaches 0.001 m.

Camera localization from monocular images has been a long-standing problem, but its robustness in dynamic environments is still not adequately addressed. Compared with classic geometric approaches, modern CNN-based methods (e.g. PoseNet) have manifested the reliability against illumination or viewpoint variations, but they still have the following limitations. First, foreground moving objects are not explicitly handled, which results in poor performance and instability in dynamic environments. Second, the output for each image is a point estimate without uncertainty quantification. In this paper, we propose a framework which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. The key idea is a prior guided dropout module coupled with a self-attention module which can guide CNNs to ignore foreground objects during both training and inference. Additionally, the dropout module enables the pose regressor to output multiple hypotheses from which the uncertainty of pose estimates can be quantified and leveraged in the following uncertainty-aware pose-graph optimization to improve the robustness further. We achieve an average accuracy of 9.98m/3.63° on RobotCar dataset, which outperforms the state-of-the-art method by 62.97%/47.08%. The source code of our implementation is available at https://github.com/zju3dv/RVL-dynamic.

This paper proposes an approach to enhance the robustness and accuracy of visual simultaneous localization and mapping (SLAM) for ground wheeled mobile robots in dynamic environments. The proposed method incorporates encoder measurements to establish optimization constraints in bundle adjustment. To further improve robustness, a geometric technique utilizing KMeans clustering with epipolar constraints and the SegNet [1] for semantic segmentation is employed to filter out features detected on moving objects. These modifications are integrated into the state-of-the-art SLAM system ORB-SLAM3 [2] and demonstrate superior accuracy and real-time performance compared to the baseline approach. The effectiveness of the proposed method is demonstrated through multiple OpenLoris and IROS Lifelong SLAM competition scenarios.

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment.

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.<;/p><;p><;i>Note to Practitioners<;/i>-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

This article focuses on achieving efficient and safe navigation for robots in dynamic and unpredictable environments. We propose a hierarchical path planning framework that integrates global path planning with local dynamic obstacle avoidance. This framework aims to quickly plan a collision-free, shortest, and safest path for the robot while adapting the navigation path according to uncertainties in the operating environment. Global path planning is conducted using the improved gray wolf optimization algorithm (IGWO), trajectory tracking is achieved through a pure pursuit control algorithm, and a dynamic switching mechanism based on deep reinforcement learning (DRL) significantly enhances the navigation performance of the robotic system. The effectiveness of this approach has been verified through simulations and experiments. Simulation results indicate that in global path planning, the IGWO algorithm achieves faster convergence compared to algorithms, such as GWO-MP and GWO-CS. The planned path lengths are reduced by approximately 4.01% and 2.27%, respectively, and the fitness values are decreased by 4.78% and 1.9%, demonstrating superior path planning performance. For local dynamic obstacle avoidance, both single-robot and multirobot systems successfully avoided obstacles in multiple experiments. Finally, physical experiments conducted in various complex scenarios show that both single-robot and multirobot systems can effectively execute global planning and respond to unexpected obstacles. These results further demonstrate the method’s wide applicability and robust performance across diverse complex environments.

This paper presents a collision-free trajectory planning algorithm for cable-driven robots (CDPRs) in dynamic environment. As the basis, the wrench feasible workspaces (WFW) of CDPRs are obtained. The tradition artificial potential field (APF) method can quickly plan a trajectory. However, APF is not time-efficient with dynamic obstacles. In order to solve this problem, a new scope of APF based on kinematic model is used to generate a trajectory in dynamic environment. Then, Bezier curve is employed to optimize the initial trajectory making motors run smoothly. To validate the proposed improved APF method, kinematic simulations are carried out. The results demonstrate the trajectory generated by improved APF can avoid obstacles in different motion states smoothly.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

Simultaneous localization and mapping (SLAM) for mobile robots has attracted research interests in the past two decades. Recent years, Rao-Blackwellized particle filter (RBPF) approach proved to be an effective means to estimate the full SLAM posterior. However, most of the SLAM practices are implemented in static environment. To navigate in dynamic environment, model based approaches have been implemented. The approaches require modeling, classification, and data association. Modeling and classification often require prior knowledge of the environment and are less robust in highly complex environment. To obviate modeling, classification and data association in dynamic environment, a solution framework based on Rao-Blackwellised genetic algorithmic filter (RBGAF) is extended for recovering the full SLAM posterior, using raw sensor measurements. The resultant extended RBGAF-SLAM recovers the negative sensor information during SLAM so that the moving objects can be efficiently identified in the particle maps. This approach permits the implementation of SLAM without prior knowledge of the environment and the static environment assumption. Simulations and experimental results obtained in an outdoor environment using a laser measurement system are presented to demonstrate the method's effectiveness.

Recently, embodied cognition for robotics has been discussed, and various types of artificial neural networks are applied for behavior learning of mobile robots in unknown and dynamic environments. In this research, the behavioral learning based on a spiking neural network to realize high adaptability of a mobile robot is proposed. The robot learn the forward relationship from sensory inputs to motor outputs as well as the predictive relationship from motor outputs to the sensory inputs. However, the behavioral leaning capability of the robot depends strongly on the network structure. Therefore, a VEGA to acquire the network structure suitable to the changing environment is applied. Finally, the effectiveness of the proposed method through experimental results on behavioral learning for collision avoidance and target tracing behavior is discussed.

With the popularity of service robots in dynamic environments such as hospitals and shopping malls, traditional path planning algorithms have problems such as inflection points of planning paths and difficulty in effectively avoiding dynamic obstacles. In response to these problems, this paper proposes an optimization algorithm based on A* and dynamic window method (DWA). First, in the global path planning, the number of turning points is reduced by deleting redundant turning points. Secondly, by establishing the motion model of the obstacle, the range of the dynamic obstacle is updated to the static environment, and the dynamic obstacle avoidance ability of the DWA algorithm is improved. Finally, the optimized A* algorithm is combined with the DWA algorithm to realize the path planning of the service robot in a dynamic environment. Experimental results show that the algorithm reduces the global path length without disturbing dynamic obstacles, the path is smoother, and the dynamic obstacle avoidance ability is more reliable.

Current visual simultaneous localization and mapping (SLAM) systems have demonstrated commendable efficacy in static environments. However, the presence of dynamic objects in real-world settings frequently leads to system discrepancies, significantly impairing the accuracy and robustness of SLAM systems. Conventional visual SLAM approaches typically utilize epipolar constraints to mitigate the impact of outliers; nevertheless, they encounter limitations when confronted with a substantial number of dynamic or planar moving objects. To tackle these issues, this paper introduces a novel dynamic visual SLAM system, termed DEG-SLAM. Initially, the system employs the YOLOv5 object detection network to identify dynamic objects, subsequently relaying the semantic information to the tracking module. During the tracking phase, both semantic information and epipolar constraints are leveraged to filter out dynamic feature points. To address the challenges posed by the malfunctioning of epipolar constraints in degenerate scenes, DEG-SLAM incorporates a degenerate constraint mechanism aimed at further eliminating dynamic feature points. Furthermore, a reprojection constraint has been introduced to enhance the filtering of absent dynamic feature points that lie outside the detection boxes. Experimental findings reveal that DEG-SLAM significantly improves accuracy and robustness when compared to traditional ORB-SLAM3 in dynamic environments. The performance benefits of DEG-SLAM are particularly evident in degenerate scenarios, thereby affirming its practicality and reliability in complex settings.

Navigating non-holonomic mobile robots in dynamic environments is challenging because it requires computing at each instant, the space of collision free velocities, characterized by a set of highly non-linear and non-convex inequalities. Moreover, uncertainty in obstacle trajectories further increases the complexity of the problem, as it now becomes imperative to relate the space of collision free velocities to a confidence measure. In this paper, we present a novel perspective towards analyzing and solving probabilistic collision avoidance constraints based on our previous works on non-linear time scaling. In particular, we have shown earlier that a time scaled version of collision cone constraints can be solved in closed form and thus can be used to efficiently characterize the space of collision free velocities. In the current proposed work, we present a probabilistic version of time scaled collision cone constraints obtained by representing obstacle states through generic probability distributions. We present a novel reformulation of the probabilistic constraints into a family of deterministic algebraic constraints. The solution space of each member of the family can be derived in closed form and at the same time, can also be related to the lower bound on confidence measure through Cantelli's inequality. Thus, the proposed work represents a significant improvement over the current state of the art frameworks where probabilistic collision avoidance constraints are solved through exhaustive sampling in the state-control space. We also present a cost metric which serves as the basis for the construction of the various collision avoidance maneuvers based on factors like deviation from the current path, acceleration/de-acceleration capability of the robot, confidence of collision avoidance etc. We very briefly explain how the current robot state can be connected to the solution space of safe velocities in smooth time optimal fashion. Finally, the validity of the proposed formulation is exhibited through extensive numerical simulation results.