Development of RGB-D simultaneous localization and mapping in dynamic environments based on motion removal and dense map reconstruction

Robust robot localization with visually adaptive consensus filters in dynamic corridor environments

The three-dimensional reconstruction of forest ecological environment by low-altitude remote sensing photography from Unmanned Aerial Vehicles (UAVs) provides a powerful basis for the fine surveying of forest resources and forest management. A stereo vision system, D-SLAM, is proposed to realize simultaneous localization and dense mapping for UAVs in complex forest ecological environments. The system takes binocular images as input and 3D dense maps as target outputs, while the 3D sparse maps and the camera poses can be obtained. The tracking thread utilizes temporal clue to match sparse map points for zero-drift localization. The relative motion amount and data association between frames are used as constraints for new keyframes selection, and the binocular image spatial clue compensation strategy is proposed to increase the robustness of the algorithm tracking. The dense mapping thread uses Linear Attention Network (LANet) to predict reliable disparity maps in ill-posed regions, which are transformed to depth maps for constructing dense point cloud maps. Evaluations of three datasets, EuRoC, KITTI and Forest, show that the proposed system can run at 30 ordinary frames and 3 keyframes per second with Forest, with a high localization accuracy of several centimeters for Root Mean Squared Absolute Trajectory Error (RMS ATE) on EuRoC and a Relative Root Mean Squared Error (RMSE) with two average values of 0.64 and 0.2 for trel and Rrel with KITTI, outperforming most mainstream models in terms of tracking accuracy and robustness. Moreover, the advantage of dense mapping compensates for the shortcomings of sparse mapping in most Smultaneous Localization and Mapping (SLAM) systems and the proposed system meets the requirements of real-time localization and dense mapping in the complex ecological environment of forests.

Current simultaneous localization and mapping (SLAM) algorithms perform well in static environments but easily fail in dynamic environments. Recent works introduce deep learning-based semantic information to SLAM systems to reduce the influence of dynamic objects. However, it is still challenging to apply a robust localization in dynamic environments for resource-restricted robots. This paper proposes a real-time RGB-D inertial odometry system for resource-restricted robots in dynamic environments named Dynamic-VINS. Three main threads run in parallel: object detection, feature tracking, and state optimization. The proposed Dynamic-VINS combines object detection and depth information for dynamic feature recognition and achieves performance comparable to semantic segmentation. Dynamic-VINS adopts grid-based feature detection and proposes a fast and efficient method to extract high-quality FAST feature points. IMU is applied to predict motion for feature tracking and moving consistency check. The proposed method is evaluated on both public datasets and real-world applications and shows competitive localization accuracy and robustness in dynamic environments. Yet, to the best of our knowledge, it is the best-performance real-time RGB-D inertial odometry for resource-restricted platforms in dynamic environments for now. The proposed system is open source at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/HITSZ-NRSL/Dynamic-VINS.git</uri>

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.<;/p><;p><;i>Note to Practitioners<;/i>-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

In the past, there has been a tremendous amount of progress in the area of autonomous robot navigation, and a large variety of robots have been developed that demonstrated robust navigation capabilities indoors, in nonurban outdoor environments, or on roads; relatively few approaches have focused on navigation in urban environments such as city centers. Urban areas, however, introduce numerous challenges for autonomous robots as they are rather unstructured and dynamic. In this paper, we present a navigation system for mobile robots designed to operate in crowded city environments and pedestrian zones. We describe the different components of this system, including a simultaneous localization and mapping module for dealing with huge maps of city centers, a planning component for inferring feasible paths, taking into account the traversability and type of terrain, a module for accurate localization in dynamic environments, and the means for calibrating and monitoring the platform. Our navigation system has been implemented and tested in several large‐scale field tests, in which a real robot autonomously navigated over several kilometers in a complex urban environment. This also included a public demonstration, during which the robot autonomously traveled along a more than 3‐km‐long route through the city center of Freiburg, Germany.

Camera localization from monocular images has been a long-standing problem, but its robustness in dynamic environments is still not adequately addressed. Compared with classic geometric approaches, modern CNN-based methods (e.g. PoseNet) have manifested the reliability against illumination or viewpoint variations, but they still have the following limitations. First, foreground moving objects are not explicitly handled, which results in poor performance and instability in dynamic environments. Second, the output for each image is a point estimate without uncertainty quantification. In this paper, we propose a framework which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. The key idea is a prior guided dropout module coupled with a self-attention module which can guide CNNs to ignore foreground objects during both training and inference. Additionally, the dropout module enables the pose regressor to output multiple hypotheses from which the uncertainty of pose estimates can be quantified and leveraged in the following uncertainty-aware pose-graph optimization to improve the robustness further. We achieve an average accuracy of 9.98m/3.63° on RobotCar dataset, which outperforms the state-of-the-art method by 62.97%/47.08%. The source code of our implementation is available at https://github.com/zju3dv/RVL-dynamic.

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

We propose an on-line algorithm for simultaneous localization and mapping of dynamic environments. Our algorithm is capable of differentiating static and dynamic parts of the environment and representing them appropriately on the map. Our approach is based on maintaining two occupancy grids. One grid models the static parts of the environment, and the other models the dynamic parts of the environment. The union of the two provides a complete description of the environment over time. We also maintain a third map containing information about static landmarks detected in the environment. These landmarks provide the robot with localization. Results in simulation and with physical robots show the efficiency of our approach and show how the differentiation of dynamic and static entities in the environment and SLAM can be mutually beneficial.

Indoor localization service is an indispensable part of modern intelligent life, among which Wi-Fi based fingerprint localization system is popular in indoor positioning researches due to its advantages of low cost and widely deployment. However, Wi-Fi based localization system is susceptible to dynamic environment, and fingerprint collection and updating are time-consuming and labor-intensive. To address this problem, we propose a novel positioning framework based on multiple transfer learning fusion using Generalized Policy Iteration (GPI). Firstly, a 1-Dimension Convolutional Autoencoder (1-D CAE) is designed to extract features from one-dimensional fingerprint data; similar to Convolutional Neural Network (CNN), it can not only pay more attention to the information of different dimensions of fingerprints, but also compress redundant information and reduce noise. After that, Domain Adversarial Neural Network (DANN) and Passive Aggressive (PA) algorithm are fused to train localization model based on unlabeled fingerprint of target domain using the theory of GPI in offline stage. Finally, the model is fine-tuned with unlabeled fingerprints and few labeled fingerprints in daily online predictions to improve the performance of the localization system. Various evaluations in five typical scenarios validate the effectiveness of proposed algorithm in dynamic environment, with low tendency, easy recalibration, long-term stabilization high accuracy and so on.

With the rapid advancement of robotics and artificial intelligence, embodied intelligent agents operating in dynamic and unstructured environments have emerged as a key technological direction within digital twin systems. Nevertheless, their exploration capabilities in indoor digital-twin scenes remain limited. Dense semantic maps incur substantial maintenance costs and exhibit poor adaptability to dynamic environments. Furthermore, current navigation processes lack mechanisms for assessing and revisiting semantic relevance, leading to redundant exploration and inefficient resource utilization. Finally, environmental changes often result in semantic anchor degradation, with existing methods lacking effective incremental update and self-evolution mechanisms to maintain long-term consistency. To address these challenges, this paper proposes a multimodal, lightweight mapping, and revisitable self-evolving exploration agent (LRSA) for indoor digital-twin scenarios. The framework consists of three key modules: a multi-scale structured environment modeling process based on sparse semantic mapping, a large-model-driven memory-intelligence system that maintains an “area-object-feature” relational network, and a perceptual-memory self-evolution mechanism that marks inactive objects upon navigation failure and uses a memory large language model (MemoLLM) to generate and iteratively refine candidate search strategies for dynamic memory updating. Experiments conducted on the HM3D indoor dataset and the TurtleBot3 platform demonstrate that LRSA achieves a Success Rate (SR) of 55.2% and a Success weighted by Path Length (SPL) of 30.5%, surpassing state-of-the-art baselines such as ZSON, PSL, and PixNav. The proposed framework eliminates reliance on dense maps, enabling lightweight sparse modeling with reduced computational cost, synergistic memory-intelligence semantic reasoning, and self-adaptive evolution in dynamic environments. Its contributions enhance the robustness and efficiency of embodied agents’ open-world navigation and provide real-time, long-term semantic updating for urban digital twin systems, supporting autonomous “self-perception, self-evolution, and self-maintenance.”

Solid-state light detection and ranging (LiDAR) has the advantages of low cost, small size and strong practicability. However, it faces challenges in simultaneous localization and mapping applications due to its small field of view and irregular scanning patterns. A solid-state LiDAR simultaneous localization and mapping system containing intensity information is proposed. In order to solve the irregular scanning characteristics of solid-state LiDAR, we introduce a data preprocessing framework and add intensity feature points to the front-end odometer. This improves the accuracy and robustness of positioning when geometric feature points are scarce, thus solving the problem of feature point degradation caused by a finite field of view. In the back-end optimization stage, we combine the geometric feature residuals with the intensity feature residuals through the proposed consistent difference function, so that the system can maintain good performance even in challenging environments. Finally, we conducted an extensive evaluation of the proposed algorithm on official datasets and various datasets collected from multiple platforms, and the results confirmed the validity of our approach. Compared with the corresponding method, in indoor scenes, the absolute trajectory error and relative attitude error are decreased by 54.5% and 5.3%. In outdoor scenes, the absolute trajectory error and relative attitude error are decreased by 29.6% and 58.8%.

Autonomous vehicles operating in real-world industrial environments have to overcome numerous challenges, chief among which is the creation and maintenance of consistent 3D world models. This paper focuses on a particularly important challenge: mapping in dynamic environments. We introduce several improvements to the recently proposed Normal Distributions Transform Occupancy Map (NDT-OM) aimed for efficient mapping in dynamic environments. A careful consistency analysis is given based on convergence and similarity metrics specifically designed for evaluation of NDT maps in dynamic environments. We show that in the context of mapping with known poses the proposed method results in improved consistency and in superior runtime performance, when compared against 3D occupancy grids at the same size and resolution. Additionally, we demonstrate that NDT-OM features real-time performance in a highly dynamic 3D mapping and tracking scenario with centimeter accuracy over a 1.5km trajectory.

With the advancement of intelligent perception in multirobot systems, cooperative localization in dynamic environments has become a critical component. However, existing multirobot cooperative localization systems still fall short in meeting high-precision localization requirements in Global Navigation Satellite System (GNSS)-denied environments. In this paper, we propose a factor-graph-based resilient cooperative localization (FG-RCL) algorithm for multirobot systems. This algorithm integrates measurements from visual sensors and Ultra-WideBand (UWB) to achieve accurate cooperative state estimation—overcoming the visibility issues of visual sensors within limited fields of view. We utilize the Joint Probabilistic Data Association (JPDA) algorithm to calculate the corresponding probabilities of multiple visual detection measurements between robots and assign them to their respective edges in the factor graph, thereby addressing the data association challenges in visual detection measurements. Finally, simulation results demonstrate that the proposed algorithm significantly reduces the influence of visual detection measurement interference on the performance of cooperative localization. Experimental results indicate that the proposed algorithm outperforms UWB-based and vision-based methods in terms of localization accuracy. The system is implemented using a factor-graph-based optimization approach, and it exhibits scalability and enables plug-and-play for sensors. Furthermore, it demonstrates resilience in abnormal situations.

Accurate localization in dynamic environments is a critical challenge for autonomous systems, particularly in GPS-denied settings and under sensor noise or failure conditions. This review paper explores state-of-the-art sensor fusion and localization techniques, including Kalman filters, particle filters, and machine learning-based approaches. The paper identifies key challenges such as operating in GPS-denied environments, managing sensor noise and failure, and ensuring scalability and real-time processing in complex scenarios. To address these issues, the paper proposes enhanced sensor fusion methods, advanced localization algorithms, and hybrid approaches that integrate traditional techniques with machine learning. These innovations are designed to improve autonomous systems' accuracy, reliability, and adaptability in increasingly complex and unpredictable environments. The paper also outlines validation strategies to ensure the effectiveness of these new methodologies, paving the way for future advancements in the field.