MVS‐SLAM: Enhanced multiview geometry for improved semantic RGBD SLAM in dynamic environment

In semantic network-based dynamic scene visual SLAM methods, the masks generated by semantic networks often over-cover dynamic objects, resulting in parts of the static background being mistakenly included in the masks. Furthermore, the system fails to identify passively moving potential dynamic objects outside the masks, which impacts the pose estimation effect. To address these challenges, this paper proposes a mask-filtering method that integrates image depth information. By utilizing the consistency of object depth, this approach effectively eliminates background regions mistakenly included in the mask. Subsequently, we developed a regional dynamic point rejection strategy and proposed an anomaly detection method based on an adaptive Gaussian model. By constructing an adaptive Gaussian model for dynamic points inside the mask, setting dynamic thresholds, and updating the model parameters, this method can detect potential dynamic points outside the mask. It effectively reduces the impact of potential dynamic objects on the system’s pose estimation. Finally, this paper presents a dynamic Gaussian SLAM system based on the ORB-SLAM3 framework, named DG-SLAM. This system is used for pose estimation and dense point cloud construction, improving localization accuracy in dynamic scenes. To validate the performance of the DG-SLAM system, we conducted tests on the TUM RGBD dataset and in real-world scenarios, comparing it with ORB-SLAM3, DynaSLAM, and SG-SLAM. Compared to ORB-SLAM3, DG-SLAM achieves an average improvement of 94.54% and a minimum improvement of 89.34% in positioning accuracy on the TUM and Bonn datasets. Experimental results show that DG-SLAM can effectively detect and eliminate the interference of potential dynamic feature points in the scene, achieving good localization accuracy and strong robustness in various dynamic environments.

Existing dynamic scene visual SLAM methods indiscriminately remove feature points on dynamic target bounding boxes when using object detection algorithms to detect prior targets, resulting in the incorrect removal of some valid static points and affecting the localization accuracy of the algorithm. This paper proposes a visual SDT-SLAM algorithm, which uses YOLOv11 to determine the prior target information of each frame of the image and designs a short-term dynamic object tracking algorithm to track targets and classify feature points in real time for different categories of bounding boxes. In the state estimation process, a depth difference estimation algorithm is used to recover some erroneously deleted static feature points, and an epipolar constraint algorithm is used to filter dynamic feature points from the scene in a fine-grained manner. This method precisely distinguishes the dynamic and static feature points in dynamic scenes. Compared with other SLAM systems, this system reasonably utilizes depth information to restore static feature points in dynamic scenes, and based on semantic information and epipolar constraints, it removes incorrect matching points, significantly preserving and fully utilizing the static feature points in dynamic scenes. This further improves the trajectory estimation accuracy of the SLAM algorithm in dynamic scenes. Without relying on computationally intensive segmentation methods, it achieves a new balance between real-time performance and accuracy. The SDT-SLAM algorithm was validated on the TUM and Bonn public datasets and in the real world. The results show that SDT-SLAM has advantages in speed and accuracy for localization in highly dynamic environments compared to other advanced algorithms.

This paper reviews the latest research progress of visual SLAM in dynamic scenes. The improved algorithms dependent on deep learning and multi-sensor fusion are discussed. First of all, the challenges of traditional visual SLAM in dynamic environments are stated, especially the interference of high dynamic objects on system feature extraction, pose estimation and mapping. Then, the application of several object detection technologies such as YOLOv5 and YOLOv7 in dynamic visual SLAM was analyzed in detail. These methods significantly improve the positioning precision and robustness of the system by identifying and eliminating dynamic feature points in the scene. In addition, this paper explores the visual SLAM system combining semantic segmentation and geometric constraint optimization, and finally introduces the multi-sensor SLAM method combining IMU data, point cloud data and visual information. Experimental results show that these improved algorithms have a significant improvement in positioning accuracy and system robustness compared with traditional visual SLAM algorithms under TUM and other dynamic data sets.

Simultaneous localization and map building (SLAM) is crucial in autonomous robot navigation. However, existing SLAM systems generally assume a static environment, which makes it difficult to cope with the interference caused by moving objects in dynamic scenes, affecting the system’s localization accuracy and robustness. To address this challenge, this paper proposes YKD-SLAM, a visual SLAM system for indoor dynamic environments, which is based on the ORB-SLAM2 framework and incorporates YOLOv8 target detection, RCF-KMeans (Region-ConstrainedFastK-Means), and epipolar geometric constraints to realize the accurate rejection of dynamic feature points and improve the localization performance in dynamic environments. YKD-SLAM first uses YOLOv8 to detect dynamic objects in the scene, generates a detection frame, optimizes the depth map through open operations, and performs multi-region segmentation of the region within the detection frame by combining RCF-KMeans. Subsequently, through the dynamic feature point rejection strategy based on epipolar geometric constraints, different regions in the detection frame are discriminated into dynamic and static regions, and the feature points in the dynamic region are rejected to improve the localization accuracy and robustness of the system in dynamic environments. The experimental results show that YKD-SLAM performs well in several dynamic scenes in the TUMRGB-D dataset. Compared with ORB-SLAM2, its ATE is reduced by 98.37%; compared with DynaSLAM, the system operation efficiency is improved by 95.35%. In addition, practical experiments conducted in indoor dynamic scenes further validate its potential in real applications.

In indoor dynamic scenes, traditional visual simultaneous localization and mapping (SLAM) algorithms based on RGB-D cameras incorrectly use dynamic features to estimate the poses of the cameras, and do not fully utilize the geometric information in the scenes, resulting in the low positioning accuracy and robustness of SLAM systems. To solve this problem, this paper proposes an RGB-D SLAM algorithm based on multiple geometric features and semantic segmentation. The core of our SLAM system is the proposed robust exclusion method of dynamic point and line features. The method consists of the following three steps: (a) identify potential dynamic point features using motion consistency checks; (b) obtain potential motion regions via semantic segmentation, and then determine dynamic regions by combining these with dynamic point features; (c) remove point and line features in dynamic regions. The exclusion method of dynamic point and line features can be easily integrated into RGB-D SLAM systems for improving the accuracy and robustness of SLAM systems in dynamic scenes. Experimental results on the Technische Universität München (TUM) dataset demonstrate that the proposed algorithm has better positioning accuracy and stability than the original dynamic semantic (DS-SLAM) algorithm in dynamic environments. The effectiveness of the proposed algorithm is verified by comparison with other classical visual SLAM algorithms. Better mapping performance is achieved by this proposed algorithm in actual indoor scenes.

Visual Simultaneous Localization and Mapping (VSLAM) has developed as the basic ability of robots in past few decades. There are a lot of open-sourced and impressive SLAM systems. However, the majority of the theories and approaches of SLAM systems at present are based on the static scene assumption, which is usually not practical in reality because moving objects are ubiquitous and inevitable under most circumstances. In this paper the DDL-SLAM (Dynamic Deep Learning SLAM) is proposed, a robust RGB-D SLAM system for dynamic scenarios that, based on ORB-SLAM2, adds the abilities of dynamic object segmentation and background inpainting. We are able to detect moving objects utilizing both semantic segmentation and multi-view geometry. Having a static scene map allows inpainting background of the frame which has been obscured by moving objects, therefore the localization accuracy is greatly improved in the dynamic environment. Experiment with a public RGB-D benchmark dataset, the results clarify that DDL-SLAM can significantly enhance the robustness and stability of the RGB-D SLAM system in the highly-dynamic environment.

Existing visual SLAM (vSLAM) systems fail to perform well in dynamic environments as they cannot effectively ignore moving objects during pose estimation and mapping. We propose a lightweight approach to improve the robustness of existing feature based RGB-D and stereo vSLAM by accurately removing dynamic outliers in the scene that contribute to failures in pose estimation and mapping. First, a novel motion state detection algorithm using the depth and feature flow information is presented to identify regions in the scene with high moving probability. This information is then fused with semantic cues via a probability framework to enable accurate and robust moving object extraction to retain the useful features for pose estimation and mapping. To reduce the computational complexity of extracting semantic information in every frame, we propose to extract semantics only on keyframes with significant changes in image content. Semantic propagation is used to compensate for the changes in the intermediate frames (i.e., non-keyframes). This is achieved by computing the dense transformation map using the available feature flow vectors. The proposed techniques can be integrated into existing vSLAM systems to increase their robustness in dynamic environments without incurring much computation cost. Our work highlights the importance of distinguishing between motion states of potential moving objects for vSLAM in highly dynamic environments. We provide extensive experimental results on four well-known RGB-D and stereo datasets to show that the proposed technique outperforms existing vSLAM methods in indoor and outdoor environments under various dynamic scenarios including crowded scenes. We also perform our experiments on a low-cost embedded platform, i.e., Jetson TX1, to demonstrate the computational efficiency of our method.

Mobile robots operating in dynamic environments are easily affected by moving objects, which results in significant discrepancies between the localization and mapping results provided by their SLAM systems and the actual conditions. To address this issue, this paper proposes a visual semantic SLAM approach for mobile robots in dynamic scenes. First, a lightweight instance segmentation network, Yolact++, is employed to segment and detect objects in the scene, removing feature points from the regions of the a priori dynamic objects. Furthermore, an LBP (Local Binary Pattern) algorithm that incorporates depth information is designed to prevent the misclassification of edge feature points from dynamic objects, thus preserving static features for the system's pose estimation. A series of simulation analyses validate the performance advantages of the proposed method.

The early applications of Visual Simultaneous Localization and Mapping (VSLAM) technology were primarily focused on static environments, relying on the static nature of the environment for map construction and localization. However, in practical applications, we often encounter various dynamic environments, such as city streets, where moving objects are present. These dynamic objects can make it challenging for robots to accurately understand their own position. This paper proposes a real-time localization and mapping method tailored for dynamic environments to effectively deal with the interference of moving objects in such settings. Firstly, depth images are clustered, and they are subdivided into sub-point clouds to obtain clearer local information. Secondly, when processing regular frames, we fully exploit the structural invariance of static sub-point clouds and their relative relationships. Among these, the concept of the sub-point cloud is introduced as novel idea in this paper. By utilizing the results computed based on sub-poses, we can effectively quantify the disparities between regular frames and reference frames. This enables us to accurately detect dynamic areas within the regular frames. Furthermore, by refining the dynamic areas of keyframes using historical observation data, the robustness of the system is further enhanced. We conducted comprehensive experimental evaluations on challenging dynamic sequences from the TUM dataset and compared our approach with state-of-the-art dynamic VSLAM systems. The experimental results demonstrate that our method significantly enhances the accuracy and robustness of pose estimation. Additionally, we validated the effectiveness of the system in dynamic environments through real-world scenario tests.

The development of traditional SLAM technology has gradually encountered bottlenecks in recent years. With deep learning, especially vision-based machine learning, a new and improved way of development has been found for SLAM named Semantic SLAM. The study of semantic SLAM has been a very hot topic in recent years. However, the related research is still in its infancy and is not systematic. In this paper, we will discuss the existing SLAM systems, semantic segmentation networks and semantic SLAM systems, introduce secondly the multiple effects of semantic network on localization, mapping and their applications, and finally put forward an idea of semantic Fusion.

In recent years, the utilization of visual SLAM with a camera as a sensor has become increasingly widespread, particularly in the context of rapidly developing artificial intelligence such as mobile robots, VR, and AR. This approach is favored due to its affordability, lightweight design, ability to capture comprehensive information, and other advantages. The traditional slam technology has achieved a very mature effect with the static scene as the assumption condition, and the classic representatives are LSD-SLAM and ORB-SLAM which will be introduced in the following. However, since dynamic scenarios are unavoidable in the real world, overcoming the influence of dynamic objects becomes a challenge if researchers want to move forward with more applications. At present, under the dynamic environment of the visual slam algorithm faces positioning problems of low accuracy and poor robustness. To address the challenges posed by dynamic objects in a scene, many researchers are incorporating deep learning techniques and exploring the use of reference semantic information to collaboratively resolve the issue. This paper reviews this and summarizes the development process and important algorithms.

To address the degradation of visual simultaneous localization and mapping (VSLAM) pose estimation accuracy caused by extensive moving objects and the resulting contamination of feature points in indoor dynamic environments, this study proposes a dynamic feature point rejection strategy that integrates a You Only Look Once (YOLO) family detector, YOLO11, with the Lucas–Kanade (LK) optical flow method within the ORB_SLAM3 framework. First, YOLO11 is employed to detect potentially dynamic targets such as pedestrians in the image, and ORB features extracted within the detection bounding boxes are labeled as high-risk dynamic points. Subsequently, based on pyramidal LK optical flow, the motion consistency of feature points is analyzed across multiple frames, and a secondary discrimination is performed on features located at the edges and in the vicinity of the detection boxes. In this way, feature points that conform to a static motion model are maximally preserved, thereby alleviating the excessive removal and misclassification of static points caused by oversized detection boxes. On this basis, a new map is reconstructed using the filtered static feature points, and the pose estimation process is further refined. Experiments on multiple indoor dynamic scenarios demonstrate that, compared with the original ORB_SLAM3, the proposed method effectively suppresses dynamic interference, enhances the stability of feature points, and significantly improves pose estimation accuracy and trajectory continuity, thus providing a valuable reference for robust VSLAM applications in dynamic environments.

Visual simultaneous localization and mapping (SLAM) algorithms in dynamic scenes will apply the moving feature points to the camera pose’s calculation, which will cause the continuous accumulation of errors. As a target-detection tool, mask R-CNN, which is often used in combination with the former, due to the limited training datasets, easily results in the semantic mask being incomplete and deformed, which will increase the error. In order to solve the above problems, we propose in this paper a visual SLAM algorithm based on an adaptive mask dilation strategy and the dynamic probability of the feature points, named MDP-SLAM. Firstly, we use the mask R-CNN target-detection algorithm to obtain the initial mask of the dynamic target. On this basis, an adaptive mask-dilation algorithm is used to obtain a mask that can completely cover the dynamic target and part of the surrounding scene. Then, we use the K-means clustering algorithm to segment the depth image information in the mask coverage area into absolute dynamic regions and relative dynamic regions. Combined with the epipolar constraint and the semantic constraint, the dynamic probability of the feature points is calculated, and then, the highly dynamic possible feature points are removed to solve an accurate final pose of the camera. Finally, the method is tested on the TUM RGB-D dataset. The results show that the MDP-SLAM algorithm proposed in this paper can effectively improve the accuracy of attitude estimation and has high accuracy and robustness in dynamic indoor scenes.

Visual simultaneous localization and mapping (VSLAM) is extensively utilized in the field of unmanned driving. Classical VSLAM typically postulates that the surroundings are stationary, and dynamic objects can seriously affect the accuracy of the system. However, detecting and tracking dynamic objects is beneficial for improving the autonomy and intelligence of robotics. In order to solve the problem of VSLAM distinguishing dynamic objects in dynamic scenes, a semantic VSLAM system based on the VDO-SLAM framework is proposed. The approach integrates semantic segmentation into the visual odometry and combines scene flow and geometric constraints to effectively reduce the negative impact of dynamic objects on system performance. Specifically, a transformer-based neural network architecture for optical flow estimation was employed, which effectively enhances the segmentation precision of foreground and background regions. And a geometric constraint was introduced to classify dynamic objects, which can reduction of erroneous segmentation of dynamic objects with effect. The data analysis results on the KITTI dataset demonstrate that our proposed algorithms can effectively enhance precision and stability of VSLAM system, reduce the tracking error of VSLAM in dynamic environments, and demonstrate good potential and application value.

Most existing vision‐based simultaneous localization and mapping systems and their variants still assume that the observation is absolutely static and cannot work well in dynamic environments. In this paper, we propose a direct geometrically constrained SLAM method based on target detection and depth image segmentation, named YGDD‐SLAM. The YGDD‐SLAM system can work robustly, accurately, and continuously in highly dynamic environments. The method first acquires static and potential dynamic feature points in the current frame through a target detection network. Then, dynamic targets are identified by combining the geometric change relationship between static and potential dynamic feature points between adjacent frames. To improve the accuracy of the dynamic judgment, the motion probability of the potential dynamic target in the past few frames is also used for judgment. Subsequently, the dynamic object regions at the pixel level are segmented out based on the double‐peak feature of the gray‐scale histogram of the dynamic target region in the depth image, which ultimately achieves the accurate deletion of all dynamic features points. Meanwhile, we validate YGDD‐SLAM on TUM data set and Bonn data set and prove that it significantly improves the localization accuracy and system stability in different types of dynamic environments.