Dynamic SLAM with Real-Time Semantic Segmentation and Inter-Frame and Multi-Frame Motion Feature Detection

In indoor dynamic scenes, traditional visual simultaneous localization and mapping (SLAM) algorithms based on RGB-D cameras incorrectly use dynamic features to estimate the poses of the cameras, and do not fully utilize the geometric information in the scenes, resulting in the low positioning accuracy and robustness of SLAM systems. To solve this problem, this paper proposes an RGB-D SLAM algorithm based on multiple geometric features and semantic segmentation. The core of our SLAM system is the proposed robust exclusion method of dynamic point and line features. The method consists of the following three steps: (a) identify potential dynamic point features using motion consistency checks; (b) obtain potential motion regions via semantic segmentation, and then determine dynamic regions by combining these with dynamic point features; (c) remove point and line features in dynamic regions. The exclusion method of dynamic point and line features can be easily integrated into RGB-D SLAM systems for improving the accuracy and robustness of SLAM systems in dynamic scenes. Experimental results on the Technische Universität München (TUM) dataset demonstrate that the proposed algorithm has better positioning accuracy and stability than the original dynamic semantic (DS-SLAM) algorithm in dynamic environments. The effectiveness of the proposed algorithm is verified by comparison with other classical visual SLAM algorithms. Better mapping performance is achieved by this proposed algorithm in actual indoor scenes.

Purpose – To build a persistent map with visual landmarks is one of the most important steps for implementing the visual simultaneous localization and mapping (SLAM). The corner detector is a common method utilized to detect visual landmarks for constructing a map of the environment. However, due to the scale-variant characteristic of corner detection, extensive computational cost is needed to recover the scale and orientation of corner features in SLAM tasks. The purpose of this paper is to build the map using a local invariant feature detector, namely speeded-up robust features (SURF), to detect scale- and orientation-invariant features as well as provide a robust representation of visual landmarks for SLAM. Design/methodology/approach – SURF are scale- and orientation-invariant features which have higher repeatability than that obtained by other detection methods. Furthermore, SURF algorithms have better processing speed than other scale-invariant detection method. The procedures of detection, description and matching of regular SURF algorithms are modified in this paper in order to provide a robust representation of visual landmarks in SLAM. The sparse representation is also used to describe the environmental map and to reduce the computational complexity in state estimation using extended Kalman filter (EKF). Furthermore, the effective procedures of data association and map management for SURF features in SLAM are also designed to improve the accuracy of robot state estimation. Findings – Experimental works were carried out on an actual system with binocular vision sensors to prove the feasibility and effectiveness of the proposed algorithms. EKF SLAM with the modified SURF algorithms was applied in the experiments including the evaluation of accurate state estimation as well as the implementation of large-area SLAM. The performance of the modified SURF algorithms was compared with those obtained by regular SURF algorithms. The results show that the SURF with less-dimensional descriptors is the most suitable representation of visual landmarks. Meanwhile, the integrated system is successfully validated to fulfill the capabilities of visual SLAM system. Originality/value – The contribution of this paper is the novel approach to overcome the problem of recovering the scale and orientation of visual landmarks in SLAM tasks. This research also extends the usability of local invariant feature detectors in SLAM tasks by utilizing its robust representation of visual landmarks. Furthermore, data association and map management designed for SURF-based mapping in this paper also give another perspective for improving the robustness of SLAM systems.

Most existing Simultaneous Localization and Mapping (SLAM) systems rely on the assumption of static environments to achieve reliable and efficient mapping. However, such methods often suffer from degraded localization accuracy and mapping consistency in dynamic settings, as they lack explicit mechanisms to distinguish between static and dynamic elements. To overcome this limitation, we present BMP-SLAM, a vision-based SLAM approach that integrates semantic segmentation and Bayesian motion estimation to robustly handle dynamic indoor scenes. To enable real-time dynamic object detection, we integrate YOLOv5, a semantic segmentation network that identifies and localizes dynamic regions within the environment, into a dedicated dynamic target detection thread. Simultaneously, the data association Bayesian mobile probability proposed in this paper effectively eliminates dynamic feature points and successfully reduces the impact of dynamic targets in the environment on the SLAM system. To enhance complex indoor robotic navigation, the proposed system integrates semantic keyframe information with dynamic object detection outputs to reconstruct high-fidelity 3D point cloud maps of indoor environments. The evaluation conducted on the TUM RGB-D dataset indicates that the performance of BMP-SLAM is superior to that of ORB-SLAM3, with the trajectory tracking accuracy improved by 96.35%. Comparative evaluations demonstrate that the proposed system achieves superior performance in dynamic environments, exhibiting both lower trajectory drift and enhanced positioning precision relative to state-of-the-art dynamic SLAM methods.

Unmanned mining is one of the most effective methods to solve mine safety and low efficiency. However, it is the key to accurate localization and mapping for underground mining environment. A novel graph simultaneous localization and mapping (SLAM) optimization method is proposed, which is based on Generalized Iterative Closest Point (GICP) three-dimensional (3D) point cloud registration between consecutive frames, between consecutive key frames and between loop frames, and is constrained by roadway plane and loop. GICP-based 3D point cloud registration between consecutive frames and consecutive key frames is first combined to optimize laser odometer constraints without other sensors such as inertial measurement unit (IMU). According to the characteristics of the roadway, the innovative extraction of the roadway plane as the node constraint of pose graph SLAM, in addition to automatic removing the noise point cloud to further improve the consistency of the underground roadway map. A lightweight and efficient loop detection and optimization based on rules and GICP is designed. Finally, the proposed method was evaluated in four scenes (such as the underground mine laboratory), and compared with the existing 3D laser SLAM method (such as Lidar Odometry and Mapping (LOAM)). The results show that the algorithm could realize low drift localization and point cloud map construction. This method provides technical support for localization and navigation of underground mining environment.

Visual Simultaneous Localization and Mapping (SLAM) plays an important role in computer vision and robotic field. With the development of Convolutional Neural Network (CNN), most scholars currently fuse CNN with visual SLAM to reduce the impact of dynamic objects on visual SLAM. To address the impact of semantic segmentation networks with lower precision and quad-tree algorithm with over-uniform distribution of feature points on the location accuracy of SLAM, we proposed an STDC-SLAM: Short-Term Dense Concatenate Network SLAM, which was based on ORB-SLAM3. In the proposed system, a real-time STDC network was used for semantic thread to segment dynamic objects. On the one hand, we designed a segmentation refinement module to optimize the semantic segmentation maps using images depth information. On the other hand, we improved the Qtree-ORB algorithm by reducing the iterations of low-quality feature points in the rejection thread. We have evaluated our SLAM in public data sheets and compared it with ORB-SLAM3, DynaSLAM, PSPNet-SLAM. Experiments showed that our SLAM improved in localization accuracy compared to DynaSLAM and in processing speed compared to DynaSLAM and PSPNet-SLAM.

Simultaneous Localization and Mapping (SLAM) plays an important role in the computer vision and robotics field. The traditional SLAM framework adopts a strong static world assumption for analysis convenience. How to cope with dynamic environments is of vital importance and attracts more attentions. Existing SLAM systems toward dynamic scenes either solely utilize semantic information, solely utilize geometry information, or naively combine the results from them in a loosely coupled way. In this paper, we present SOF-SLAM: Semantic Optical Flow SLAM, a visual semantic SLAM system toward dynamic environments, which is built on RGB-D mode of ORB-SLAM2. A new dynamic features detection approach called semantic optical flow is proposed, which is a kind of tightly coupled way and can fully take advantage of feature's dynamic characteristic hidden in semantic and geometry information to remove dynamic features effectively and reasonably. The pixel-wise semantic segmentation results generated by SegNet serve as mask in the proposed semantic optical flow to get a reliable fundamental matrix, which is then used to filter out the truly dynamic features. Only the remaining static features are reserved in the tracking and optimization module to achieve accurate camera pose estimation in dynamic environments. Experiments on public TUM RGB-D dataset and in real-world environment are conducted. Compared with ORB-SLAM2, the proposed SOF-SLAM achieves averagely 96.73% improvements in high-dynamic scenarios. It also outperforms the other four state-of-the-art SLAM systems which cope with the dynamic environments.

Facing the realistic demands of the application environment of robots, the application of simultaneous localisation and mapping (SLAM) has gradually moved from static environments to complex dynamic environments, while traditional SLAM methods usually result in pose estimation deviations caused by errors in data association due to the interference of dynamic elements in the environment. This problem is effectively solved in the present study by proposing a SLAM approach based on light detection and ranging (LiDAR) under semantic constraints in dynamic environments. Four main modules are used for the projection of point cloud data, semantic segmentation, dynamic element screening, and semantic map construction. A LiDAR point cloud semantic segmentation network SANet based on a spatial attention mechanism is proposed, which significantly improves the real-time performance and accuracy of point cloud semantic segmentation. A dynamic element selection algorithm is designed and used with prior knowledge to significantly reduce the pose estimation deviations caused by SLAM dynamic elements. The results of experiments conducted on the public datasets SemanticKITTI, KITTI, and SemanticPOSS show that the accuracy and robustness of the proposed approach are significantly improved.

Localization and mapping in a dynamic scene is a crucial problem for the indoor visual simultaneous localization and mapping (SLAM) system. Most existed visual odometry (VO) or SLAM systems are based on the assumption that the environment is static. The performance of a SLAM system may degenerate when it is operated in a severely dynamic environment. The assumption limits the applications of RGB-D SLAM in the dynamic environment. In this paper, we propose a workflow to segment the objects accurately, which will be marked as the potentially dynamic-object area based on the semantic information. A novel approach for motion detection and removal from the moving camera is introduced. We integrate the semantics-based motion detection and the segmentation approach with an RGB-D SLAM system. To evaluate the effectiveness of the proposed approach, we conduct the experiments on the challenging dynamic sequences of TUM-RGBD datasets. The experimental results suggest that our approach improves the accuracy of localization and outperforms the state-of-the-art dynamic-removal-based SLAM system in both severely dynamic and slightly dynamic scenes.

• Design a self-attention aggregation module using a pre-trained diffusion model. • Integrate high-precision masks into RGB-D SLAM for robust dynamic tracking. • Validate superior accuracy and efficiency on dynamic simulation datasets. In dynamic environments, performing RGB-D SLAM (Simultaneous Localization and Mapping) faces significant challenges primarily due to the presence of moving objects. The motion of these objects can introduce tracking errors and inaccuracies in map construction, thereby compromising the stability and overall performance of the system. To maintain high-precision localization and mapping under such conditions, a SLAM system must effectively detect and handle dynamic objects. To address these challenges, this paper presents a novel RGB-D SLAM method, referred to as DMSAA-SLAM (Dynamic Scene SLAM Based on Diffusion Model Self-Attention Aggregation). The core idea is to leverage a pre-trained stable diffusion model, particularly its self-attention layers, to handle the complexity of dynamic scenes. By employing a multi-resolution aggregation approach, combined with iterative merging and nonmaximum suppression, the proposed method generates high-precision segmentation masks. These masks enable fine-grained segmentation of moving objects and effectively eliminate dynamic feature points, thereby mitigating the impact of dynamic elements on the SLAM process and ensuring efficient and accurate tracking and mapping.

Simultaneous Localization and Mapping (SLAM) technology can help mobile intelligent robots to understand and perceive scenes in unknown environments, so it is playing an increasingly important role in the fields of intelligent robots, intelligent cars, and so on. And because camera sensors have wide applicability, visual SLAM in dynamic scenes has become a relatively popular research direction in recent years. And the SLAM algorithm needs to be tested and validated, which requires choosing appropriate datasets based on different application scenarios. Therefore, this paper comprehensively introduces the excellent open-source data sets commonly used in the research of visual SLAM. Since deep learning networks are often used to improve the performance of the SLAM system, this paper summarizes the advanced techniques in recent years for solving the common problems of visual SLAM in dynamic scenes based on deep learning networks.<br />

A visual localization approach for dynamic objects based on hybrid semantic-geometry information is presented. Due to the interference of moving objects in the real environment, the traditional simultaneous localization and mapping (SLAM) system can be corrupted. To address this problem, we propose a method for static/dynamic image segmentation that leverages semantic and geometric modules, including optical flow residual clustering, epipolar constraint checks, semantic segmentation, and outlier elimination. We integrated the proposed approach into the state-of-the-art ORB-SLAM2 and evaluated its performance on both public datasets and a quadcopter platform. Experimental results demonstrated that the root-mean-square error of the absolute trajectory error improved, on average, by 93.63% in highly dynamic benchmarks when compared with ORB-SLAM2. Thus, the proposed method can improve the performance of state-of-the-art SLAM systems in challenging scenarios.

Traditional visual simultaneous localization and mapping (SLAM) systems mostly based on small-area static environments. In recent years, some studies focused on combining semantic information with visual SLAM. However, most of them are hard to obtain better performance in the large-scale dynamic environment. And the accuracy, rapidity of the system still needs to strengthen. In this paper, we develop a more efficient semantic SLAM system in the two-wheeled mobile robot by using semantic segmentation to recognize people, chairs, and other objects in every keyframe. With a preliminary understanding of the environment, fusing the RGB-D camera and encoders information, to localization and creating a dense colored octree map without dynamic objects. Besides, for the incomplete identification of movable objects, we used image processing algorithms to enhance the semantic segmentation effect. In the proposed method, enhanced semantic segmentation in keyframes dramatically increases the efficiency of the system. Moreover, fusing the different sensors can highly raise localization accuracy. We conducted experiments on various datasets and in some real environments and compared them with DRE-SLAM, DS-SLAM, to evaluate the performance of the proposed approach. The results suggest we significantly improve the processing efficiency, robustness, and quality of the map.

Typical visual simultaneous localization and mapping (SLAM) systems rely on purely geometric elements. Recently semantic SLAM has become popular due to its capability to obtain high-level understanding of the environment. However, most semantic SLAM systems merely aim to produce semantic maps rather than exploiting semantic constraints to boost the accuracy performance. In this paper, we propose a new RGB-D SLAM system for the use in indoor environments. Given that a ground plane represents a significant part of most indoor scenes, our new system exploit a semantic ground plane constraint inferred from a deep neural network along with geometric constraints on the state estimation to improve the performance. More specifically, we introduce a semantically meaningful geometric primitive, namely, a global ground plane to the optimization process of visual SLAM. The global ground model is able to provide an extra constraint on the state estimation through which both frame-to-keyframe and frame-to-model tracking processes are combined, and low-texture ground plane is effectively utilized. Experimental results for the TUM dataset and a real scene have demonstrated the improved performance brought by the semantic constraint in our method.

Visual simultaneous localization and mapping (SLAM) algorithms in dynamic scenes can incorrectly add moving feature points to the camera pose calculation, which leads to low accuracy and poor robustness of pose estimation. In this paper, we propose a visual SLAM algorithm based on object detection and static probability update strategy for dynamic scenes, named YKP-SLAM. Firstly, we use the YOLOv5 target detection algorithm and the improved K-means clustering algorithm to segment the image into static regions, suspicious static regions, and dynamic regions. Secondly, the static probability of feature points in each region is initialized and used as weights to solve for the initial camera pose. Then, we use the motion constraints and epipolar constraints to update the static probability of the feature points to solve the final pose of the camera. Finally, it is tested on the TUM RGB-D dataset. The results show that the YKP-SLAM algorithm proposed in this paper can effectively improve the pose estimation accuracy. Compared with the ORBSLAM2 algorithm, the absolute pose estimation accuracy is improved by 56.07% and 96.45% in low dynamic scenes and high dynamic scenes, respectively, and the best results are almost obtained compared with other advanced dynamic SLAM algorithms.

The multi-sensor fusion for a stochastic approach in the simultaneous localization and mapping (SLAM) should take into consideration the filtering performance against modeling error and required computational time. We propose a recursive particle filter based on the recursively marginalized Bayesian SLAM posterior considering geometric constraints. The recursively marginalized Bayesian SLAM posterior is based on approximating a traditional Bayesian SLAM as a product of the marginal probabilities related to estimates of the state and incorporates concurrently geometric constraints which keep geometric relationships between landmarks similar to correlations among landmarks. The proposed recursive particle filter based SLAM with geometric constraints allows a particle filter not only to directly update the state but also a larger number of particles to be incorporated for updating those particles may help the filtering performance. Furthermore, it can deal with any geometric constraints representing geometric relationships between landmarks which play an important role for stable map such as correlations among landmarks. The simulation result demonstrates a validity of the proposed method in estimation accuracy and efficiency.