YKD-SLAM: a visual SLAM system in dynamic environments based on object detection and region segmentation

Visual simultaneous localization and mapping (VSLAM) is extensively utilized in the field of unmanned driving. Classical VSLAM typically postulates that the surroundings are stationary, and dynamic objects can seriously affect the accuracy of the system. However, detecting and tracking dynamic objects is beneficial for improving the autonomy and intelligence of robotics. In order to solve the problem of VSLAM distinguishing dynamic objects in dynamic scenes, a semantic VSLAM system based on the VDO-SLAM framework is proposed. The approach integrates semantic segmentation into the visual odometry and combines scene flow and geometric constraints to effectively reduce the negative impact of dynamic objects on system performance. Specifically, a transformer-based neural network architecture for optical flow estimation was employed, which effectively enhances the segmentation precision of foreground and background regions. And a geometric constraint was introduced to classify dynamic objects, which can reduction of erroneous segmentation of dynamic objects with effect. The data analysis results on the KITTI dataset demonstrate that our proposed algorithms can effectively enhance precision and stability of VSLAM system, reduce the tracking error of VSLAM in dynamic environments, and demonstrate good potential and application value.

Nowadays, moving object detection in sequence images has become a hot topic in computer vision research, and has a very wide range of practical applications in many fields of military and daily life. In this paper, fast detection of moving objects in complex background is studied, and fast detection methods for moving objects in static and dynamic scenes are proposed respectively. Firstly, based on image preprocessing, aiming at the difficulty of feature extraction of moving targets in low illumination at night, Gamma change is used to process. Secondly, for the fast detection of moving objects in static scenes, this paper designs a detection method combining background difference and edge frame difference. Finally, aiming at the fast detection of moving objects in dynamic scenes, a feature matching detection method based on the SIFT algorithm is designed in this paper. Simulation experiments show that the method designed in this paper has good detection performance.

Moving objects' detection in dynamic scenes is a very important task in video processing. In the applications of image processing (for example, video-surveillance), more attention is paid to whether there is an interested object in the scene rather than where the object is located. As a matter of fact, similar to static backgrounds, the statistical histograms of the most dynamic backgrounds have favorable stability, and these histograms would change clearly only when big and contrasting objects enter or move out of the scenes. This is a very important and interesting property for dynamic background. We propose a fast and simple algorithm, combining histograms in multiple color spaces and the superposition principle of statistical histogram, called Multiple Color Space Histogram Models (MCSHM). MCSHM first calculates statistical histograms of many color components in multiple color space and then use the changes of statistical histograms to determine whether there is an object rather than the changes in pixels or pixel-level regions. Thus, the computational complexity of MCSHM is kept at a very low level. The basic steps are as follows: firstly, convert each frame from RGB space to other color spaces and calculate the histograms of selected color components, then we can obtain the background histogram model; secondly, detect the objects using statistical histogram superposition principle; finally, update MCSHM by the result of detection. The experimental results demonstrate that our method can quickly and accurately detect moving objects in dynamic scenes.

Existing dynamic scene visual SLAM methods indiscriminately remove feature points on dynamic target bounding boxes when using object detection algorithms to detect prior targets, resulting in the incorrect removal of some valid static points and affecting the localization accuracy of the algorithm. This paper proposes a visual SDT-SLAM algorithm, which uses YOLOv11 to determine the prior target information of each frame of the image and designs a short-term dynamic object tracking algorithm to track targets and classify feature points in real time for different categories of bounding boxes. In the state estimation process, a depth difference estimation algorithm is used to recover some erroneously deleted static feature points, and an epipolar constraint algorithm is used to filter dynamic feature points from the scene in a fine-grained manner. This method precisely distinguishes the dynamic and static feature points in dynamic scenes. Compared with other SLAM systems, this system reasonably utilizes depth information to restore static feature points in dynamic scenes, and based on semantic information and epipolar constraints, it removes incorrect matching points, significantly preserving and fully utilizing the static feature points in dynamic scenes. This further improves the trajectory estimation accuracy of the SLAM algorithm in dynamic scenes. Without relying on computationally intensive segmentation methods, it achieves a new balance between real-time performance and accuracy. The SDT-SLAM algorithm was validated on the TUM and Bonn public datasets and in the real world. The results show that SDT-SLAM has advantages in speed and accuracy for localization in highly dynamic environments compared to other advanced algorithms.

We present a mapping approach to road scene awareness based on active stereo vision. We generalise traditional static multi-camera rectification techniques to enable active epipolar rectification with a mosaic representation of the output. The approach is used to apply standard static depth mapping and optical flow techniques to the active case. We use the framework to extract the ground plane and segment moving objects in dynamic scenes using arbitrarily moving cameras on a moving vehicle. The approach enables an estimation of the velocity of the vehicle relative to the road, and the velocity of objects in the scene. We provide footage of preliminary results of the system operating in real-time, including dynamic object extraction and tracking, ground plane extraction, and recovery of vehicle velocity.

Agnostic attribute segmentation of dynamic scenes with limited spatio-temporal resolution

Current visual simultaneous localization and mapping (SLAM) systems have demonstrated commendable efficacy in static environments. However, the presence of dynamic objects in real-world settings frequently leads to system discrepancies, significantly impairing the accuracy and robustness of SLAM systems. Conventional visual SLAM approaches typically utilize epipolar constraints to mitigate the impact of outliers; nevertheless, they encounter limitations when confronted with a substantial number of dynamic or planar moving objects. To tackle these issues, this paper introduces a novel dynamic visual SLAM system, termed DEG-SLAM. Initially, the system employs the YOLOv5 object detection network to identify dynamic objects, subsequently relaying the semantic information to the tracking module. During the tracking phase, both semantic information and epipolar constraints are leveraged to filter out dynamic feature points. To address the challenges posed by the malfunctioning of epipolar constraints in degenerate scenes, DEG-SLAM incorporates a degenerate constraint mechanism aimed at further eliminating dynamic feature points. Furthermore, a reprojection constraint has been introduced to enhance the filtering of absent dynamic feature points that lie outside the detection boxes. Experimental findings reveal that DEG-SLAM significantly improves accuracy and robustness when compared to traditional ORB-SLAM3 in dynamic environments. The performance benefits of DEG-SLAM are particularly evident in degenerate scenarios, thereby affirming its practicality and reliability in complex settings.

Simultaneous Localization and Mapping (SLAM) technology has important implications for the autonomous navigation of intelligent mobile robots. In the past few years, many excellent visual SLAM systems were born, and most of them can do a good job in a static environment. However, in dynamic scenes, unreliable feature points in the scene will lead to the decline of system positioning accuracy and even cause system failure. Traditional methods often use the removal of dynamic points to solve dynamic scene problems, but in some environments, the reduction of feature points will also affect the positioning accuracy. Therefore, based on the ORB-SLAM2 visual SLAM system, this paper proposes a semantic point and line SLAM system for an indoor dynamic environment. The improved SLAM system has good performance in an indoor dynamic environment. Finally, we evaluate our algorithm on the TUM RGB-D dynamic dataset. The results show that the absolute Trajectory Accuracy of SPL-SLAM is significantly improved compared with the original ORB-SLAM2.

Most existing algorithms solve the visual odometry problem in dynamic scenes by removing dynamic objects. However, some scenarios such as autonomous driving, visual surveillance and virtual reality, require explicit motion information of the objects to help making decisions and understanding scenes. In this paper, we propose DMVO (Dymamic Multi-motion Visual Odometry), a robust and effective visual odometry algorithm that estimate camera pose as well as motion of objects in dynamic scenes using information from stereo cameras. DMVO combines disparity and depth estimation networks to obtain high-precision depth information, which improves the accuracy of the system. Local pose graph and global factor graph of dynamic and static structure are constructed for optimization. The DMVO system is tested on an indoor Oxford multi-motion dataset and an outdoor KITTI tracking dataset. The results show that our system achieves improved performance compared with the state-of-the-art algorithms.

In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior with the aim that completed object geometry leads to better object reconstruction and tracking accuracy. For each incoming RGB-D frame, we perform instance segmentation to detect objects and build data associations between the detection and the existing object maps. A new object map will be created for each unmatched detection. For each matched object, we jointly optimise its pose and latent geometry representations using geometric residual and differential rendering residual towards its shape prior and completed geometry. Our approach shows better tracking and reconstruction performance compared to methods using traditional volumetric mapping or learned shape prior approaches. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.

Moving vehicles segmentation based on Bayesian framework for Gaussian motion model

Visual simultaneous localization and mapping (SLAM) algorithms in dynamic scenes will apply the moving feature points to the camera pose’s calculation, which will cause the continuous accumulation of errors. As a target-detection tool, mask R-CNN, which is often used in combination with the former, due to the limited training datasets, easily results in the semantic mask being incomplete and deformed, which will increase the error. In order to solve the above problems, we propose in this paper a visual SLAM algorithm based on an adaptive mask dilation strategy and the dynamic probability of the feature points, named MDP-SLAM. Firstly, we use the mask R-CNN target-detection algorithm to obtain the initial mask of the dynamic target. On this basis, an adaptive mask-dilation algorithm is used to obtain a mask that can completely cover the dynamic target and part of the surrounding scene. Then, we use the K-means clustering algorithm to segment the depth image information in the mask coverage area into absolute dynamic regions and relative dynamic regions. Combined with the epipolar constraint and the semantic constraint, the dynamic probability of the feature points is calculated, and then, the highly dynamic possible feature points are removed to solve an accurate final pose of the camera. Finally, the method is tested on the TUM RGB-D dataset. The results show that the MDP-SLAM algorithm proposed in this paper can effectively improve the accuracy of attitude estimation and has high accuracy and robustness in dynamic indoor scenes.

SLAM is a critical technology for enabling autonomous navigation and positioning in unmanned vehicles. Traditional visual simultaneous localization and mapping algorithms are built upon the assumption of a static scene, overlooking the impact of dynamic targets within real-world environments. Interference from dynamic targets can significantly degrade the system's localization accuracy or even lead to tracking failure. To address these issues, we propose a dynamic visual SLAM system named BY-SLAM, which is based on BEBLID and semantic information extraction. Initially, the BEBLID descriptor is introduced to describe Oriented FAST feature points, enhancing both feature point matching accuracy and speed. Subsequently, FasterNet replaces the backbone network of YOLOv8s to expedite semantic information extraction. By using the results of DBSCAN clustering object detection, a more refined semantic mask is obtained. Finally, by leveraging the semantic mask and epipolar constraints, dynamic feature points are discerned and eliminated, allowing for the utilization of only static feature points for pose estimation and the construction of a dense 3D map that excludes dynamic targets. Experimental evaluations are conducted on both the TUM RGB-D dataset and real-world scenarios and demonstrate the effectiveness of the proposed algorithm at filtering out dynamic targets within the scenes. On average, the localization accuracy for the TUM RGB-D dataset improves by 95.53% compared to ORB-SLAM3. Comparative analyses against classical dynamic SLAM systems further corroborate the improvement in localization accuracy, map readability, and robustness achieved by BY-SLAM.

The paper is devoted to the problem of constructing detailed 3D-reconstruction of objects in dynamic scenes when photographing the terrain by underwater autonomous vehicles. An image processing method is being considered that makes it possible to identify dynamic objects and points in a scene, which can be used to position an underwater autonomous vehicle. Identification of points in video stream images and 3D-reconstruction of point data are based on the ideas of the "seed" algorithm. Goal – increasing the accuracy of 3D-reconstruction of dynamic scenes when performing work by autonomous vehicles in previously unknown conditions. Statistical data characterizing the operation of algorithms for identifying objects and their points have been obtained. Conclusions are drawn about the features of working with small-polygonal and multi-polygonal objects. Approaches to determining the location of an underwater autonomous vehicle by analyzing point data are proposed. The use of image-based positioning systems is relevant in various applications, from unmanned vehicles and robot couriers to autonomous research vehicles. Decisions in this area may affect, for example, the effectiveness of search and rescue or repair work in difficult-to-reach conditions.

Observers’ memory for a person’s appearance can be compromised by the presence of a weapon, a phenomenon known as the weapon-focus effect (WFE). According to the unusual-item hypothesis, attention shifts from the perpetrator to the weapon because a weapon is an unusual object in many contexts. To test this assumption, we monitored participants’ eye movements while they watched a mock-crime video. The video was presented with sound and featured a female perpetrator holding either a weapon, a non-threatening unusual object, or a neutral object. Contrary to the predictions of current theories, there were no significant differences in total viewing times for the three objects. For the perpetrator, total viewing time was reduced when she held the non-threatening unusual object, but not when she held the weapon. However, weapon presence led to an attentional shift from the perpetrator’s face toward her body. Detailed time-course analyses revealed that the effects of object type were more pronounced during early scene viewing. Thus, our results do not support the idea of extended attentional shifts from the perpetrator toward the unusual objects, but instead suggest more complex attentional effects. Contrary to previous research, memory for the perpetrator’s appearance was not affected by object type. Thus, there was no WFE. An additional online experiment using the same videos and methodology produced a WFE, but this effect disappeared when the videos were presented without sound.