Integrating Sparse Learning-Based Feature Detectors into Simultaneous Localization and Mapping—A Benchmark Study

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment.

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$[u,v,1]^T$</tex-math> </inline-formula> become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b> isual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> nertial <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SLAM</b> framework for cameras with extremely <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> arge <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</b> oV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method based on the unit length representation is integrated into the loop closure module. We collect the PALVIO dataset using a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</b> anoramic <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> nnular <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> ens (PAL) system with an entire FoV of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$360^\circ{\times}(40^\circ{\sim}120^\circ)$</tex-math> </inline-formula> and an Inertial Measurement Unit (IMU) for <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b> isual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> nertial <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</b> dometry (VIO) to address the lack of panoramic SLAM datasets. Experiments on the established PALVIO and public datasets show that the proposed LF-VISLAM outperforms state-of-the-art SLAM methods. Our code will be open-sourced at https://github.com/flysoaryun/LF-VISLAM. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Motivated by the challenges of handling large-FoV cameras in SLAM applications, this paper proposes LF-VISLAM, a novel SLAM framework that uses a large-FoV camera and IMU sensors. Our framework is equipped with a loop closure thread that can use attitude information to eliminate accumulated errors. We have made algorithmic adjustments and optimizations to the negative half-plane features to better adapt to cameras with large FoV. Experimental evaluations demonstrate that LF-VISLAM significantly outperforms traditional SLAM methods. Additionally, the code will be open-sourced, providing easy access for research and implementation. Overall, LF-VISLAM is a promising solution to improve the performance of SLAM in challenging environments of autonomous driving and robotics.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

Simultaneous Localization and Mapping (SLAM) is a fundamental problem in mobile robotics. However, the majority of Visual SLAM algorithms assume a static scenario, limiting their applicability in real-world environments. Dealing with dynamic content in Visual SLAM is still an open problem, with solutions usually relying on direct or feature-based methods. Deep learning techniques can improve the SLAM solution in environments with a priori dynamic objects, providing high-level information of the scene. This paper presents a new approach to SLAM in human populated environments using deep learning-based techniques. The system is built on ORB-SLAM2, a state-of-the-art SLAM system. The proposed methodology is evaluated using a benchmark dataset, outperforming other Visual SLAM methods in highly dynamic scenarios.

Simultaneous Localization and Mapping (SLAM) plays an important role in the computer vision and robotics field. The traditional SLAM framework adopts a strong static world assumption for analysis convenience. How to cope with dynamic environments is of vital importance and attracts more attentions. Existing SLAM systems toward dynamic scenes either solely utilize semantic information, solely utilize geometry information, or naively combine the results from them in a loosely coupled way. In this paper, we present SOF-SLAM: Semantic Optical Flow SLAM, a visual semantic SLAM system toward dynamic environments, which is built on RGB-D mode of ORB-SLAM2. A new dynamic features detection approach called semantic optical flow is proposed, which is a kind of tightly coupled way and can fully take advantage of feature's dynamic characteristic hidden in semantic and geometry information to remove dynamic features effectively and reasonably. The pixel-wise semantic segmentation results generated by SegNet serve as mask in the proposed semantic optical flow to get a reliable fundamental matrix, which is then used to filter out the truly dynamic features. Only the remaining static features are reserved in the tracking and optimization module to achieve accurate camera pose estimation in dynamic environments. Experiments on public TUM RGB-D dataset and in real-world environment are conducted. Compared with ORB-SLAM2, the proposed SOF-SLAM achieves averagely 96.73% improvements in high-dynamic scenarios. It also outperforms the other four state-of-the-art SLAM systems which cope with the dynamic environments.

The robot simultaneous localization and mapping (SLAM) is a very important and useful technology in the robotic field. However, the environmental map constructed by the traditional visual SLAM method contains little semantic information, which cannot satisfy the needs of complex applications. The semantic map can deal with this problem efficiently, which has become a research hot spot. This paper proposed an improved deep residual network- (ResNet-) based semantic SLAM method for monocular vision robots. In the proposed approach, an improved image matching algorithm based on feature points is presented, to enhance the anti-interference ability of the algorithm. Then, the robust feature point extraction method is adopted in the front-end module of the SLAM system, which can effectively reduce the probability of camera tracking loss. In addition, the improved key frame insertion method is introduced in the visual SLAM system to enhance the stability of the system during the turning and moving of the robot. Furthermore, an improved ResNet model is proposed to extract the semantic information of the environment to complete the construction of the semantic map of the environment. Finally, various experiments are conducted and the results show that the proposed method is effective.

Visual Simultaneous Localization and Mapping (SLAM) is crucial to many applications such as self-driving vehicles and robot tasks. However, it is still challenging for existing visual SLAM approaches to achieve good performance in low-texture or illumination-changing scenes. In recent years, some researchers have turned to edge-based SLAM approaches to deal with the challenging scenes, which are more robust than feature-based and direct SLAM methods. Nevertheless, existing edge-based methods are computationally expensive and inferior than other visual SLAM systems in terms of accuracy. In this study, we propose EdgeSLAM, a novel RGB-D edge-based SLAM approach to deal with challenging scenarios that is efficient, accurate, and robust. EdgeSLAM is built on two innovative modules: efficient edge selection and adaptive robust motion estimation. The edge selection module can efficiently select a small set of edge pixels, which significantly improves the computational efficiency without sacrificing the accuracy. The motion estimation module improves the system's accuracy and robustness by adaptively handling outliers in motion estimation. Extensive experiments were conducted on TUM RGBD, ICL-NUIM and ETH3D datasets, and experimental results show that EdgeSLAM significantly outperforms five state-of-the-art (SOTA) methods in terms of efficiency, accuracy, and robustness, which achieves 29.17% accuracy improvements with a high processing speed of up to 120 FPS and a high positioning success rate of 97.06%.

Typical visual simultaneous localization and mapping (SLAM) systems rely on purely geometric elements. Recently semantic SLAM has become popular due to its capability to obtain high-level understanding of the environment. However, most semantic SLAM systems merely aim to produce semantic maps rather than exploiting semantic constraints to boost the accuracy performance. In this paper, we propose a new RGB-D SLAM system for the use in indoor environments. Given that a ground plane represents a significant part of most indoor scenes, our new system exploit a semantic ground plane constraint inferred from a deep neural network along with geometric constraints on the state estimation to improve the performance. More specifically, we introduce a semantically meaningful geometric primitive, namely, a global ground plane to the optimization process of visual SLAM. The global ground model is able to provide an extra constraint on the state estimation through which both frame-to-keyframe and frame-to-model tracking processes are combined, and low-texture ground plane is effectively utilized. Experimental results for the TUM dataset and a real scene have demonstrated the improved performance brought by the semantic constraint in our method.

Simultaneous localization and mapping (SLAM) techniques are widely researched, since they allow the simultaneous creation of a map and the sensors’ pose estimation in an unknown environment. Visual-based SLAM techniques play a significant role in this field, as they are based on a low-cost and small sensor system, which guarantees those advantages compared to other sensor-based SLAM techniques. The literature presents different approaches and methods to implement visual-based SLAM systems. Among this variety of publications, a beginner in this domain may find problems with identifying and analyzing the main algorithms and selecting the most appropriate one according to his or her project constraints. Therefore, we present the three main visual-based SLAM approaches (visual-only, visual-inertial, and RGB-D SLAM), providing a review of the main algorithms of each approach through diagrams and flowcharts, and highlighting the main advantages and disadvantages of each technique. Furthermore, we propose six criteria that ease the SLAM algorithm’s analysis and consider both the software and hardware levels. In addition, we present some major issues and future directions on visual-SLAM field, and provide a general overview of some of the existing benchmark datasets. This work aims to be the first step for those initiating a SLAM project to have a good perspective of SLAM techniques’ main elements and characteristics.

This paper proposes an appearance-based method to detect loop closure in visual SLAM (Simultaneous Localization and Mapping). To solve this problem, we make use of omnidirectional images and the internal odometry captured by a robot in a real indoor environment. We build an appearance-based model and, subsequently, two maps of the environment are constructed, one metric and other topological with relationships between them. These relationships are updated in each step of our hybrid approach. The topological map is a graph built from the appearance information in the scenes. A new node is added when the new visual information is different enough from the previous information. At the same time, we check a possible topological loop closure with previous nodes. On the other hand we estimate the metric position of the new pose using a Monte-Carlo approach with the aim of building a metric map. The experimental results demonstrate the reasonable performance of our method.KeywordsAppearance-base descriptorOmnidirectional ImagesMonte-Carlo SLAMHybrid Metric-Topological mappingLoop closure detection

Current visual simultaneous localization and mapping (SLAM) systems have demonstrated commendable efficacy in static environments. However, the presence of dynamic objects in real-world settings frequently leads to system discrepancies, significantly impairing the accuracy and robustness of SLAM systems. Conventional visual SLAM approaches typically utilize epipolar constraints to mitigate the impact of outliers; nevertheless, they encounter limitations when confronted with a substantial number of dynamic or planar moving objects. To tackle these issues, this paper introduces a novel dynamic visual SLAM system, termed DEG-SLAM. Initially, the system employs the YOLOv5 object detection network to identify dynamic objects, subsequently relaying the semantic information to the tracking module. During the tracking phase, both semantic information and epipolar constraints are leveraged to filter out dynamic feature points. To address the challenges posed by the malfunctioning of epipolar constraints in degenerate scenes, DEG-SLAM incorporates a degenerate constraint mechanism aimed at further eliminating dynamic feature points. Furthermore, a reprojection constraint has been introduced to enhance the filtering of absent dynamic feature points that lie outside the detection boxes. Experimental findings reveal that DEG-SLAM significantly improves accuracy and robustness when compared to traditional ORB-SLAM3 in dynamic environments. The performance benefits of DEG-SLAM are particularly evident in degenerate scenarios, thereby affirming its practicality and reliability in complex settings.

Although tremendous progress has been made in Simultaneous Localization and Mapping (SLAM), the scene rigidity assumption limits wide usage of visual SLAMs in the real-world environment of computer vision, smart robotics and augmented reality. To make SLAM more robust in dynamic environments, outliers on the dynamic objects, including unknown objects, need to be removed from tracking process. To address this challenge, we present a novel real-time visual SLAM system, KMOP-vSLAM, which adds the capability of unsupervised learning segmentation and human detection to reduce the drift error of tracking in indoor dynamic environments. An efficient geometric outlier detection method is proposed, using dynamic information of the previous frames as well as a novel probability model to judge moving objects with the help of geometric constraints and human detection. Outlier features belonging to moving objects are largely detected and removed from tracking. The well-known dataset, TUM, is used to evaluate tracking errors in dynamic scenes where people are walking around. Our approach yields a significantly lower trajectory error compared to state-of-the-art visual SLAMs using an RGB-D camera.

In the automatic driving system, the external environment of the vehicle is perceived, in fact, there is also a perception sensor that has been silently dedicated in the system, that is, the positioning module. This paper explores the role of SLAM (Simultaneous Localization and Mapping) technology in autonomous vehicles, particularly in automatic lane change behavior prediction and environment perception. It emphasizes the limitations of traditional methods and the advantages of SLAM, especially visual SLAM, for accurate positioning and mapping. The discussion covers SLAM fundamentals, challenges, and the significance of visual SLAM's higher perception ability. Case studies on Waymo and Tesla illustrate the application of visual SLAM in achieving high-precision navigation and lane change prediction. The paper concludes by highlighting future research directions to enhance the intelligence and adaptability of automated lane change systems through advancements in AI and sensor technology, alongside optimizing SLAM algorithms for reliable driving in various scenarios.

Building maps is one of the most fundamental tasks for an autonomous robot. This robot should be able to construct a map of the environment and, at the same time, localize itself in it. This problem, known as Simultaneous Localization and Mapping (SLAM), has received great interest in the last years (Leonard & Durrant-Whyte, 1991). In our particular case, the robots build their maps using the FastSLAM algorithm (Montemerlo et al., 2002). The main idea of the FastSLAM algorithm is to separate the two fundamental aspects of the SLAM problem: the estimate of the robot’s pose and the estimate of the map. This algorithm uses a particle set that represents the uncertainty of the robot’s pose (localization problem) meanwhile each particle has its own associated map (several individual estimates of the landmarks conditioned to the robot’s path). The solution to the SLAM problem is performed by means of a sampling and particle generation process, in which the particles whose current observations do not fit with their associated map are eliminated. The FastSLAM algorithm has proved to be robust to false data association and it is able to represent models of non-linear movements in a reliable way (Howard, 2006). In relation to the sensors used to build the maps, many authors use range sensors such as sonar (Kwak et al., 2008; Wijk & Christensen, 2000) or laser (Thrun, 2004; Triebel & Burgard, 2005). Nevertheless, there is an increasing interest on using cameras as sensors. This approach is denoted as visual SLAM (Valls Miro et al., 2006). These devices obtain a higher amount of information from the environment and they are less expensive than laser as well. Furthermore, 3D information can be obtained when using stereo cameras. These are the sensors used in this work. Most visual SLAM approaches are landmark-based. These landmarks consist of a set of distinctive points which are referred to a global reference system. The main advantage of landmark-based maps is the compactness of their representation. By contrast, this kind of maps requires the existence of structures or objects that are distinguishable enough. The map building problem can be solved by a single robot (Moutalier & Chatila, 1989), but it will be more efficiently managed if there is a team of robots, which collaborates in the construction of the map (Howard, 2006). In this case, the space can be divided so that the distances traversed are shorter and thus the odometry errors will be smaller. In this work, 7

RGB-D cameras have become a popular sensor for Simultaneous Localization and Mapping (SLAM) because they can provide RGB and depth information at the same time. The spatial location of feature points of SLAM is obtained by its corresponding depth information from RGB-D sensors. However, the depth information of the feature at the edge of the object is very noisy, and the measurement is often inaccurate. At the same time, there is poor real-time performance, low accuracy, and much difficulties of dense construction in ORB-based Visual SLAM (visual SLAM, VSLAM). Therefore, an improved ORB features based SLAM framework with corresponding method using RGB-D data is proposed in this paper. We extract and match features with improved ORB. Then the local map is used to perform local optimization for pose estimation, and an efficient key frame screening method is designed. The proposed method effectively reduces errors, enhances robustness, and builds a dense map of the environment.