A Review of Multi-Sensor Fusion SLAM Systems Based on 3D LIDAR

In the last decades, Simultaneous Localization and Mapping (SLAM) proved to be a fundamental topic in the field of robotics, due to the many applications, ranging from autonomous driving to 3D reconstruction. Many systems have been proposed in literature exploiting a heterogeneous variety of sensors. State-of-the-art methods build their own map from scratch, using only data coming from the equipment of the robot, and not exploiting possible reconstructions of the environment. Moreover, temporary loss of data proves to be a challenge for SLAM systems, as it demands efficient re-localization to continue the localization process. In this paper, we present a SLAM system that exploits additional information coming from mapping services like OpenStreetMaps, hence the name OSM-SLAM, to face these issues. We extend an existing LiDAR-based Graph SLAM system, ART-SLAM, making it able to integrate the 2D geometry of buildings in the trajectory estimation process, by matching a prior OpenStreetMaps map with a single LiDAR scan. Each estimated pose of the robot is then associated with all buildings surrounding it. This association allows to improve localization accuracy, but also to adjust possible mistakes in the prior map. The pose estimates coming from SLAM are then jointly optimized with the constraints associated with the various OSM buildings, which can assume one of the following types: Buildings are always fixed (Prior SLAM); buildings surrounding a robot are movable in chunks, for every scan (Rigid SLAM); and every single building is free to move independently from the others (Non-rigid SLAM). Lastly, OSM maps can also be used to re-localize the robot when sensor data is lost. We compare the accuracy of the proposed system with existing methods for LiDAR-based SLAM, including the baseline, also providing a visual inspection of the results. The comparison is made by evaluating the estimated trajectory displacement using the KITTI odometry dataset. Moreover, the experimental campaign, along with an ablation study on the re-localization capabilities of the proposed system and its accuracy in loop detection-denied scenarios, allow a discussion about how the quality of prior maps influences the SLAM procedure, which may lead to worse estimates than the baseline.

Accurate localization and reliable mapping is essential for autonomous navigation of robots. As one of the core technologies for autonomous navigation, Simultaneous Localization and Mapping (SLAM) has attracted widespread attention in recent decades. Based on vision or LiDAR sensors, great efforts have been devoted to achieving real-time SLAM that can support a robot’s state estimation. However, most of the mature SLAM methods generally work under the assumption that the environment is static, while in dynamic environments they will yield degenerate performance or even fail. In this paper, first we quantitatively evaluate the performance of the state-of-the-art LiDAR-based SLAMs taking into account different pattens of moving objects in the environment. Through semi-physical simulation, we observed that the shape, size, and distribution of moving objects all can impact the performance of SLAM significantly, and obtained instructive investigation results by quantitative comparison between LOAM and LeGO-LOAM. Secondly, based on the above investigation, a novel approach named EMO to eliminating the moving objects for SLAM fusing LiDAR and mmW-radar is proposed, towards improving the accuracy and robustness of state estimation. The method fully uses the advantages of different characteristics of two sensors to realize the fusion of sensor information with two different resolutions. The moving objects can be efficiently detected based on Doppler effect by radar, accurately segmented and localized by LiDAR, then filtered out from the point clouds through data association and accurate synchronized in time and space. Finally, the point clouds representing the static environment are used as the input of SLAM. The proposed approach is evaluated through experiments using both semi-physical simulation and real-world datasets. The results demonstrate the effectiveness of the method at improving SLAM performance in accuracy (decrease by 30% at least in absolute position error) and robustness in dynamic environments.

This paper presents a review and comparison of the common 2D SLAM (Simultaneous Localization and Mapping) systems in an indoor static environment, by utilizing the ROS-based SLAM libraries on a experimental mobile robot equipped with a 2D LIDAR module, IMU and wheel encoders. The three common algorithms (GMapping, Hector-SLAM, Google Cartographer) are the metrical map generating approaches of SLAM, which are categorized as filter-based or graph-based SLAM. The experimental results are acquired from the similar robot trajectory in both the simulated and real-world environment for further analysis under different circumstances. Overall, this paper describes the strength and weaknesses of the algorithms and visualizes the differences in terms of constructed maps, as it is mandatory to select the most appropriate system according to the intended application, as well as to identify the potential direction of optimization in the future.

The question of is central in the context of dense simultaneous localization and mapping (SLAM). Learning-based approaches have the potential to leverage data or task performance to directly inform the representation. However, blending representation learning approaches with classical SLAM systems has remained an open question, because of their highly modular and complex nature. A SLAM system transforms raw sensor inputs into a distribution over the state(s) of the robot and the environment. If this transformation (SLAM) were expressible as a differentiable function, we could leverage task-based error signals over the outputs of this function to learn representations that optimize task performance. However, this is infeasible as several components of a typical dense SLAM system are non-differentiable. In this work, we propose ∇SLAM (gradSLAM), a methodology for posing SLAM systems as differentiable computational graphs, which unifies gradient-based learning and SLAM. We propose differentiable trust-region optimizers, surface measurement and fusion schemes, and raycasting, without sacrificing accuracy. This amalgamation of dense SLAM with computational graphs enables us to backprop all the way from 3D maps to 2D pixels, opening up new possibilities in gradient-based learning for SLAM1.

A Simultaneous Localization and Mapping(SLAM) system is a complex program consisting of several interconnected components with different functionalities such as optimization, tracking or loop detection. Whereas the literature addresses in detail how enhancing the algorithmic aspects ofthe individual components improves SLAM performance, the modal aspects, such as when to localize, relocalize or close a loop, are usually left aside. In this paper, we address the modal aspects of a SLAM system and show that the design of the modal controller has a strong impact on SLAM performance in particular in terms of robustness against unforeseen events such as sensor failures, perceptual aliasing or kidnapping. We preset a novel taxonomy for the components of a modern SLAM system, investigate their interplay and propose a highly modular architecture of a generic SLAM system using the Unified Modeling LanguageTM(UML) state machine formalism. The result, called SLAM state machine, is compared to the modal controller of several state-of-the-art SLAM systems and evaluated in two experiments. We demonstrate that our state machine handles unforeseen events much more robustly than the state-of-the-art systems.

Purpose – To build a persistent map with visual landmarks is one of the most important steps for implementing the visual simultaneous localization and mapping (SLAM). The corner detector is a common method utilized to detect visual landmarks for constructing a map of the environment. However, due to the scale-variant characteristic of corner detection, extensive computational cost is needed to recover the scale and orientation of corner features in SLAM tasks. The purpose of this paper is to build the map using a local invariant feature detector, namely speeded-up robust features (SURF), to detect scale- and orientation-invariant features as well as provide a robust representation of visual landmarks for SLAM. Design/methodology/approach – SURF are scale- and orientation-invariant features which have higher repeatability than that obtained by other detection methods. Furthermore, SURF algorithms have better processing speed than other scale-invariant detection method. The procedures of detection, description and matching of regular SURF algorithms are modified in this paper in order to provide a robust representation of visual landmarks in SLAM. The sparse representation is also used to describe the environmental map and to reduce the computational complexity in state estimation using extended Kalman filter (EKF). Furthermore, the effective procedures of data association and map management for SURF features in SLAM are also designed to improve the accuracy of robot state estimation. Findings – Experimental works were carried out on an actual system with binocular vision sensors to prove the feasibility and effectiveness of the proposed algorithms. EKF SLAM with the modified SURF algorithms was applied in the experiments including the evaluation of accurate state estimation as well as the implementation of large-area SLAM. The performance of the modified SURF algorithms was compared with those obtained by regular SURF algorithms. The results show that the SURF with less-dimensional descriptors is the most suitable representation of visual landmarks. Meanwhile, the integrated system is successfully validated to fulfill the capabilities of visual SLAM system. Originality/value – The contribution of this paper is the novel approach to overcome the problem of recovering the scale and orientation of visual landmarks in SLAM tasks. This research also extends the usability of local invariant feature detectors in SLAM tasks by utilizing its robust representation of visual landmarks. Furthermore, data association and map management designed for SURF-based mapping in this paper also give another perspective for improving the robustness of SLAM systems.

A Simultaneous Localization and Mapping (SLAM) system must be robust to support long-term mobile vehicle and robot applications. However, camera and LiDAR based SLAM systems can be fragile when facing challenging illumination or weather conditions which degrade the utility of imagery and point cloud data. Radar, whose operating electromagnetic spectrum is less affected by environmental changes, is promising although its distinct sensor model and noise characteristics bring open challenges when being exploited for SLAM. This paper studies the use of a Frequency Modulated Continuous Wave radar for SLAM in large-scale outdoor environments. We propose a full radar SLAM system, including a novel radar motion estimation algorithm that leverages radar geometry for reliable feature tracking. It also optimally compensates motion distortion and estimates pose by joint optimization. Its loop closure component is designed to be simple yet efficient for radar imagery by capturing and exploiting structural information of the surrounding environment. Extensive experiments on three public radar datasets, ranging from city streets and residential areas to countryside and highways, show competitive accuracy and reliability performance of the proposed radar SLAM system compared to the state-of-the-art LiDAR, vision and radar methods. The results show that our system is technically viable in achieving reliable SLAM in extreme weather conditions on the RADIATE Dataset, for example, heavy snow and dense fog, demonstrating the promising potential of using radar for all-weather localization and mapping.

Simultaneous localization and mapping (SLAM) is an important problem to solve in robotics theory in order to build truly autonomous mobile robots. This work presents a novel method for implementing a SLAM system based on a single camera sensor. The SLAM with a single camera, or monocular SLAM, is probably one of the most complex SLAM variants. In this case, a single camera, which is freely moving through its environment, represents the sole sensor input to the system. The sensors have a large impact on the algorithm used for SLAM. Cameras are used more frequently, because they provide a lot of information and are well adapted for embedded systems: they are light, cheap and power-saving. Nevertheless, and unlike range sensors, which provide range and angular information, a camera is a projective sensor providing only angular measurements of image features. Therefore, depth information (range) cannot be obtained in a single step. In this case, special techniques for feature system-initialization are needed in order to enable the use of angular sensors (as cameras) in SLAM systems. The main contribution of this work is to present a novel and robust scheme for incorporating and measuring visual features in filtering-based monocular SLAM systems. The proposed method is based in a two-step technique, which is intended to exploit all the information available in angular measurements. Unlike previous schemes, the values of parameters used by the initialization technique are derived directly from the sensor characteristics, thus simplifying the tuning of the system. The experimental results show that the proposed method surpasses the performance of previous schemes.

In this work, we present a comparative analysis of the trajectories estimated from various Simultaneous Localization and Mapping (SLAM) systems in a simulation environment for vineyards. Vineyard environment is challenging for SLAM methods, due to visual appearance changes over time, uneven terrain, and repeated visual patterns. For this reason, we created a simulation environment specifically for vineyards to help studying SLAM systems in such a challenging environment. We evaluated the following SLAM systems: LIO-SAM, StaticMapping, ORB-SLAM2, and RTAB-MAP in four different scenarios. The mobile robot used in this study equipped with 2D and 3D lidars, IMU, and RGB-D camera (Kinect v2). The results show good and encouraging performance of RTAB-MAP in such an environment.

Simultaneous localization and mapping (SLAM) is a core element of every autonomous mobile robot. The underlying engine of a SLAM system is its back-end, which aims at optimally estimating the trajectory and map of the environment based on sensor data abstractions. Over the past decade, SLAM solutions based on graph optimization approaches prevailed over the filtering based solutions, since they dominated in performance over a wider range of applications. In this paper we propose a novel filtering based SLAM back-end based on the exactly sparse delayed state filter (ESDSF) derived on Lie groups (LG-ESDSF). The proposed filter retains all the good characteristics of the classic ESDSF, but also respects the state space geometry by employing filtering equations directly on Lie groups. We have compared our SLAM system with two current state-of-the-art SLAM solutions, namely ORB-SLAM and LSD-SLAM, on the KITTI vision benchmark suite. Test results show that the proposed SLAM based on the LG-ESDSF back-end can achieve same level of accuracy as the methods based on the graph optimization techniques, while maintaining lower computation times.

Localization and mapping in a dynamic scene is a crucial problem for the indoor visual simultaneous localization and mapping (SLAM) system. Most existed visual odometry (VO) or SLAM systems are based on the assumption that the environment is static. The performance of a SLAM system may degenerate when it is operated in a severely dynamic environment. The assumption limits the applications of RGB-D SLAM in the dynamic environment. In this paper, we propose a workflow to segment the objects accurately, which will be marked as the potentially dynamic-object area based on the semantic information. A novel approach for motion detection and removal from the moving camera is introduced. We integrate the semantics-based motion detection and the segmentation approach with an RGB-D SLAM system. To evaluate the effectiveness of the proposed approach, we conduct the experiments on the challenging dynamic sequences of TUM-RGBD datasets. The experimental results suggest that our approach improves the accuracy of localization and outperforms the state-of-the-art dynamic-removal-based SLAM system in both severely dynamic and slightly dynamic scenes.

Augmented Reality (AR) has increasingly benefited from the use of Simultaneous Localization and Mapping (SLAM) systems. This technology has enabled developers to create AR markerless applications, but lack semantic understanding of their environment. The inclusion of this information would empower AR applications to better react to the surroundings more realistically. To gain semantic knowledge, in recent years, focus has shifted toward fusing SLAM systems with neural networks, giving birth to the field of Semantic SLAM. Building on existing research, this paper aimed to create a SLAM system that generates a 3D map using ORB-SLAM2 and enriches it with semantic knowledge originated from the Fast-SCNN network. The key novelty of our approach is a new method for improving the predictions of neural networks, employed to balance the loss of accuracy introduced by efficient real-time models. Exploiting sensor information provided by a smartphone, GPS coordinates are utilized to query the OpenStreetMap database. The returned information is used to understand which classes are currently absent in the environment, so that they can be removed from the network’s prediction with the goal of improving its accuracy. We achieved 87.40% Pixel Accuracy with Fast-SCNN on our custom version of COCO-Stuff and showed an improvement by involving GPS data for our self-made smartphone dataset resulting in 90.24% Pixel Accuracy. Having in mind the use on smartphones, the implementation aimed to find a trade-off between accuracy and efficiency, making the system achieve an unprecedented speed. To this end, the system was carefully designed and a strong focus on lightweight neural networks is also fundamental. This enabled the creation of an above real-time Semantic SLAM system that we called EnvSLAM (Environment SLAM). Our extensive evaluation reveals the efficiency of the system features and the operability in above real-time (48.1 frames per second with an input image resolution of 640 × 360 pixels). Moreover, the GPS integration indicates an effective improvement of the network’s prediction accuracy.

In indoor dynamic scenes, traditional visual simultaneous localization and mapping (SLAM) algorithms based on RGB-D cameras incorrectly use dynamic features to estimate the poses of the cameras, and do not fully utilize the geometric information in the scenes, resulting in the low positioning accuracy and robustness of SLAM systems. To solve this problem, this paper proposes an RGB-D SLAM algorithm based on multiple geometric features and semantic segmentation. The core of our SLAM system is the proposed robust exclusion method of dynamic point and line features. The method consists of the following three steps: (a) identify potential dynamic point features using motion consistency checks; (b) obtain potential motion regions via semantic segmentation, and then determine dynamic regions by combining these with dynamic point features; (c) remove point and line features in dynamic regions. The exclusion method of dynamic point and line features can be easily integrated into RGB-D SLAM systems for improving the accuracy and robustness of SLAM systems in dynamic scenes. Experimental results on the Technische Universität München (TUM) dataset demonstrate that the proposed algorithm has better positioning accuracy and stability than the original dynamic semantic (DS-SLAM) algorithm in dynamic environments. The effectiveness of the proposed algorithm is verified by comparison with other classical visual SLAM algorithms. Better mapping performance is achieved by this proposed algorithm in actual indoor scenes.

Simultaneous Localization and Mapping (SLAM) plays an important role in the computer vision and robotics field. The traditional SLAM framework adopts a strong static world assumption for analysis convenience. How to cope with dynamic environments is of vital importance and attracts more attentions. Existing SLAM systems toward dynamic scenes either solely utilize semantic information, solely utilize geometry information, or naively combine the results from them in a loosely coupled way. In this paper, we present SOF-SLAM: Semantic Optical Flow SLAM, a visual semantic SLAM system toward dynamic environments, which is built on RGB-D mode of ORB-SLAM2. A new dynamic features detection approach called semantic optical flow is proposed, which is a kind of tightly coupled way and can fully take advantage of feature's dynamic characteristic hidden in semantic and geometry information to remove dynamic features effectively and reasonably. The pixel-wise semantic segmentation results generated by SegNet serve as mask in the proposed semantic optical flow to get a reliable fundamental matrix, which is then used to filter out the truly dynamic features. Only the remaining static features are reserved in the tracking and optimization module to achieve accurate camera pose estimation in dynamic environments. Experiments on public TUM RGB-D dataset and in real-world environment are conducted. Compared with ORB-SLAM2, the proposed SOF-SLAM achieves averagely 96.73% improvements in high-dynamic scenarios. It also outperforms the other four state-of-the-art SLAM systems which cope with the dynamic environments.

Typical visual simultaneous localization and mapping (SLAM) systems rely on purely geometric elements. Recently semantic SLAM has become popular due to its capability to obtain high-level understanding of the environment. However, most semantic SLAM systems merely aim to produce semantic maps rather than exploiting semantic constraints to boost the accuracy performance. In this paper, we propose a new RGB-D SLAM system for the use in indoor environments. Given that a ground plane represents a significant part of most indoor scenes, our new system exploit a semantic ground plane constraint inferred from a deep neural network along with geometric constraints on the state estimation to improve the performance. More specifically, we introduce a semantically meaningful geometric primitive, namely, a global ground plane to the optimization process of visual SLAM. The global ground model is able to provide an extra constraint on the state estimation through which both frame-to-keyframe and frame-to-model tracking processes are combined, and low-texture ground plane is effectively utilized. Experimental results for the TUM dataset and a real scene have demonstrated the improved performance brought by the semantic constraint in our method.