Cross-scene loop-closure detection with continual learning for visual simultaneous localization and mapping

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment.

With the development of automatic driving cars, simultaneous localization and mapping (SLAM) becomes a widely discussed topic. Loop closure detection, as an important module in lidar slam, can provide strong constraint for localization and mapping. In classical lidar SLAM algorithm, traditional loop closure detection method often fails to find out the matching nodes, or even has false positive results, especially in challenging environment like the crossroads or sharp turns. In this paper, a lidar SLAM loop closure detection method using adjacent nodes to check the reliability is introduced. A series of experiments is implemented to verify the effectiveness of the method.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

Loop closure detection is essential for simultaneous localization and mapping systems to decrease accumulating drift of trajectory estimations. Robust loop closure detection is specifically important in large-scale mapping, but it gets more challenging as the mapping environment grows. This paper proposes a SLAM system utilizing a two-pass loop closure detection method to improve mapping accuracy in large-scale environments. The proposed system finds loop closure candidates by employing global and local image features together. After selecting a group of candidates by similarity of global features, the system applies keypoint matching on this group to improve scene matching accuracy and determines loop closure candidates. We extensively evaluate the system on the widely used TUM RGB-D dataset, which contains sequences of small to large-scale indoor environments, with respect to different parameter combinations. The results show that the proposed method increases accuracy substantially and achieves large-scale mapping with acceptable overhead.

A Simultaneous Localization and Mapping (SLAM) system must be robust to support long-term mobile vehicle and robot applications. However, camera and LiDAR based SLAM systems can be fragile when facing challenging illumination or weather conditions which degrade the utility of imagery and point cloud data. Radar, whose operating electromagnetic spectrum is less affected by environmental changes, is promising although its distinct sensor model and noise characteristics bring open challenges when being exploited for SLAM. This paper studies the use of a Frequency Modulated Continuous Wave radar for SLAM in large-scale outdoor environments. We propose a full radar SLAM system, including a novel radar motion estimation algorithm that leverages radar geometry for reliable feature tracking. It also optimally compensates motion distortion and estimates pose by joint optimization. Its loop closure component is designed to be simple yet efficient for radar imagery by capturing and exploiting structural information of the surrounding environment. Extensive experiments on three public radar datasets, ranging from city streets and residential areas to countryside and highways, show competitive accuracy and reliability performance of the proposed radar SLAM system compared to the state-of-the-art LiDAR, vision and radar methods. The results show that our system is technically viable in achieving reliable SLAM in extreme weather conditions on the RADIATE Dataset, for example, heavy snow and dense fog, demonstrating the promising potential of using radar for all-weather localization and mapping.

A high efficient multi-robot simultaneous localization and mapping system using partial computing offloading assisted cloud point registration strategy

Simultaneous localization and mapping (SLAM) is mainly used to solve the problem of mobile robot navigation in an unknown environment. The loop closure detection is a key step in visual SLAM and plays an important role in building consistent maps and reducing pose cumulative errors. A new loop closure detection method based on the bag of semantic word (BOSW) is proposed in this paper, extracting semantic descriptors by using a neural network pre-trained on the scene recognition data set. Traditionally, the visual bag of word (BoW) model adopts the features such as ORB and SURF which is easily affected by the environment. With advantages of better ability to deal with changes in lighting, the convolutional neural network can extract the abstract features of the image. For these reasons, we proposed a new algorithm that combines the bag of word model and the convolutional neural network. The feature maps, extracted by the VGG16 neural network which is trained on the dataset named Places365 (VGG16-Places365), are used as the semantic descriptors instead of ORB descriptors. The workflow of building the vocabulary of the semantic words, extracting the feature vector of the image and computing the similarity score is presented. The performance of BOSW is evaluated by a comparison study with the traditional visual bag of word model and Inception-based deep learning method which use Inception's fully connected layer output as the feature vector of the picture. Overall, the experiments on New College datasets verified that our mode show stronger generalization and can achieve higher accuracy in the loop closure detection.

Loop closure detection plays an important role in simultaneous localization and mapping (SLAM) system. Traditional visual-based methods of loop closure detection are hard to relieve the issue of dynamic objects and change of light in complex environments. To improve robustness of loop closure detection, a loop closure detection algorithm named SV-Loop combines semantic and visual information is proposed in this paper. This work leverages the latest panoptic segmentation network to extract semantic information. Firstly, similarity of details is calculated through visual module in micro perspective. Then similarity of objects is computed by semantic module from a macro perspective. Finally, visual and semantic loop candidates are integrated to determine loop closure. Experiments have been conducted on open source datasets and practical scene. Experimental results have shown higher accuracy and better robustness of proposed method in complex environmental changes.

A LiDAR-intensity SLAM and loop closure detection method using an intensity cylindrical-projection shape context descriptor

Loop closure detection is crucial for simultaneous localization and mapping (SLAM), as it can effectively correct the accumulated errors. Complex scenarios put forward high requirements on the robustness of loop closure detection. Traditional feature-based loop closure detection methods often fail to meet these challenges. To solve this problem, this paper proposes a robust and efficient deep-learning-based loop closure detection approach. We employ MixVPR to extract global descriptors from keyframes and construct a global descriptor database. For local feature extraction, SuperPoint is utilized. Then, the constructed global descriptor database is used to find the loop frame candidates, and LightGlue is subsequently used to match the most similar loop frame and current keyframe with the local features. After matching, the relative pose can be computed. Our approach is first evaluated on several public datasets, and the results prove that our approach is highly robust to complex environments. The proposed approach is further validated on a real-world dataset collected by a drone and achieves accurate performance and shows good robustness in challenging conditions. Additionally, an analysis of time and memory costs is also conducted and proves that our approach can maintain accuracy and have satisfactory real-time performance as well.

Loop closure detection (LCD) is an important module of visual SLAM (Simultaneous Localization and Mapping), which can be regarded as an image retrieval task. The correct LCD can effectively reduce the cumulative errors during visual SLAM localization and obtain a globally consistent map. The currently common approach to LCD is based on a bag-of-words (BoW) and traditional manually designed features. However, The BoW-based method ignores the spatial location information and semantic information of visual features, and image aliasing will affect the detection accuracy of the algorithm. Meanwhile, the artificially designed features are very sensitive to light changes. This paper proposes a LCD algorithm based on convolutional neural networks (CNN) to extract global and local features. Extensive experiments on six publicly available datasets have shown that the method provides high recall with 100% accuracy in most of the evaluated datasets relative to traditional algorithms and some deep learning-based methods.

Loop closure detection (LCD) is crucial for the simultaneous localization and mapping system of an autonomous robot. Image features from a convolution neural network (CNN) have been widely used for LCD in recent years. Instead of directly using the feature vectors to compute the image similarity, we propose a novel and easy-to-implement method that manages features from a CNN via a novel approach to improve the performance. In this method, the elements of feature maps from the higher layer of the CNN are clustered to generate CNN words (CNNW). To encode spatial information of CNNW, we create word pairs (CNNWP) that are based on single words to improve the performance. In addition, traditional tricks that are used in methods that are based on bag of words (BoW) are integrated into our approach. We also demonstrate that the feature maps from lower layers can be used as descriptors to conduct local region matching between images. Via this approach, we can perform geometric verification for possible loop closures, similar to BoW methods, in our approach. The experimental results demonstrate that our method substantially outperforms state-of-the-art methods that directly use CNN features for LCD.

Solid-state light detection and ranging (LiDAR) has the advantages of low cost, small size and strong practicability. However, it faces challenges in simultaneous localization and mapping applications due to its small field of view and irregular scanning patterns. A solid-state LiDAR simultaneous localization and mapping system containing intensity information is proposed. In order to solve the irregular scanning characteristics of solid-state LiDAR, we introduce a data preprocessing framework and add intensity feature points to the front-end odometer. This improves the accuracy and robustness of positioning when geometric feature points are scarce, thus solving the problem of feature point degradation caused by a finite field of view. In the back-end optimization stage, we combine the geometric feature residuals with the intensity feature residuals through the proposed consistent difference function, so that the system can maintain good performance even in challenging environments. Finally, we conducted an extensive evaluation of the proposed algorithm on official datasets and various datasets collected from multiple platforms, and the results confirmed the validity of our approach. Compared with the corresponding method, in indoor scenes, the absolute trajectory error and relative attitude error are decreased by 54.5% and 5.3%. In outdoor scenes, the absolute trajectory error and relative attitude error are decreased by 29.6% and 58.8%.

This study addresses the challenges of robotic localization and navigation in visually degraded environments, such as low illumination and adverse weather conditions, by proposing a novel thermal infrared visual SLAM (Simultaneous Localization and Mapping) algorithm. The research introduces a new infrared visual odometry that integrates feature-based methods with optical flow techniques, enhancing image processing capabilities to mitigate the issues of high time overhead and cumulative errors in traditional feature-based odometry. Additionally, an improved bag-of-words model is employed to develop a novel loop closure detection method that addresses the challenge of scale drift. The purpose of this paper is to address the shortcomings in robustness and accuracy encountered by existing visual SLAM algorithms when processing low-texture thermal infrared images. Experimental validation using the JPL, Airey, and ViViD++ thermal infrared datasets demonstrates that the proposed algorithm exhibits superior real-time performance and robustness across various environments. Compared to mainstream thermal infrared visual SLAM algorithms, WTI-SLAM significantly improves the robot localization accuracy in weak-texture thermal infrared image scenarios, reducing the localization error by approximately 46%. This research offers an innovative and effective solution for achieving stable SLAM systems for robots operating in complex and visually degraded environments.

In simultaneous localization and mapping (SLAM) system, loop closing is defined as the correct identification of a previously visited location. Loop closing is essential for the precise self-localisation of the robot; however, the performance of loop closure detection is seriously affected by dynamic objects and perceptual aliasing in the environment. In the traditional likelihood matching methods, the number of matching words and the difference between them are not considered. This paper proposes a method based on mixed similarity to calculate the similarity score, thereby improving the performance of closed-loop detection. Experiments are performed on datasets from dynamic environments and visual repetitive environments, and then this method can produce a higher recall rate with 100% accuracy compared to the latest methods.