Appearance-based loop closure detection combining lines and learned points for low-textured environments

Humans maintain good memory and recognition capability of previous environments when they are learning about new ones. Thus humans are able to continually learn and increase their experience. It is also obvious importance for autonomous mobile robot. The simultaneous localization and mapping system plays an important role in localization and navigation of robot. The loop-closure detection method is an indispensable part of the relocation and map construction, which is critical to correct mappoint errors of simultaneous localization and mapping. Existing visual loop-closure detection methods based on deep learning are not capable of continual learning in terms of cross-scene environment, which bring a great limitation to the application scope. In this article, we propose a novel end-to-end loop-closure detection method based on continual learning, which can effectively suppress the decline of the memory capability of simultaneous localization and mapping system by introducing firstly the orthogonal projection operator into the loop-closure detection to overcome the catastrophic forgetting problem of mobile robot in large-scale and multi-scene environments. Based on the three scenes from public data sets, the experimental results show that the proposed method has a strong capability of continual learning in the cross-scene environment where existing state-of-the-art methods fail.

Loop closure detection is crucial for simultaneous localization and mapping (SLAM), as it can effectively correct the accumulated errors. Complex scenarios put forward high requirements on the robustness of loop closure detection. Traditional feature-based loop closure detection methods often fail to meet these challenges. To solve this problem, this paper proposes a robust and efficient deep-learning-based loop closure detection approach. We employ MixVPR to extract global descriptors from keyframes and construct a global descriptor database. For local feature extraction, SuperPoint is utilized. Then, the constructed global descriptor database is used to find the loop frame candidates, and LightGlue is subsequently used to match the most similar loop frame and current keyframe with the local features. After matching, the relative pose can be computed. Our approach is first evaluated on several public datasets, and the results prove that our approach is highly robust to complex environments. The proposed approach is further validated on a real-world dataset collected by a drone and achieves accurate performance and shows good robustness in challenging conditions. Additionally, an analysis of time and memory costs is also conducted and proves that our approach can maintain accuracy and have satisfactory real-time performance as well.

A typical approach to describe an image in loop closure detection for visual SLAM is to extract a set of local patch descriptors and encode them into a co-occurrence vector. The most common patch encoding strategy is known as bag-of-visual-words (BoVW) representation, which consists of clustering the local descriptors into visual vocabulary. The distinctiveness of images is difficult to represent since most of them contain similar texture information, which may lead to false positive results. In this paper, the vocabulary is used as a whole by adopting the Fisher kernel (FK) framework. The new representation describes the image as the gradient vector of the likelihood function. The efficiently computed vectors can be compressed with a minimal loss of accuracy using product quantization and perform well in the task of loop closure detection. The proposed method achieves a higher recall rate with 100% precision in loop closure detection compared with state-of-the-art methods, and the detection on bidirectional loops is also enhanced. vSLAM systems may perceive the environment more efficiently by constructing a globally consistent map with the proposed loop closure detection method, which is potentially valuable for applications such as autonomous driving.KeywordsFisher kernelLoop closure detectionvisual SLAM

Simultaneous localization and mapping (SLAM) plays a crucial role in the field of intelligent mobile robots. However, the traditional Visual SLAM (VSLAM) framework is based on strong assumptions about static environments, which are not applicable to dynamic real-world environments. The correctness of re-localization and recall of loop closure detection are both lower when the mobile robot loses frames in a dynamic environment. Thus, in this paper, the re-localization and loop closure detection method with a semantic topology graph based on ORB-SLAM2 is proposed. First, we use YOLOv5 for object detection and label the recognized dynamic and static objects. Secondly, the topology graph is constructed using the position information of static objects in space. Then, we propose a weight expression for the topology graph to calculate the similarity of topology in different keyframes. Finally, the re-localization and loop closure detection are determined based on the value of topology similarity. Experiments on public datasets show that the semantic topology graph is effective in improving the correct rate of re-localization and the accuracy of loop closure detection in a dynamic environment.

In this work, fast and efficient appearance-based methods for visual loop-closure detection are proposed. The widely used technique based on the Bag-of-Words image representation has shown some limitations especially with aliasing problem. In this work, however, an appearance-based approach for loop closure detection using local invariant and colour features is proposed. The first technique uses Bayes Decision Theory for loop closure detection based on Gaussian Mixture Model (GMM). A new technique based on the combination of GMM with the KD-Tree data structure is presented as well. The techniques have been validated using monocular image sequences from several environments.

Loop closure detection is essential for simultaneous localization and mapping systems to decrease accumulating drift of trajectory estimations. Robust loop closure detection is specifically important in large-scale mapping, but it gets more challenging as the mapping environment grows. This paper proposes a SLAM system utilizing a two-pass loop closure detection method to improve mapping accuracy in large-scale environments. The proposed system finds loop closure candidates by employing global and local image features together. After selecting a group of candidates by similarity of global features, the system applies keypoint matching on this group to improve scene matching accuracy and determines loop closure candidates. We extensively evaluate the system on the widely used TUM RGB-D dataset, which contains sequences of small to large-scale indoor environments, with respect to different parameter combinations. The results show that the proposed method increases accuracy substantially and achieves large-scale mapping with acceptable overhead.

Simultaneous localization and mapping (SLAM) is mainly used to solve the problem of mobile robot navigation in an unknown environment. The loop closure detection is a key step in visual SLAM and plays an important role in building consistent maps and reducing pose cumulative errors. A new loop closure detection method based on the bag of semantic word (BOSW) is proposed in this paper, extracting semantic descriptors by using a neural network pre-trained on the scene recognition data set. Traditionally, the visual bag of word (BoW) model adopts the features such as ORB and SURF which is easily affected by the environment. With advantages of better ability to deal with changes in lighting, the convolutional neural network can extract the abstract features of the image. For these reasons, we proposed a new algorithm that combines the bag of word model and the convolutional neural network. The feature maps, extracted by the VGG16 neural network which is trained on the dataset named Places365 (VGG16-Places365), are used as the semantic descriptors instead of ORB descriptors. The workflow of building the vocabulary of the semantic words, extracting the feature vector of the image and computing the similarity score is presented. The performance of BOSW is evaluated by a comparison study with the traditional visual bag of word model and Inception-based deep learning method which use Inception's fully connected layer output as the feature vector of the picture. Overall, the experiments on New College datasets verified that our mode show stronger generalization and can achieve higher accuracy in the loop closure detection.

A LiDAR-intensity SLAM and loop closure detection method using an intensity cylindrical-projection shape context descriptor

Loop closure detection (LCD) is an important module of visual SLAM (Simultaneous Localization and Mapping), which can be regarded as an image retrieval task. The correct LCD can effectively reduce the cumulative errors during visual SLAM localization and obtain a globally consistent map. The currently common approach to LCD is based on a bag-of-words (BoW) and traditional manually designed features. However, The BoW-based method ignores the spatial location information and semantic information of visual features, and image aliasing will affect the detection accuracy of the algorithm. Meanwhile, the artificially designed features are very sensitive to light changes. This paper proposes a LCD algorithm based on convolutional neural networks (CNN) to extract global and local features. Extensive experiments on six publicly available datasets have shown that the method provides high recall with 100% accuracy in most of the evaluated datasets relative to traditional algorithms and some deep learning-based methods.

Binary GAN based Approach for Unsupervised Loop Closure Detection in Autonomous Unmanned Systems

We address the problems of localization, mapping, and guidance for robots with limited computational resources by combining vision with the metrical information given by the robot odometry. We propose in this article a novel light and robust topometric simultaneous localization and mapping framework using appearance-based visual loop-closure detection enhanced with the odometry. The main advantage of this combination is that the odometry makes the loop-closure detection more accurate and reactive, while the loop-closure detection enables the long-term use of odometry for guidance by correcting the drift. The guidance approach is based on qualitative localization using vision and odometry, and is robust to visual sensor occlusions or changes in the scene. The resulting framework is incremental, real-time, and based on cheap sensors provided on many robots (a camera and odometry encoders). This approach is, moreover, particularly well suited for low-power robots as it is not dependent on the image processing frequency and latency, and thus it can be applied using remote processing. The algorithm has been validated on a Pioneer P3DX mobile robot in indoor environments, and its robustness is demonstrated experimentally for a large range of odometry noise levels.

Loop closure detection with patch-level local features and visual saliency prediction

This paper introduces a novel globally consistent dense RGB-D SLAM system designed to address the challenges of online dense reconstruction under fast camera motion. To achieve robust pose tracking and globally consistent map building, we present a random optimization pose estimation framework that integrates both geometric and photometric information. Furthermore, we propose a voxel revisitation-based loop closure detection scheme that relies solely on depth maps. Innovatively, we introduce a local curvature feature, to address loop closure detection when photometric information is unavailable, efficiently extracting 3D geometric information directly from 2D images. In addition, we leverage voxel revisitation observations to accelerate the search for potential loop closures. This is achieved by maintaining a dynamic active map and detecting voxel outflow or revisits, thereby enhancing loop closure detection efficiency. This method is the first voxel revisitation-based loop closure detection scheme that relies entirely on depth information. Combined with a random optimization pose estimation framework utilizing both geometric and photometric features, it adapts well to fast camera motion, enabling accurate and globally consistent dense map construction. Extensive evaluations on public datasets demonstrate the effectiveness of our method under fast camera motion and highlight its significant advantages over existing methods.

The multi-robot Collaborative SLAM (C-SLAM) has long been a key challenge due to communication limitations, computational complexity, and unreliable loop closure detection. This paper proposes a comprehensive system framework to address these issues. Firstly, the front -end employs a dynamic descriptor matching method based on cross-validation to improve the accuracy of inter-robot loop closure detection. Secondly, a graph sparsification approach using dynamic thresholding is introduced to deal with computational complexity. Thirdly, the back-end utilizes the Graduated Non-Convexity (GNC) solver to optimize the pose map, enhancing the accuracy and robustness of localization and mapping. Finally, compared with existing methods, extensive experiments on public datasets and real-world deployments have shown that the paper’s method demonstrates significant advantages in terms of trajectory accuracy and communication efficiency and validates its practical applicability in resource-constrained multi-robot systems.

Visual Simultaneous Localization and Mapping (SLAM) algorithms relying on cameras have gained significant attention with advancements in computer vision. However, existing approaches often depend on high-performance hardware, limiting their applicability in resource-constrained scenarios. This study addresses indoor planar motion applications, such as service robots and automated guided vehicles (AGV), by proposing a cost-effective Visual-Inertial SLAM(VI-SLAM) framework tailored for low-performance platforms using a low-cost binocular camera and an inertial measurement unit (IMU). A localization module is developed to enhance feature stability through inertial prediction and reprojection cross validation, using grayscale images, depth images, and IMU data. A map correction module is introduced to integrate backend optimized poses, loop closure detection, and depth information, enabling the generation of globally consistent probability grid maps via submap stitching and multi-state depth image conversion. Comprehensive evaluations are conducted on public datasets and real-world environments. The results demonstrate the accuracy, robustness, and feasibility of our approach for low-performance and low-cost devices, highlighting its potential for commercialization and practical applications.