Hybrid self-supervised monocular visual odometry system based on spatio-temporal features

In this paper we consider a Simultaneous Localization and Mapping (SLAM) problem for a moving agent using Visual Odometry (VO) while measuring the range from a set of Ultra Wide Band (UWB) antennas, deployed in unknown position in the environment. The solution approach is based on a switching observer which, under standard working conditions, for each observed UWB, uses a two dimensional Extended Kalman Filter (EKF) providing an estimate of the range and bearing of the observed UWB with respect to the agent. This information is then used in a Robust EKF algorithm which solves the SLAM problem with performances that, even before closing the loop, are comparable to the ones that a VO algorithm (namely ORB-SLAM2) would obtain only after closing the loop. Moreover, a resilient module is added to the algorithm to evaluate the reliability of the position estimate of each observed UWB. When the Visual Odometry is not available, the switching observer uses an auxiliary EKF to provide an estimate of the agent position. This makes the proposed approach robust with respect to several kinds of unmodeled disturbances, like multipath effects, and automatically adapts to sensor failures with resilience (e.g. when Visual Odometry or UWB measurements are not available).

Visual odometry (VO) plays a vital role in Simultaneous Localization and Mapping (SLAM). Most of the current learning-based VO methods utilize Convolutional Neural Network (CNN) as framework. However, CNN is weak in integrating global context information. In the design of loss function, the majority of these methods ignore the restraint relationship between translation and rotation prediction. In this work we propose an end-to-end global-context-aware visual odometry scheme with Epipolar-Geometry-constrained loss function to estimate the relative 6-DoF poses of monocular camera. The proposed scheme designs an augmented-attention-enhanced global context block on top of contextual CNN to learn the long-range dependencies and internal correlation. To overcome the problem of mutual restraint between the translation and rotation errors, an Epipolar-Geometry-constrained loss function is developed to simultaneously improve the prediction accuracy of both translation and rotation. The evaluation results on public datasets and self-collected dataset show that the proposed system outperforms state-of-the-art learning-based VO methods with a large margin.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

To solve the accurate positioning problem of mobile robots, simultaneous localization and mapping (SLAM) or visual odometry (VO) based on visual information are widely used. However, most visual SLAM or VO cannot meet the accuracy requirements in dynamic indoor environments. This paper proposes a robust visual odometry based on deep learning to eliminate feature points matching error. However, when a camera and dynamic objects are in relative motion, the frames of camera will produce ghosting, especially in high-dynamic environments, which bring additional positioning error; in view of this problem, a novel method based on the average optical flow value of the dynamic region is proposed to identify feature points of the ghosting, and then the feature points of the ghosting and dynamic region are removed. After the remaining feature points are matched, we use a non-linear optimization method to calculate the pose. The proposed algorithm is tested on TUM RGB-D dataset, and the results show that our VO improves the positioning accuracy than other robust SLAM or VO and is strongly robust especially in high-dynamic environments.

Pose estimation is critical for mobile robots to fulfill various tasks, such as path following or mapping the environment. This is usually accomplished by simultaneous localization and mapping (SLAM). However, computationally constrained systems, such as planetary rovers, rely on less intensive guidance navigation and control (GNC) solutions generally derived solely from visual odometry (VO), wheel odometry, and the onboard inertial measurement unit. Although providing adequate localization performances, the drift accumulated overtime is not compensated by loop closing capabilities, typical of SLAM. Usually, rovers send surface images to the ground station, and these images are used for multiple purposes, such as scientific and operational planning. The number of images is constrained by the communication bandwidth and power budget. The set of transmitted images can be used as a means to correct the robot’s trajectory in an off-line manner. In this work, a solution is presented to the problem of selecting the optimal set of viewpoints belonging to the planned path from which to capture and transmit images: 1) it guarantees accurate trajectory correction and 2) complies with the maximum number of images that can be transmitted to ground control given the available data budget. To this end, it is proposed: 1) a delocalized/decentralized sensor fusion approach based on pose graph optimization and structure from motion and 2) a strategy to select a minimal set of viewpoints along the trajectory that, given a tentative geometry of the environment and the global path that the rover must follow, minimizes the uncertainty of all the robot poses. Optimal camera viewpoint positions are selected as a function of the planned trajectory, the approximate scene geometry, and the maximum transmittable number of images. The proposed method has been tested on a dataset of stereo-images collected in a representative Martian environment, the ALTEC Mars Terrain Simulator (MTS), with the ExoMars testing rover (ExoTeR—European Space Agency, Paris, France, property). Rover stereo-images ground truth was given with millimetric accuracy by a motion capture (MC) system.

Tracking drift issues resulting from the lack of global positioning constraints prevent visual simultaneous localization and mapping (SLAM) systems from providing accurate pose estimation services, especially in indoor environments. Similar to the function of the Global Navigation Satellite System (GNSS), a new indoor equipment called workshop Measurement Positioning System (wMPS) has been developed to provide millimeter-level global positioning information in the large-volume metrology industrial community but is rarely applied to SLAM solutions. In this paper, we introduce a novel wMPS-SLAM framework that combines 3 degree-of-freedom (DoF) wMPS signals into our monocular pose estimation algorithm, enabling drift-free global 6DoF pose estimation in indoor scenarios. Firstly, a system calibration method is proposed to establish the relationship between the camera and wMPS and fuse the sensor information. Secondly, a variant similarity transformation scheme is employed to jointly estimate the scale factor and extrinsic parameters of the camera frame w.r.t the world frame for estimating the global camera poses. Lastly, a pose graph optimization method that minimizes global position and relative pose errors is utilized to fuse Visual Odometry (VO) estimation and wMPS measurements. Evaluation experiments conducted on two datasets demonstrate that wMPS-SLAM outperforms the state-of-the-art monocular visual solution (ORB-SLAM3) in terms of accuracy. Specifically, the absolute translation error (ATE) of the proposed system is reduced from 28.58mm to 7.018mm in real-world experiments.

Several works have focused on Simultaneous Localization and Mapping (SLAM), which is a topic that has been studied for more than a decade to meet the needs of robots to navigate in an unknown environment. SLAM is an essential perception functionality in several applications, especially in robotics and autonomous vehicles. RGB-D cameras are among the sensors commonly used with recent SLAM algorithms. They provide an RGB image and the associated depth map, making it possible to solve scale drift with less complexity and create a dense 3D environment representation. Many RGB-D SLAM algorithms have been studied and evaluated on publicly available datasets without considering sensor specifications or image acquisition modes that could improve or decrease localization accuracy. In this work, we deal with indoor localization, taking into account the sensor specifications. In this context, our contribution is a deep experimental study to highlight the impact of the sensor acquisition modes on the localization accuracy, and a parametric optimization protocol for a precise localization in a given environment. Furthermore, we apply the proposed protocol to optimize a depth-related parameter of the SLAM algorithm. The study is based on a publicly available dataset in an indoor environment with a depth sensor. The reconstruction results’ analysis is founded on the study of different metrics involving translational and rotational errors. These metrics errors are compared with those obtained with a state-of-the-art stereo vision-based SLAM algorithm.

In this paper we describe VOPT (Visual Odometry by Patch Tracking), a robust algorithm for visual odometry, which is able to operate with sparse or dense maps computed by simultaneous localization and mapping (SLAM) algorithms. By using an iterative multi-scale procedure, VOPT is able to estimate the individual motion, photometric correction and reliability tracking confidence of a set of planar patches. In order to overcome the high computational cost of the patch adjustment, we use a GPU-based least-square solver, achieving real-time performance. The algorithm can also be used as a building block to other procedures for automatic initialization and recovery of 3D scene. Our tests show that VOPT outperforms the well-known PTAMM and the state-of-art ORB-SLAM algorithm in challenging videos using the same input maps.

Simultaneous localization and mapping (SLAM) is a key technique for autonomous navigation of a robot in unknown environments. Recently, many researchers focus on the algorithm optimization of visual-inertial odometry (VIO) as a real-time SLAM method. In this paper, different from pure algorithm research, we propose a novel low-cost hardware solution based on the system on a chip (SoC) to achieve real-time performance and low power consumption of VIO. We design a novel hardware architecture to accelerate a popular VIO algorithm, with the time-costly image feature processing computed on FPGA and floating-point numerical computation been done on ARM. On the algorithm side, we fuse odometry information from the two wheels of the ground robot in a unified optimization method, which results in accurate metric scale estimation of the VIO. The experimental results show that our design and implementation have superior performance for ground robot applications, allowing real-time localization of a robot with a very low-cost hardware for computation in complex environments.

<div><p class="4">Various researches in the field of robotics have made great progress in developing methods to effectively determine the position of robots in unknown environments. The simultaneous localization and mapping (SLAM) task make determining the current position of the robot and performing path mapping possible. In this mapping, solid elements (landmarks) existing in the actual environment are even detected, which indicate that the direction of the robot changes during walking. This scheme provides the implementation analysis of the probabilistic particle filter method, which ensures the correct performance in the controlled actual scene under specific conditions, obtains the non-network connection environment information by storing the data in the temperature value sampling in the CVS file, and monitors the temperature measurement by displaying the heat map. Successful analysis must ensure the robustness of the results obtained when implementing these systems and take into account the feasibility of applying this work to the proposed objectivesd.</p></div>

The current visual simultaneous localization and mapping (SLAM) systems require the use of matched feature point pairs to estimate camera pose and construct environmental maps. Therefore, they suffer from poor performance of the visual feature matchers. To address this problem, a visual SLAM using deep feature matcher is proposed, which is mainly composed of three parallel threads: Visual Odometry, Backend Optimizer and LoopClosing. In the Visual Odometry, the deep feature extractor with convolutional neural networks is utilized for extracting feature points in each image frame. Then, the deep feature matcher is used for obtaining the corresponding feature landmark pairs. Afterwards, a fusion method based on the last and the reference frame is proposed for camera pose estimation. The Backend Optimizer is designed to execute local bundle adjustment for a part of camera poses and landmarks (map points). While LoopClosing, consisting of a lightweight deep loop closure detector and the same matcher as the one used in Visual Odometry is utilized for loop correction based on pose graph. The proposed system has been tested extensively on most of benchmark KITTI odometry dataset. The experimental results show that our system yields better performance than the existing visual SLAM systems. It can not only run in real-time at a speed of 0.08 s per frame, but also reduce estimation error by at least 0.1 m.

Visual odometry (VO) is the process of estimating the ego-motion of an agent sequentially (e.g. vehicle, human, and robot) using only the input of single or multiple cameras attached to it. VO plays an important role in Simultaneous Localization and Mapping (SLAM), which has been widely applied in many fields. The development of VO would greatly promote the transition of visual mapping and navigation to the industry. This paper focuses on direct methods and aims to summarize recent direct algorithms from technical views. Image representation and frame matching strategy are the two key components of VO. This paper is organized based on these two aspects. A qualitative analysis of existing work is then presented. This paper simultaneously serves as a position and tutorial paper.

This paper presents a simultaneous localization,mapping (SLAM) and object tracking(OT) method based on interacting multiple model (IMM) filter to achieve simultaneous estimation of robot and object's trajectories in unknown environments. The proposed approach integrates IMM based object tracking and EKF-based SLAM. Since the implementation of IMM, the method can help the robot to track object and estimate its motion model, which improves the accuracy of object localization. Moreover, this approach can help robot to track the object that have more ability to escape, and improve it practicality. Simulation and real robot experiment results validate the effectiveness of the proposed method in robot and target localization and mapping. The results also show the accuracy of target motion model estimation and the system ability to track maneuvering target.

In the last decades, Simultaneous Localization and Mapping (SLAM) proved to be a fundamental topic in the field of robotics, due to the many applications, ranging from autonomous driving to 3D reconstruction. Many systems have been proposed in literature exploiting a heterogeneous variety of sensors. State-of-the-art methods build their own map from scratch, using only data coming from the equipment of the robot, and not exploiting possible reconstructions of the environment. Moreover, temporary loss of data proves to be a challenge for SLAM systems, as it demands efficient re-localization to continue the localization process. In this paper, we present a SLAM system that exploits additional information coming from mapping services like OpenStreetMaps, hence the name OSM-SLAM, to face these issues. We extend an existing LiDAR-based Graph SLAM system, ART-SLAM, making it able to integrate the 2D geometry of buildings in the trajectory estimation process, by matching a prior OpenStreetMaps map with a single LiDAR scan. Each estimated pose of the robot is then associated with all buildings surrounding it. This association allows to improve localization accuracy, but also to adjust possible mistakes in the prior map. The pose estimates coming from SLAM are then jointly optimized with the constraints associated with the various OSM buildings, which can assume one of the following types: Buildings are always fixed (Prior SLAM); buildings surrounding a robot are movable in chunks, for every scan (Rigid SLAM); and every single building is free to move independently from the others (Non-rigid SLAM). Lastly, OSM maps can also be used to re-localize the robot when sensor data is lost. We compare the accuracy of the proposed system with existing methods for LiDAR-based SLAM, including the baseline, also providing a visual inspection of the results. The comparison is made by evaluating the estimated trajectory displacement using the KITTI odometry dataset. Moreover, the experimental campaign, along with an ablation study on the re-localization capabilities of the proposed system and its accuracy in loop detection-denied scenarios, allow a discussion about how the quality of prior maps influences the SLAM procedure, which may lead to worse estimates than the baseline.

Visual odometry is a core component of many visual navigation systems like visual simultaneous localization and mapping (SLAM). Grid cells have been found as part of the path integration system in the rat's entorhinal cortex, and they provide inputs for place cells in the rat's hippocampus. Together with other cells, they constitute a positioning system in the brain. Some computational models of grid cells based on continuous attractor networks have also been proposed in the computational biology community, and using these models, self-motion information can be integrated to realize dead-reckoning. However, so far few researchers have tried to use these computational models of grid cells directly in robot visual navigation in the robotics community. In this paper, we propose to apply continuous attractor network model of grid cells to integrate the robot's motion information estimated from the vision system, so a biologically inspired visual odometry can be realized. The experimental results show that good dead-reckoning can be achieved for different mobile robots with very different motion velocities using our algorithm. We also implement a full visual SLAM system by simply combining the proposed visual odometry with a quite direct loop closure detection derived from the well-known RatSLAM, and comparable results can be achieved in comparison with RatSLAM.