Adaptive Essential Matrix Based Stereo Visual Odometry with Joint Forward-Backward Translation Estimation

The most significant part of any autonomous intelligent robot is the localization module that gives the robot knowledge about its position and orientation. This knowledge assists the robot to move to the location of its desired goal and complete its task. Visual Odometry (VO) measures the displacement of the robots’ camera in consecutive frames which results in the estimation of the robot position and orientation. Deep Learning, nowadays, helps to learn rich and informative features for the problem of VO to estimate frame-by-frame camera movement. Recent Deep Learning-based VO methods train an end-by-end network to solve VO as a regression problem directly without visualizing and sensing the label of training data in the training procedure. In this paper, a new approach to train Convolutional Neural Networks (CNNs) for the regression problems, such as VO, is proposed. The proposed method first changes the problem to a classification problem to learn different subspaces with similar observations. After solving the classification problem, the problem converts to the original regression problem to solve using the knowledge achieved by solving the classification problem. This approach helps CNN to solve regression problem globally in a local domain learned in the classification step, and improves the performance of the regression module for approximately 10%.

Motion estimation of a moving vehicle is one of the most important aspects of an autonomous navigation system. With increasing demand of navigation in GPS denied environments, vision-based navigation proves to be a promising area for applications varying from inter-terrestrial to small form factor mobile robotics. In this work, an inertia constrained visual odometry technique has been developed that helps in extracting rotation and translation information over consequent image frames. The inertia is considered here to be a bounding factor that restricts the position of the feature point in the incoming image frame. This constraint is found to reduce the overall outlier percentage and increase the estimation accuracy as compared to the traditional visual odometry pipeline. The proposed scheme is applied on the KITTI dataset logged for stereo camera and is compared with their ground truth poses.

A novel RGB-D visual odometry method for dynamic environment is proposed. Majority of visual odometry systems can only work in static environments, which limits their applications in real world. In order to improve the accuracy and robustness of visual odometry in dynamic environment, a Feature Regions Segmentation algorithm is proposed to resist the disturbance caused by the moving objects. The matched features are divided into different regions to separate the moving objects from the static background. The features in the largest region which belong to the static background are used to estimate the camera pose finally. The effectiveness of our visual odometry method is verified in a dynamic environment of our lab. Furthermore, an exhaustive experimental evaluation is conducted on benchmark datasets including static environments and dynamic environments compared with the state-of-art visual odometry systems. The accuracy comparison results show that the proposed algorithm outperforms those systems in large scale dynamic environments. Our method tracks the camera movement correctly while others failed. In addition, our method can give the same good performances in static environment. Experiments demonstrate that the proposed RGB-D visual odometry can obtain accurate and robust estimation results in dynamic environments.

In this study, a hierarchical clustering matching (HCM) algorithm is proposed to match features with ambiguity due to repetitive patterns in visual odometry. Visual odometry is a real-time system that estimates the motions of camera setups, in which feature matching is a key step for tracking and relocalization. However, it is still difficult to remove outliers fast and reliably when a high proportion of outliers exist. The proposed HCM algorithm solves this problem by clustering accurate matches hierarchically. Dubious matches excluded from any clusters are removed during the iterations, which finally converges when new clusters no longer generate. Local geometric consistency and descriptor of features are both considered to be a metric to link two clusters using a centroid linkage criterion. Experimental results demonstrate that the proposed method works well on solving the problem mentioned above. Compared to state-of-the-art methods on feature matching, HCM performs much better on efficiency with comparable accuracy.

Initiated and financed by the Canadian Space Agency, Micro-Rover Platform with Tooling Arm (MRPTA) project has targeted the development of a robotic system with remote control and autonomous navigation capabilities for testing a large variety of analogous planetary missions. Although developed for planetary exploration, the navigation system is highly suitable to such areas of application as construction and in mining. Autonomous navigation system is capable of moving the platform to predefined position(s)/orientation(s) while continuously mapping the terrain, assessing its traversability and choosing most appropriate path to follow. Designed according to the JAUS (Joint Architecture for Unmanned Systems) framework, the system consists of several interconnected components. Following components are of particular importance: Pose Estimator (PE), Terrain Evaluator (TE), Map Manager (MP), and Path Planner (PP). The PE uses a minimal sensor configuration consisting of an azimuth gyroscope, inclinometer and wheel odometry. The PE provides reliable pose estimates in spite of pronounced slippage by employing extensive use of fused data. The TE uses a nodding laser scanner which continuously sweeps the area in front of the platform and constructs a “traversability grid” (TG) of the surrounding area. Multiple TGs are constantly constructed along the platform motion path are combined into a terrain map maintained by the MP. The PP uses this map to compute a motion path Obtained path is executed by the motion controller and remains current until it is contradicted by the most recent map – at which point it is re-planned. This paper addresses integration issues, lessons learned and also outlines the possible applications for construction and mining.

Visual odometry is critical in visual simultaneous localization and mapping for robot navigation. However, the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight. Herein, a new human visual attention mechanism for point-and-line stereo visual odometry, which is called point-line-weight-mechanism visual odometry (PLWM-VO), is proposed to describe scene features in a global and balanced manner. A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism, where sufficient attention is assigned to position-distinctive objects (sparse features in the environment). Furthermore, the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features. Compared with the state-of-the-art method (ORB-VO), PLWM-VO show a 36.79% reduction in the absolute trajectory error on the Kitti and Euroc datasets. Although the time consumption of PLWM-VO is higher than that of ORB-VO, online test results indicate that PLWM-VO satisfies the real-time demand. The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry, but also quantitatively demonstrates the superiority of the human visual attention mechanism.

The current visual simultaneous localization and mapping (SLAM) systems require the use of matched feature point pairs to estimate camera pose and construct environmental maps. Therefore, they suffer from poor performance of the visual feature matchers. To address this problem, a visual SLAM using deep feature matcher is proposed, which is mainly composed of three parallel threads: Visual Odometry, Backend Optimizer and LoopClosing. In the Visual Odometry, the deep feature extractor with convolutional neural networks is utilized for extracting feature points in each image frame. Then, the deep feature matcher is used for obtaining the corresponding feature landmark pairs. Afterwards, a fusion method based on the last and the reference frame is proposed for camera pose estimation. The Backend Optimizer is designed to execute local bundle adjustment for a part of camera poses and landmarks (map points). While LoopClosing, consisting of a lightweight deep loop closure detector and the same matcher as the one used in Visual Odometry is utilized for loop correction based on pose graph. The proposed system has been tested extensively on most of benchmark KITTI odometry dataset. The experimental results show that our system yields better performance than the existing visual SLAM systems. It can not only run in real-time at a speed of 0.08 s per frame, but also reduce estimation error by at least 0.1 m.

Laparoscopic surgery is a widely used minimally invasive technique to treat a variety of pathologies within the abdominopelvic region. Compared to conventional open surgery, laparoscopic surgery reduces iatrogenic trauma and hospital stay length. Nevertheless, surgeons have a limited field of view during operation because surgical instruments are inserted through the patient's body via small incisions. To improve surgeon's perception, surgical navigation techniques are implemented to improve visualization. A number of surgical navigation systems being developed are integrated with computer algorithms to enhance their efficiency. In this work, we generate novel views, depths, and camera poses of corresponding laparoscopic images to augment our dataset, which can then be used to train and evaluate learning-based computer vision algorithms such as simultaneous localization and mapping (SLAM) and visual odometry. Our existing dataset was generated from porcine tissue images using a laparoscope and an optical tracking system to capture camera poses. We performed tissue surface reconstruction of the dataset via the NeRF-SLAM algorithm. Camera poses were transformed to acquire novel views of RGB laparoscopic images and their corresponding depth images. The results show that the simulated RGB images have an average peak signal-to-noise ratios (PSNR) between 29.17 - 29.79 dB, structural similarity index measure (SSIM) value between 0.6061 - 0.6174, and learned perceptual image patch similarity (LPIPS) value between 0.568 - 0.592 compared to the corresponding input images. This work provides a foundation for the generation of synthetic datasets, which could be useful for learning-based autonomous surgical navigation systems.

Visual Odometry (VO) is vital for the navigation of autonomous systems, providing accurate position and orientation estimates at reasonable costs. While traditional VO methods excel in some conditions, they struggle with challenges like variable lighting and motion blur. Deep learning-based VO, though more adaptable, can face generalization problems in new environments. Addressing these drawbacks, this paper presents a novel hybrid visual odometry (VO) framework that leverages pose-only supervision, offering a balanced solution between robustness and the need for extensive labeling. We propose two cost-effective and innovative designs: a self-supervised homographic pre-training for enhancing optical flow learning from pose-only labels and a random patch-based salient point detection strategy for more accurate optical flow patch extraction. These designs eliminate the need for dense optical flow labels for training and significantly improve the generalization capability of the system in diverse and challenging environments. Our pose-only supervised method achieves competitive performance on standard datasets and greater robustness and generalization ability in extreme and unseen scenarios, even compared to dense optical flow-supervised state-of-the-art methods.

Visual odometry (VO) has been widely used for many purposes in the past decade. However, the three assumptions of VO are not often met in the reality, and therefore, the uncertainty of VO output (the pose of agent) should be estimated for safety’s sake, which can be done suitably by the integrity monitoring. To construct the integrity monitoring framework of VO, the first step is to establish the model of errors of measurements and calculate the fault rate, which has not been found in the literature to our knowledge. In response, this paper aims at establishing the model of errors of spatial points and calculating the fault rate in the stereo VO based on the feature point method. In this work, we describe the principle of stereo VO based on the feature point method in a deep and comprehensive way. The errors and faults of spatial points in stereo VO are defined, distinguished and classified in detail. The error propagation from pixel to spatial point is deduced, and the model of errors of spatial points is constructed. The KITTI odometry dataset is employed to evaluate the fault rate and standard deviation of errors of spatial points. And multiple sets of sensitive analyses are carried out to address the impact of RANdom SAmple Consensus (RANSAC) threshold, RANSAC iterations and operational scenario on spatial point error and fault rate. This paper could be a reference for constructing the integrity monitoring framework of stereo VO based on the feature point method.

An autonomous navigation system for near-Earth spacecraft is described; this system allows determination of the satellite orbit and prediction of its motion parameters. Radio navigation measurements of GLONASS and GPS satellite systems are used for this purpose. The autonomous navigation system is designated for operation on near-Earth orbits which do not go beyond the navigation areas of GLONASS and/or GPS and on orbits with large eccentricity whose apocenter is at a distance of 50–70 thousand km from the Earth's surface. The developed methods and algorithms for orbit determination are based on the application of laws of motion dynamics of a spacecraft directly at processing primary phase measurements of the carrier frequency and code pseudo-range using an extended measurement base. Algorithms for determination of motion parameters of the spacecraft and results of simulation and operation of a model system are presented. The possibility of creation of an onboard autonomous navigation system with precision and reliability higher than those of the ground measuring complex is demonstrated.

Visual odometry (VO) in underwater environments presents significant challenges due to poor visibility and dynamic scene changes, which render conventional (in-air) VO solutions unsuitable for underwater applications. We propose an underwater robust monocular visual odometry (UR-MVO) pipeline tailored for underwater scenarios with feature extraction and matching based on SuperPoint and SuperGlue models, respectively. We enhance the robustness of the feature extractor through field-specific fine-tuning of the SuperPoint model using few-shot unsupervised learning. This tuning was done on real images of underwater scenes in order to enhance its performance in the harsh underwater image conditions. Moreover, we integrate semantic segmentation trained on underwater images into our pipeline to eliminate unreliable features belonging to dynamic objects and background. We evaluated the proposed solution on the Aqualoc dataset, demonstrating higher localization accuracy compared to other SOTA direct and feature-based monocular VO methods like DSO and SVO and also obtained very competitive results compared to more resource-intensive monocular VSLAM approaches with loop closure process like LDSO, UVS, and ORB-SLAM. The results show a high potential for our approach for further applications in underwater exploration and mapping using affordable sensory setups. We publish the code for the benefit of the community https://github.com/be2rlab/UR-MVO

Event-based cameras are bioinspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (three-dimensional (3-D) reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semidense 3-D map of the scene by fusing depth estimates from multiple viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source license to foster research in the emerging topic of event-based simultaneous localization and mapping.

In this paper, we propose a novel Visual Odometry (VO) system using a feature detector and feature matcher based on neural networks. The networks for feature detectors and descriptors learning consists of a conventional CNN for feature detection and description, and a graph neural network (GNN) final feature matching. The learned feature has several advantages over traditional handcrafted features such as being robust to light variation and scale. By applying state-of-the-art deep learning-based feature Matcher-SuperGlue, we developed a new monocular VO framework which can exploit the advantages of deep learning-based feature detector and matcher, which performs better than many other learning-based VO methods.

Enhanced calibration of camera setups for high-performance visual odometry