Visual Odometry with Improved Adaptive Feature Tracking

Recovering rigid registration between successive camera poses lies at the heart of 3D reconstruction, SLAM and visual odometry. Registration relies on the ability to compute discriminative 2D features in successive camera images for determining feature correspondences, which is very challenging in feature-poor environments, i.e. low-texture and/or low-light environments. In this paper, we aim to address the challenge of recovering rigid registration between successive camera poses in feature-poor environments in a Visual Inertial Odometry (VIO) setting. In addition to inertial sensing, we instrument a small aerial robot with an RGBD camera and propose a framework that unifies the incorporation of 3D geometric entities: points, lines, and planes. The tracked 3D geometric entities provide constraints in an Extended Kalman Filtering framework. We show that by directly exploiting 3D geometric entities, we can achieve improved registration. We demonstrate our approach on different texture-poor environments, with some containing only flat texture-less surfaces providing essentially no 2D features for tracking. In addition, we evaluate how the addition of different 3D geometric entities contributes to improved pose estimation by comparing an estimated pose trajectory to a ground truth pose trajectory obtained from a motion capture system. We consider computationally efficient methods for detecting 3D points, lines and planes, since our goal is to implement our approach on small mobile robots, such as drones.

UVMO: Deep unsupervised visual reconstruction-based multimodal-assisted odometry

Visual odometry is an essential component in robot navigation and autonomous driving; however, visual sensors are vulnerable in fast motion or sudden illumination changes. This weakness can be compensated with inertial measurement units (IMUs), which maintain the short-term motion when visual sensing becomes unstable and enhance the quality of estimated motion with inertial information. An omnidirectional multi-view visual odometry (ROVO) has recently demonstrated superior performance and stability with the unceasing feature observation of the omnidirectional setup; however, the shortcomings of visual odometry remain. This letter introduced an omnidirectional visual-inertial odometry system (ROVINS) that could seamlessly integrate the inertial information into the omnidirectional visual odometer algorithm: (a) The soft relative pose constraints from the inertial measurement are first added to the pose optimization formulation, which enables blind motion estimation when all visual features are lost; (b) Using the prediction results from the estimated velocity, the visual features in tracking are initialized, resulting in feature tracking that is more robust to visual disturbances. The experimental results showed that the proposed ROVINS algorithm outperforms the vision-only algorithm by a significant margin.

Using Visual Odometry a robot can track its trajectory using video input. This allows more accurate ego-motion estimation when compared to classical odometry which relies on measurement of wheel motion. The Microsoft Kinect sensor provides 3D imagery, similar to a LASER or LIDAR scanner, which can be used for visual odometry with a single sensor. This diers from usual implementations that require stereo vision with two or more standard image sensors. The system has advantages over a laser scanner in that it provides a video image as well as depth information such that matching using feature detectors such as SIFT or SURF is possible. Visual odometry is performed by matching 3D points that have 2D descriptors. This paper presents and implements a visual odometry system for a mobile robot that utilizes feature detection and tracking combined with a low cost 3D video sensor, Microsoft's Kinect.

Recent direct visual odometry and SLAM algorithms have demonstrated impressive levels of precision. However, they require a photometric camera calibration in order to achieve competitive results. Hence, the respective algorithm cannot be directly applied to an off-the-shelf-camera or to a video sequence acquired with an unknown camera. In this work we propose a method for online photometric calibration which enables to process auto exposure videos with visual odometry precisions that are on par with those of photometrically calibrated videos. Our algorithm recovers the exposure times of consecutive frames, the camera response function, and the attenuation factors of the sensor irradiance due to vignetting. Gain robust KLT feature tracks are used to obtain scene point correspondences as input to a nonlinear optimization framework. We show that our approach can reliably calibrate arbitrary video sequences by evaluating it on datasets for which full photometric ground truth is available. We further show that our calibration can improve the performance of a state-of-the-art direct visual odometry method that works solely on pixel intensities, calibrating for photometric parameters in an online fashion in realtime.

Abstract. Localization is the process of determining the path, position and orientation of a robot in an environment. The data from different sensors such as LiDAR, inertial measurement units (IMU) or cameras are used in the localization process. This task is of fundamental importance to enable robots to navigate autonomously and perform tasks effectively. The robot localization can be performed utilizing different techniques either using hardware or software designs. Visual localization algorithms can be shown as one of the localization techniques that enable robots to determine their position and orientation. In this aspect, visual odometry is one of the mostly used methods. It is a technique that enables robots to determine their positions and movements by analysing sequential images. It tracks features in consecutive images to determine the movement of the camera between those frames which enables the determination of the robot’s position in the environment. The process commonly involves detecting and matching key points or features in the images, such as corners or edges, and then using algorithms to calculate the camera's motion. Visual odometry is useful in environments where Global Navigation Satellite Systems (GNSS) or other external positioning systems cannot operate. In this study, the use of visual odometry is assessed in comparison with the iPad Pro LiDAR and steel tape results to determine the distances between each image-taking point. The iPad and steel tape results were taken as the ground truth and the root mean square values were determined by comparing the algorithm and their results.

This paper proposes a framework for tightly coupled monocular visual-inertial odometry (VIO) via epipolar constraints, which achieves accurate and effective 6 degrees-of-freedom (DOF) state estimation. After extracting and tracking of features, initial values of system state are then estimated for the VIO. The algorithm proposed establishes visual-inertial odometry atop a factor graph, which estimates the pose, velocity and bias of the IMU and the external between camera and IMU, allowing multiple measurements to be captured by different sensors as factors. The performance of the algorithm proposed is testified through the EuRoC datasets, and compared against other state-of-art VIO algorithms. The experimental results indicate that the introduction of epipolar constraints improves the state estimation accuracy effectively.

The YTU dataset and recurrent neural network based visual-inertial odometry

The most significant part of any autonomous intelligent robot is the localization module that gives the robot knowledge about its position and orientation. This knowledge assists the robot to move to the location of its desired goal and complete its task. Visual Odometry (VO) measures the displacement of the robots’ camera in consecutive frames which results in the estimation of the robot position and orientation. Deep Learning, nowadays, helps to learn rich and informative features for the problem of VO to estimate frame-by-frame camera movement. Recent Deep Learning-based VO methods train an end-by-end network to solve VO as a regression problem directly without visualizing and sensing the label of training data in the training procedure. In this paper, a new approach to train Convolutional Neural Networks (CNNs) for the regression problems, such as VO, is proposed. The proposed method first changes the problem to a classification problem to learn different subspaces with similar observations. After solving the classification problem, the problem converts to the original regression problem to solve using the knowledge achieved by solving the classification problem. This approach helps CNN to solve regression problem globally in a local domain learned in the classification step, and improves the performance of the regression module for approximately 10%.

Visual odometry (VO), a method that estimates odometry using visual sensors, is hard to operate successfully with the low-resolution and noisy image sequences. To address this problem, a super-resolution technique is applied to input data before performing VO. Since most conventional super-resolution literature mainly deals with the resolution increment, we present a novel deep neural super-resolution network, which can remove noises as well. The execution time is also taken into account by adjusting the number of CNN layers for a real-time VO. By applying the proposed super-resolution approach, the resolution increases and noises disappear with a suitable speed, hence VO can be performed successfully. Experimental results show that the proposed method improves the VO performance compared with the conventional VO which uses low-resolution and noisy image sequences.

Visual Odometry (VO) is a proven technology for planetary exploration rovers, facilitating their localization with a small error over medium-sized trajectories. However, due to VO's incremental mode of operation, its estimation error accumulates over time, resulting in considerable drift for long trajectories. This paper proposes a global localization method that counters VO drift by matching boulders extracted from overhead and ground images and using them periodically to relocalize the rover and refine VO estimates. The performance of the proposed method is evaluated with the aid of overhead imagery of different resolutions. Experimental results demonstrate that a very terse representation, consisting of approximate boulder locations only, suffices for significantly improving the accuracy of VO over long traverses.

A method that optimizes visual odometry, especially using visual odometry in the scene with absence of features is proposed in this paper. First, in order to estimate the pose of camera when the effect of using feature-based method is not good enough, direct method is implemented as the solution. Second, comparing with traditional visual odometry method, this method takes the environment restriction into consideration. On the one hand, if enough feature points are detected in a frame, feature-based method is to be used; on the other hand, direct method is used in the scene of missing features so that the accuracy of pose estimation could be guaranteed and improved. Finally, the feasibility of this method is proven by a number of experiments conducted on the experiment platform.

Recently, the event camera has become a popular and promising vision sensor in the research of simultaneous localization and mapping and computer vision owing to its advantages: low latency, high dynamic range, and high temporal resolution. As a basic part of the feature-based SLAM system, the feature tracking method using event cameras is still an open question. In this article, we present a novel asynchronous event feature generation and tracking algorithm operating directly on event-streams to fully utilize the natural asynchronism of event cameras. The proposed algorithm consists of an event-corner detection unit, a descriptor construction unit, and an event feature tracking unit. The event-corner detection unit addresses a fast and asynchronous corner detector to extract event-corners from event-streams. For the descriptor construction unit, we propose a novel asynchronous gradient descriptor inspired by the scale-invariant feature transform descriptor, which helps to achieve quantitative measurement of similarity between event feature pairs. The construction of the gradient descriptor can be decomposed into three stages: speed-invariant time surface maintenance and extraction, principal orientation calculation, and descriptor generation. The event feature tracking unit combines the constructed gradient descriptor and an event feature matching method to achieve asynchronous feature tracking. We implement the proposed algorithm in C++ and evaluate it on a public event dataset. The experimental results show that our proposed method achieves improvement in terms of tracking accuracy and real-time performance when compared with the state-of-the-art asynchronous event-corner tracker and with no compromise on the feature tracking lifetime.

In this study, a hierarchical clustering matching (HCM) algorithm is proposed to match features with ambiguity due to repetitive patterns in visual odometry. Visual odometry is a real-time system that estimates the motions of camera setups, in which feature matching is a key step for tracking and relocalization. However, it is still difficult to remove outliers fast and reliably when a high proportion of outliers exist. The proposed HCM algorithm solves this problem by clustering accurate matches hierarchically. Dubious matches excluded from any clusters are removed during the iterations, which finally converges when new clusters no longer generate. Local geometric consistency and descriptor of features are both considered to be a metric to link two clusters using a centroid linkage criterion. Experimental results demonstrate that the proposed method works well on solving the problem mentioned above. Compared to state-of-the-art methods on feature matching, HCM performs much better on efficiency with comparable accuracy.

Aim of intense research in the field computational vision, dense 3D reconstruction achieves an important landmark with first methods running in real time with millimetric precision, using RGBD cameras and GPUs. However, these methods are not suitable for low computational resources. The goal of this work is to show a method of visual odometry using regular cameras, without using a GPU. The proposed method is based on techniques of sparse Structure from Motion (SFM), using data provided by dense 3D reconstruction. Visual odometry is the process of estimating the position and orientation of an agent (a robot, for instance), based on images. This paper compares the proposed method with the odometry calculated by Kinect Fusion. Odometry provided by this work can be used to model a camera position and orientation from dense 3D reconstruction.