UVMO: Deep unsupervised visual reconstruction-based multimodal-assisted odometry

Unsupervised monocular visual odometry with decoupled camera pose estimation

The traditional visual inertial simultaneous localisation and mapping system does not fully consider the dynamic objects in the scene, which can reduce the quality of visual feature point matching. In addition, dynamic objects in the scene can cause illumination changes which reduce the performance of the visual front end and loop closure detection of the system. To address this problem, this study combines 3D light detection and ranging (LiDAR), camera, and inertial measurement units in a tightly coupled manner to estimate the pose of mobile robots, thereby proposing a robust LiDAR visual inertial odometry that can effectively filter out dynamic feature points. In addition, a dynamic feature point detection algorithm with attention mechanism is introduced for target detection and optical flow tracking. In experimental analyses on public datasets and real indoor scenes, the proposed method improved the accuracy and robustness of pose estimation in scenes with dynamic objects and varying illumination compared with traditional methods.

A method that optimizes visual odometry, especially using visual odometry in the scene with absence of features is proposed in this paper. First, in order to estimate the pose of camera when the effect of using feature-based method is not good enough, direct method is implemented as the solution. Second, comparing with traditional visual odometry method, this method takes the environment restriction into consideration. On the one hand, if enough feature points are detected in a frame, feature-based method is to be used; on the other hand, direct method is used in the scene of missing features so that the accuracy of pose estimation could be guaranteed and improved. Finally, the feasibility of this method is proven by a number of experiments conducted on the experiment platform.

SkeletonPose: Exploiting human skeleton constraint for 3D human pose estimation

<abstract><p>For the autonomous and intelligent operation of robots in unknown environments, simultaneous localization and mapping (SLAM) is essential. Since the proposal of visual odometry, the use of visual odometry in the mapping process has greatly advanced the development of pure visual SLAM techniques. However, the main challenges in current monocular odometry algorithms are the poor generalization of traditional methods and the low interpretability of deep learning-based methods. This paper presented a hybrid self-supervised visual monocular odometry framework that combined geometric principles and multi-frame temporal information. Moreover, a post-odometry optimization module was proposed. By using image synthesis techniques to insert synthetic views between the two frames undergoing pose estimation, more accurate inter-frame pose estimation was achieved. Compared to other public monocular algorithms, the proposed approach showed reduced average errors in various scene sequences, with a translation error of $ 2.211\% $ and a rotation error of $ 0.418\; ^{\circ}/100m $. With the help of the proposed optimizer, the precision of the odometry algorithm was further improved, with a relative decrease of approximately 10$ \% $ intranslation error and 15$ \% $ in rotation error.</p></abstract>

The paper proposes a set of visual odometry and terrain reconstruction algorithms for quadruped robot in unstructured environments. Visual odometry can accurately measure the change value of the robot body's 6 degrees of freedom in 3D space. Terrain reconstruction can aid the robot to determine foot location around obstacles. With the visual odometry, the robot can remember the obstacle position even if the obstacles are out of vision. In order to simplify the process, two parts will also use the same sparse stereo visual image processing methods. The detail of the algorithms and the implementation process are introduced briefly. At last, the experiment results show that the algorithms are accurate and robust for quadruped robot navigation.

In this paper, we propose a new scheme based on the Sampson distance (SD) to describe visual feature residuals for visual-inertial odometry (VIO). Unlike the epipolar-constraint-based SD for visual odometry (VO), the proposed SD is formulated based on the perspective projection constraint. We proved in theory that the proposed SD retains the good properties of those earlier SD criteria in the literature of VO and it represents a visual feature residual more accurately than the prevailing transfer distance (TD) in existing VIO methods. We formulate three distance criteria, including TD, reprojection error (RE), and SD, and compared their performances by simulation. The results show that the SD is much more accurate than the TD and it is a very accurate estimate of the gold standard criteria—RE. Based on the SD, we modified VINS-Mono by replacing its TD-based visual residuals with the SD-based residuals and study the SD's efficacy in pose estimation by experiments with several public datasets. The results reveal that the SD-based VINS-Mono has a substantial improvement over the original VINS-Mono in pose estimation accuracy. This indicates that the SD is a better distance criterion than the TD for representing visual feature residuals. The proposed SD may find its applications to broader areas in computer vision and robotics.

To solve the accurate positioning problem of mobile robots, simultaneous localization and mapping (SLAM) or visual odometry (VO) based on visual information are widely used. However, most visual SLAM or VO cannot meet the accuracy requirements in dynamic indoor environments. This paper proposes a robust visual odometry based on deep learning to eliminate feature points matching error. However, when a camera and dynamic objects are in relative motion, the frames of camera will produce ghosting, especially in high-dynamic environments, which bring additional positioning error; in view of this problem, a novel method based on the average optical flow value of the dynamic region is proposed to identify feature points of the ghosting, and then the feature points of the ghosting and dynamic region are removed. After the remaining feature points are matched, we use a non-linear optimization method to calculate the pose. The proposed algorithm is tested on TUM RGB-D dataset, and the results show that our VO improves the positioning accuracy than other robust SLAM or VO and is strongly robust especially in high-dynamic environments.

A pose (relative attitude and position) estimation approach to support proximity operations such as autonomous landing of unmanned air vehicles or spacecraft on-orbit servicing is investigated in this paper. An algorithm is developed based on a feature detection capability using the point cloud data that a flash lidar (light detection and ranging) produces. The feature refers to an edge, a corner, or a distinct geometrical shape of an object. Once features are detected, orthogonal triads that are attached to the object are constructed using them. Pose estimation is then performed based on the orientation of the triads with respect to the lidar sensor frame. Since the algorithm employs detected features, it allows pose estimation of an object from a single set of point cloud data. In contrast, the conventional iterative closest point based algorithms typically require two sets of point cloud data. The estimated pose thus represents the changes in pose between the two data sets, leaving the pose of the object with respect to the sensor still to be determined. This featurebased pose estimation capability performs directly an initial pose acquisition with respect to the sensor. The algorithm may also help reduce the number of data points required for pose estimation since a small, selected feature of an object within the sensor field of view can be employed to estimate the pose. Employing the algorithm developed, pose estimation tests were conducted and their preliminary results are presented, which were obtained using a flash lidar in a laboratory testing environment.

With the rapid development of autonomous vehicle technologies, how to perform high-precision localization in unknown complex outdoor environment has become an important issue. Visual odometry is one of the low-cost and the most widely utilized localization methods. Traditional methods predict relative pose based on the principle of multi-view geometry, which is sensitive to camera parameters and environmental changes. This paper studies deep learning-based methods which can be more robust. A novel end-to-end unsupervised visual odometry framework based on confidence evaluation is proposed. Its process can be divided into two stages. The first is predicting the initial relative pose transformation with the help of confidence mask which is generated by measuring the relative similarity of geometric corresponding regions in associated images. The second is evaluating the confidence of the output pose estimate based on the trajectory geometric consistency and then refining it. Quantitative and qualitative evaluation of the proposed approach on KITTI dataset are presented to demonstrate its effectiveness in improving pose estimation accuracy and robustness.

Visual navigation technology enables the pose of a robot to be estimated and the surrounding environment to be perceived using a vision sensor mounted on the robot. This technology is essential to autonomous driving systems in unmanned mobile vehicles and has been actively researched in visual odometry (VO) and visual simultaneous localization and mapping (vSLAM). Generally, the vision-based navigation algorithms perform data association and pose estimation under the assumption that the brightness of surrounding environments does not change over time and that the scene obtained from vision sensors is static. However, in realistic industrial sites or urban environments, the brightness of the environment varies, and dynamic objects such as workers and cars are present. These conditions may lead to a decline in the reliability and performance of visual navigation. Research on robust visual navigation under environmental variations, such as illumination changes and dynamic circumstances, has sought to solve this problem. This study proposes a state-of-the-art robust visual navigation system that is robust to illumination changes and dynamic environments. Moreover, our analysis and classification is based on the methodology used in each robust visual navigation.

Estimating relative camera pose is the key problem of visual odometry (VO). To achieve better efficiency, sparse key-points are usually relied on for the estimation. Yet, feature extraction and matching are still computationally demanding, hindering the overall VO from real-time processing. Exploiting the superiorities of an FPGA in terms of high efficiency, low power consumption, and low cost, this paper proposes a multiple master-slave FPGA architecture for an SIFT-based stereo VO. The master-slave design enables high reconfiguration for the data throughputs among various modules. These modules include SIFT, matching, pose estimation, and their corresponding controllers. In the SIFT module, hardware implemented image pyramid is proposed, where scales are determined off-line via a minimization approach. Local linear exhausted search (LES) matching is considered for both the stereo and the frame matching. In the pose estimation module, a novel hardware design of deriving closest orthogonal matrix for 3D-3D correspondences of relative pose estimation is proposed. Experimental results show that 33.2 fps can be achieved using KITTI dataset without the need of a large number of hardware resources. The proposed reconfigurable design also facilitates its expansions of adopting CCD cameras as well as developing SLAM and other applications.

Traditional visual SLAM methods are built on the strong assumption that the system operates in static environments, with limited consideration of moving objects. This assumption often leads to significant performance degradation when dynamic elements are present. To mitigate the impact of moving objects and enhance both localization accuracy and mapping quality, we propose a visual SLAM framework that explicitly removes dynamic object interference from the visual odometry and mapping modules. First, we refine the data association process in visual odometry by introducing motion consistency constraints, which reduce incorrect feature matches and thereby improve pose estimation accuracy. At the same time, depth information from RGB-D sensors is used to validate potentially dynamic feature points. Second, within the mapping module, we formulate keyframe selection as a vertex cover problem to ensure the local representativeness of keyframes. This approach not only reduces mapping artifacts but also enables the comprehensive detection and removal of dynamic objects. Finally, experiments conducted on the TUM RGB-D dataset demonstrate that our system achieves higher accuracy, robustness, and stability compared to baseline methods.

In this paper, fast techniques are proposed to achieve real time and robust monocular visual odometry. We apply an iterative 5-point method to estimate instantaneous camera motion parameters in the context of a RANSAC algorithm to cope with outliers efficiently. In our method, landmarks are localized in space using a probabilistic triangulation method utilized to enhance the estimation of the last camera pose. The enhancement is performed by multiple observations of landmarks and minimization of a cost function consisting of epipolar geometry constraints for far landmarks and projective constraints for close landmarks. The performance of the proposed method is demonstrated through application to the challenging KITTI visual odometry dataset.

Tracking drift issues resulting from the lack of global positioning constraints prevent visual simultaneous localization and mapping (SLAM) systems from providing accurate pose estimation services, especially in indoor environments. Similar to the function of the Global Navigation Satellite System (GNSS), a new indoor equipment called workshop Measurement Positioning System (wMPS) has been developed to provide millimeter-level global positioning information in the large-volume metrology industrial community but is rarely applied to SLAM solutions. In this paper, we introduce a novel wMPS-SLAM framework that combines 3 degree-of-freedom (DoF) wMPS signals into our monocular pose estimation algorithm, enabling drift-free global 6DoF pose estimation in indoor scenarios. Firstly, a system calibration method is proposed to establish the relationship between the camera and wMPS and fuse the sensor information. Secondly, a variant similarity transformation scheme is employed to jointly estimate the scale factor and extrinsic parameters of the camera frame w.r.t the world frame for estimating the global camera poses. Lastly, a pose graph optimization method that minimizes global position and relative pose errors is utilized to fuse Visual Odometry (VO) estimation and wMPS measurements. Evaluation experiments conducted on two datasets demonstrate that wMPS-SLAM outperforms the state-of-the-art monocular visual solution (ORB-SLAM3) in terms of accuracy. Specifically, the absolute translation error (ATE) of the proposed system is reduced from 28.58mm to 7.018mm in real-world experiments.