Salient Sparse Visual Odometry With Pose-Only Supervision

Recent visual odometry (VO) methods incorporating geometric algorithm into deep-learning architecture have shown outstanding performance on the challenging monocular VO task. Despite encouraging results are shown, previous methods ignore the requirement of generalization capability under noisy environment and various scenes. To address this challenging issue, this work first proposes a novel optical flow network (PANet). Compared with previous methods that predict optical flow as a direct regression task, our PANet computes optical flow by predicting it into the discrete position space with optical flow probability volume, and then converting it to optical flow. Next, we improve the bundle adjustment module to fit the self-supervised training pipeline by introducing multiple sampling, ego-motion initialization, dynamic damping factor adjustment, and Jacobi matrix weighting. In addition, a novel normalized photometric loss function is advanced to improve the depth estimation accuracy. The experiments show that the proposed system not only achieves comparable performance with other state-of-the-art self-supervised learning-based methods on the KITTI dataset, but also significantly improves the generalization capability compared with geometry-based, learning-based and hybrid VO systems on the noisy KITTI and the challenging outdoor (KAIST) scenes.

Recovering rigid registration between successive camera poses lies at the heart of 3D reconstruction, SLAM and visual odometry. Registration relies on the ability to compute discriminative 2D features in successive camera images for determining feature correspondences, which is very challenging in feature-poor environments, i.e. low-texture and/or low-light environments. In this paper, we aim to address the challenge of recovering rigid registration between successive camera poses in feature-poor environments in a Visual Inertial Odometry (VIO) setting. In addition to inertial sensing, we instrument a small aerial robot with an RGBD camera and propose a framework that unifies the incorporation of 3D geometric entities: points, lines, and planes. The tracked 3D geometric entities provide constraints in an Extended Kalman Filtering framework. We show that by directly exploiting 3D geometric entities, we can achieve improved registration. We demonstrate our approach on different texture-poor environments, with some containing only flat texture-less surfaces providing essentially no 2D features for tracking. In addition, we evaluate how the addition of different 3D geometric entities contributes to improved pose estimation by comparing an estimated pose trajectory to a ground truth pose trajectory obtained from a motion capture system. We consider computationally efficient methods for detecting 3D points, lines and planes, since our goal is to implement our approach on small mobile robots, such as drones.

In this paper, we propose UnOS, an unified system for unsupervised optical flow and stereo depth estimation using convolutional neural network (CNN) by taking advantages of their inherent geometrical consistency based on the rigid-scene assumption. UnOS significantly outperforms other state-of-the-art (SOTA) unsupervised approaches that treated the two tasks independently. Specifically, given two consecutive stereo image pairs from a video, UnOS estimates per-pixel stereo depth images, camera ego-motion and optical flow with three parallel CNNs. Based on these quantities, UnOS computes rigid optical flow and compares it against the optical flow estimated from the FlowNet, yielding pixels satisfying the rigid-scene assumption. Then, we encourage geometrical consistency between the two estimated flows within rigid regions, from which we derive a rigid-aware direct visual odometry (RDVO) module. We also propose rigid and occlusion-aware flow-consistency losses for the learning of UnOS. We evaluated our results on the popular KITTI dataset over 4 related tasks, \ie stereo depth, optical flow, visual odometry and motion segmentation.

Addressing on monocular visual odometry problem, this paper presents a novel end-to-end network for estimation of camera ego-motion. The network learns the latent space of optical flow (OF) and models sequential dynamics so that the motion estimation is constrained by the relations between sequential images. We compute the OF field of consecutive images and extract the latent OF representation in a self-encoding manner. A Recurrent Neural Network is then followed to examine the OF changes, i.e., to conduct sequential learning. The extracted sequential OF latent space is used to compute the regression of the 6-dimensional pose vector. Particularly, we separately train the encoder in an unsupervised manner. By this means, we avoid non-convergence during the training of the whole network and allow more generalized and effective feature representation. Substantial experiments have been conducted on KITTI and Malaga datasets, and the results demonstrate that our model outperforms most learning-based VO approaches.

The localization system is the most important part of the overall drone navigation system. The Global Positioning System (GPS) or Global Navigation Satellite System (GNSS) is the main device commonly used in a drone. However, under certain conditions, GPS or GNSS may not function optimally, such as in situations of signal jamming or enclosed environments. This paper implemented a new approach to address this issue by combining GNSS data with Visual Odometry (VO) through Machine Learning (ML) methods. The followed process consists of three main stages. First, performing speed and orientation estimation using VO. Second, performing left and right feature separation on the images to generate a more stable and robust estimation of speed and rotation. Third, refining speed and orientation estimation by integrating GNSS data through ML-based data fusion. The proposed method strives to enhance drone localization accuracy, despite disruptions or unavailability of GNSS signals. The research results indicate that the introduced method significantly reduces Absolute Translation Error (ATE) compared to utilizing VO or GNSS separately. The average ATE produced reached 4.38 m and an orientation of 8.26°, indicating that this data fusion approach provides a significant improvement in drone localization accuracy, making it reliable in operational scenarios with limited GNSS signals.

Accurate orientation estimation is a crucial component of 3D molecular structure reconstruction, both in single-particle cryo-electron microscopy (cryo-EM) and in the increasingly popular field of cryo-electron tomography (cryo-ET). The dominant approach, which involves searching for the orientation that maximizes cross-correlation relative to given templates, is sub-optimal, particularly under low signal-to-noise conditions. In this work, we propose a Bayesian framework for more accurate and flexible orientation estimation, with the minimum mean square error (MMSE) estimator serving as a key example. Through simulations, we demonstrate that the MMSE estimator consistently outperforms the cross-correlation-based method, especially in challenging low signal-to-noise scenarios, and we provide a theoretical framework that supports these improvements.When incorporated into iterative refinement algorithms in the 3D reconstruction pipeline, the MMSE estimator markedly improves reconstruction accuracy, reduces model bias, and enhances robustness to the “Einstein from Noise” artifact. Crucially, we demonstrate that orientation estimation accuracy has a decisive effect on downstream structural heterogeneity analysis. In particular, integrating the MMSE-based pose estimator into frameworks for continuous heterogeneity recovery yields accuracy improvements approaching those obtained with ground-truth poses, establishing MMSE-based pose estimation as a key enabler of high-fidelity conformational landscape reconstruction. These findings indicate that the proposed Bayesian framework could substantially advance cryo-EM and cryo-ET by enhancing the accuracy, robustness, and reliability of 3D molecular structure reconstruction, thereby facilitating deeper insights into complex biological systems.

<abstract><p>For the autonomous and intelligent operation of robots in unknown environments, simultaneous localization and mapping (SLAM) is essential. Since the proposal of visual odometry, the use of visual odometry in the mapping process has greatly advanced the development of pure visual SLAM techniques. However, the main challenges in current monocular odometry algorithms are the poor generalization of traditional methods and the low interpretability of deep learning-based methods. This paper presented a hybrid self-supervised visual monocular odometry framework that combined geometric principles and multi-frame temporal information. Moreover, a post-odometry optimization module was proposed. By using image synthesis techniques to insert synthetic views between the two frames undergoing pose estimation, more accurate inter-frame pose estimation was achieved. Compared to other public monocular algorithms, the proposed approach showed reduced average errors in various scene sequences, with a translation error of $ 2.211\% $ and a rotation error of $ 0.418\; ^{\circ}/100m $. With the help of the proposed optimizer, the precision of the odometry algorithm was further improved, with a relative decrease of approximately 10$ \% $ intranslation error and 15$ \% $ in rotation error.</p></abstract>

Human body orientation estimation from microinertial/magnetic sensor units is highly important for synthetic environments, robotics, and other human-computer interaction applications. In practice, the main challenge is how to deal with linear acceleration interference and magnetic disturbance which always cause significant attitude-estimation errors. In this paper, we present a novel quaternion-based Kalman filter with vector selection scheme for accurate human body orientation estimation using an inertial/magnetic sensor unit. In the proposed algorithm, the gyroscope measurement is used as an input to construct the linear process equation, and the accelerometer and magnetometer measurements are manipulated to establish the linear pseudomeasurement equation. A linear Kalman filter is then deployed to estimate the body orientation. In the Kalman filter framework, a vector selection scheme is designed to protect the algorithm against undesirable conditions such as temporary intensive movement and magnetic disturbance and enable it to acquire more accurate orientation estimation. The experimental results have shown that the proposed algorithm can provide accurate attitude estimations with regard to the ground truth.

Visual odometry systems are usually categorized as feature-based visual odometry systems and direct visual odometry systems. The feature-based visual odometry systems spend time on calculating descriptors while direct visual odometry systems directly calculate residuals. The optimization of direct visual odometry systems is sensitive with initial states while feature-based visual odometry systems did not suffer from this problem. This paper presents a features assisted stereo direct visual odometry system with virtual wide field-of-view tracking. It combines the advantages of feature-based methods and direct methods. It is achieved by incorporating a feature assisted predicting module and a virtual wide field-of-view tracking module into stereo direct sparse odometry (SDSO) framework. The feature-based matching is combined in direct tracking module for efficient and robust data association. Furthermore, before data association, a virtual wide field-of-view inverse depth frame is created in case of tracking lost in aggressive rotating conditions. Evaluation results on the EuRoC datasets and the KITTI dataset show that, with the aid of the new tracking module, the stereo visual odometry becomes more robust and more accurate.

Intelligent visual localization of wireless capsule endoscopes enhanced by color information

This paper presents an innovative visual odometry system for fast motion estimation of autonomous space exploration vehicles. In order to improve a vehicle position recovery, the use of accelerometers and gyros is usually supported by wheels odometry. The visual system can improve the reliability of the overall rover navigation system when used in conjunction with a classical odometry system, usually affected by drift or sliding problems. The visual odometry system is based on artificial vision and reconstructs the rover position through the analysis of sequences of images acquired by two stereo cameras. The proposed algorithm is based on stereo geometry and discrete optical ow computation. From stereo geometry it is possible to locate the image points in the three dimensional environment, while from the optical ow the relative motion of the camera with respect to these points is computed. Moreover, in order to reduce the computational burden usually associated with the optical ow computation, it has been decided to first search the images for features and then to compute the discrete optical ow only associated to them. The features detection and tracking is done by using the KLT algorithm. Tests made with low resolution cameras, showing encouraging results, are presented.

Precise and robust localization for micro Unmanned Aerial Vehicles (UAVs) in GNSS-denied environments is hindered by the lack of diverse datasets and the limited real-world performance of existing visual matching methods. To address these gaps, we introduce two contributions: (1) the SatLoc dataset, a new benchmark featuring synchronized, multi-source data from varied real-world scenarios tailored for UAV-to-satellite image matching, and (2) SatLoc-Fusion, a hierarchical localization framework. Our proposed pipeline integrates three complementary layers: absolute geo-localization via satellite imagery using DinoV2, high-frequency relative motion tracking from visual odometry with XFeat, and velocity estimation using optical flow. An adaptive fusion strategy dynamically weights the output of each layer based on real-time confidence metrics, ensuring an accurate and self-consistent state estimate. Deployed on a 6 TFLOPS edge computer, our system achieves real-time operation at over 2 Hz, with an absolute localization error below 15 m and effective trajectory coverage exceeding 90%, demonstrating state-of-the-art performance. The SatLoc dataset and fusion pipeline provide a robust and comprehensive baseline for advancing UAV navigation in challenging environments.

In this paper, we propose a novel algorithm for the optical flow estimation of a moving object by using the motion blur in a scene. The proposed technique is applied to the visual tracking system for the object moving on the 2D plane that is perpendicular to the axis of lens of a camera. To estimate the motion parameters, we use the motion blur, which has been treated as noises under the conventional method. And we use the feature of motion blur in the frequency domain. The first step is the estimation of orientation of motion, which is achieved by computing the log power spectrum of a local image patch, and operating the coordinate transformation. The second step is the estimation of velocity by using the cepstral analysis. It is achieved by detecting the period of sine function relating to the length of motion blur. In order to evaluate the feasibility of the proposed algorithm, we investigate the accuracy of the estimated optical flow with artificial blurred images and real images. And we apply the proposed algorithm to the tracking vision system, which consists of a camera performing the pan motion, a video processing board, and a standard PC. Through some experiments, we show that the proposed technique is advantageous to the high-speed-tracking system.

It has been a challenging task to estimate optical flow for videos in which either foreground or background exhibits remarkable motion information (i.e., large displacement), or those with insufficient resolution due to artifacts like motion blur or noise. We present a novel optical flow algorithm, which approaches the above problem as solving the task of energy minimization, which exploits image data and smoothness terms at the superpixel level. Our proposed method can be considered as an extended mean-shift algorithm, which advances color and gradient information of superpixels across consecutive frames with smoothness guarantees. Since we do not require assumptions of linearlization during optimization (as standard optical flow approaches do), we are able to alleviate local minimum problems and thus produce improved estimation results. Empirical results on the MPI-Sintel video dataset verify the effectiveness of our proposed method.

This study addresses challenges in sensor fusion for accurate and robust joint orientation estimation in human movement analysis using wearable inertial measurement units (IMUs). A magnetometer-free refined Kalman filter (KF) approach is presented and validated to address various indoor environmental constraints and challenges posed by human movement. These include variability in motion and dynamics, as well as magnetic disturbances caused by ferromagnetic materials or electronic interferences. Our proposed approach utilizes a Kalman-filter-based framework that analyzes the accelerometer’s alignment with the Earth’s frame to estimate orientation and correct gyroscope readings, eliminating reliance on magnetometer inputs. The algorithm was tested on both controlled robotic movements and real-world upper-limb-motion-monitoring scenarios. First, a comparative analysis was conducted on the double-stage Kalman filter (DSKF) and complementary filter using the collected robot motion encoder data. The results demonstrated superior performance in orientation estimation, particularly in yaw measurements, where the proposed method significantly improved accuracy. It achieved a lower root mean square error (RMSE = ) and mean absolute error (MAE = ), outperforming both the DSKF and complementary filter approaches. Additionally, the study’s findings were validated against a standard motion capture system, revealing error metrics within generally acceptable ranges ( of the joint range of motion [ROM]) and strong correlation coefficients (). However, some deviations were observed during complex motion cycle intervals, highlighting opportunities for further refinement. These findings suggest that the proposed approach presents a promising alternative for human joint orientation estimation in industrial settings with magnetic distortions.