Semi-Direct Monocular Visual-Inertial Odometry Using Point and Line Features for IoV

PurposeThis paper aims to enable the unmanned aerial vehicles to inspect the surface condition of wind turbine in close range when the global positioning system signal is not reliable, and further improve its intelligence. So a visual-inertial odometry with point and line features is developed.Design/methodology/approachVisual front-end combining point and line features, as well as its purification strategies, are first presented to improve the robustness of feature tracking in low-textured scene and rapidity of segment detector. Additionally, the inertial measurement is integrated between keyframes as constrain to reduce tracking error existed in visual-only system. Second, the graph-based visual-inertial back-end is constructed. To parameterize line features effectively, the infinite line representation not sensitive to outdoor light is employed, in which Plücker and Cayley are selected for line re-projection and nonlinear optimization. Furthermore, Jacobians of the line re-projection errors are analytically derived for better accuracy.FindingsExperiments are performed in various scenes of the wind farm. The results demonstrate that the tight-coupled visual-inertial odometry with point and line features is more precise on all the samples than conventional algorithms in complex wind farm environments. Additionally, the constructed line feature map can be used in the following research for autonomous navigation.Originality/valueThe proposed visual-inertial odometry works robustly in strong electromagnetic interference, low-textured and illumination-change wind farm.

In this study, we built a monocular visual inertial odometry (VIO) dataset with a proposed self-calibrating platform for an inertial measurement unit (IMU). For a learning-based method, lack of database is the biggest limitation. One goal of this study is to generate a high-quality VIO database. Meanwhile, a properly calibrated inertial measurement unit (IMU) is critical for improving VIO system accuracy and precision. However, a professional IMU calibration tool is usually expensive and large. The proposed self-calibrating platform shows its benefits in eliminating the manual work in calibration and reducing the cost of the expensive platform, which brings more opportunities to students and researchers. This self-calibrating system was useful for collecting monocular visual inertial odometry datasets while driving. . The calibrated IMU generates 14 numbers at 100Hz: time, 3-axis accelerometer measurements, 3-axis gyroscope measurements, 3 Euler angles, and 4 quaternions. This VIO system is composed of a monocular camera, GoPro Hero 3, an IMU, BNO055, and a GPS, ANT-555. The sensors are mounted on a John Deere Gator™ Full-Size Crossover Utility Vehicle and are operated on an Intel NUC, a mini-size computer. The final dataset contains monocular images, IMU measurements, and GPS outputs.

Most mainstream visual-inertial SLAM systems rely on the point feature method of front-end images for motion estimation and localization. However, the SLAM system based on the point feature method, in the case of weakly textured environments, may not be able to estimate the motion due to the lack of point features. For this reason, the introduction of line features has received extensive attention. Line features in the environment can improve the localization accuracy of SLAM, but the extraction and matching of line features are relatively time-consuming and cannot satisfy real-time requirements. Based on the EDLines algorithm, we propose a fast visual-inertial odometry that fuses point and line features. The improved EDLines algorithm is used to process line features. By adjusting the internal parameters of the algorithm, the extraction time of line features is decreased while the extraction effect is the same. And the line features screening strategy is proposed to filter out invalid short line segments and merge adjacent line segments. We use the more effective line features after screening for graph matching and back-end optimization, which reduces the calculation amount of line features matching and ensures the accuracy. The proposed algorithm has been tested on the public datasets, and the comparison results with PL-VIO and VINS-Mono show that the algorithm in this paper improved real-time performance and localization accuracy.

Tracking the pose of a robot has been gaining importance in the field of Robotics, e.g., paving the way for robot navigation. In recent years, monocular visual–inertial odometry (VIO) is widely used to do the pose estimation due to its good performance and low cost. However, VIO cannot estimate the scale or orientation accurately when robots move along straight lines or circular arcs on the ground. To address the problem, in this paper we take the wheel encoder into account, which can provide us with stable translation information as well as small accumulated errors and momentary slippage errors. By jointly considering the kinematic constraints and the planar moving features, an odometry algorithm tightly coupled with monocular camera, IMU, and wheel encoder is proposed to get robust and accurate pose sensing for mobile robots, which mainly contains three steps. First, we present the wheel encoder preintegration theory and noise propagation formula based on the kinematic mobile robot model, which is the basis of accurate estimation in backend optimization. Second, we adopt a robust initialization method to obtain good initial values of gyroscope bias and visual scale in reality, by making full use of the camera, IMU and wheel encoder measurements. Third, we bound the high computation complexity with a marginalization strategy that conditionally eliminates unnecessary measurements in the sliding window. We implement a prototype and several extensive experiments showing that our system can achieve robust and accurate pose estimation, in terms of the scale, orientation and location, compared with the state-of-the-art.

For the visual-inertial system, the key frame loss always emerges in the rotation with the weak texture and strong illumination, which seriously affects the camera positioning accuracy. To solve this problem, a tight-coupling-based visual fusion scheme leveraging line features is proposed in our work, called VinsFusion-Line. It first deeply integrates line features, inertial measurement unit(IMU) pre-integration, and stereo visual observations through a tightly-coupled framework. And the infinitely extended line features provide more geometric and structural information, compared with the base point and line feature tracing methods. Hence, the positioning accuracy is improved greatly. In order to further make the calculation of 3D spatial line features faster and more efficient, Plücker coordinates and straight lines are used to represent the features. In addition, the camera state is optimized by a multi-source cost function that embeds both IMU pre-integration residuals and line feature reprojection errors. This design enables tight-coupled fusion of IMU and binocular visual information, rather than relying solely on point features as in existing methods. Finally, experiments on the public data show that under the strong light and weak texture, the binocular VinsFusion-Line system with infinite extension line in our work has better positioning accuracy than the single-purpose Vins-Fusion system.

The point feature, whose dynamic robustness has been widely studied, dominates in the field of visual simultaneous localization and mapping (SLAM) or visual-inertial odometry (VIO). When discussing a dynamic scene, line features are not given enough attention. This paper proposes a pre-processing step for VIO to reduce the influence of lines upon dynamic objects on system robustness and merges it into a state-of-the-art optimization-based VIO pipeline. First, it is determined whether the line feature is a potential dynamic line based upon the result of the semantic segmentation, optical flow and re-projection error. Then, instead of filtering them out, the information matrixes of these line features in the optimization function is adjusted by a weight-based method according to their tracked size. A simulated challenged visual-inertial dataset is used to evaluate the proposed algorithm against other state-of-the-art methods. The results shows that proposed method can increase robustness to dynamic scenes and make the line-based map neater and more intuitive through avoiding the drawing of dynamic line features during the mapping procedure.

The Internet of Vehicles (IoV) marks a revolutionary leap in transportation, integrating vehicles with the Internet to enhance convenience, intelligence, and efficiency. As IoV applications continue to expand, a spectrum of challenges and untapped opportunities emerge. Among these, the security of IoV stands out as a critical issue in transportation systems, given its direct impact on road safety. To this end, this survey conducts an in-depth analysis of IoV security challenges and introduces a distinctive approach by systematically categorizing threats into inside-vehicle and outside-vehicle domains, providing a comprehensive understanding of the full spectrum of risks. Meanwhile, we examine defense mechanisms designed to counter these threats and explore proactive strategies to enhance IoV security. In addition, this study explores emerging protection techniques, such as AI-driven intrusion detection, blockchain-based trust management, and 5G-enabled secure routing, demonstrating how these technologies can be effectively integrated to safeguard IoV systems. Furthermore, this survey provides actionable recommendations and forward-looking research directions to guide the development of real-world implementations, standardized procedures, and regulatory frameworks, benefiting stakeholders, policymakers, and researchers. As IoV continues its rapid evolution, these insights offer a comprehensive roadmap to strengthen IoV security and contribute to a safer, more resilient intelligent transportation ecosystem.

This paper presents a novel approach for estimating the ego-motion of a vehicle in dynamic and unknown environments using tightly-coupled inertial and visual sensors. To improve the accuracy and robustness, we exploit the combination of point and line features to aid navigation. The mathematical framework is based on trifocal geometry among image triplets, which is simple and unified for point and line features. For the fusion algorithm design, we employ the Extended Kalman Filter (EKF) for error state prediction and covariance propagation, and the Sigma Point Kalman Filter (SPKF) for robust measurement updating in the presence of high nonlinearities. The outdoor and indoor experiments show that the combination of point and line features improves the estimation accuracy and robustness compared to the algorithm using point features alone.

In the γ radioactive environment, high-energy photons induce degradation of the image sensor, which effects feature detection and tracking in the Visual Inertial Odometry (VIO) algorithm and deteriorates its localization performance. To address this issue, in this work, we propose a monocular VIO method using edge-based point features. To mitigate the effects of radiation noise, firstly, in the image preprocessing module, the median filter is used for real-time image denoising. Secondly, in the data association module, both Shi-Tomasi and edge-based point features are detected. The edge-based point feature is the endpoint or corner point in the salient edge map, which is more robust to radiation noise. Then, the bi-directional motion parallaxes and the RANdom SAmple Consensus (RANSAC) method are exploited to reject outliers. Finally, the point features measurements and Inertial Measurement Unit (IMU) pre-integration measurements are added into a tightly-coupled sliding window optimization VIO framework for localization estimation. The proposed method is verified by synthetic and real γ radioactive environment datasets. The experimental results show that the proposed method achieves more accurate and robust localization than the state-of-the-art VIO approaches in the γ radioactive environments.

State of the art visual-inertial odometry approaches suffer from the requirement of high gradients and sufficient visual texture. Even direct photometric approaches select a subset of the image with high-gradient areas and ignore smooth gradients or generally low-textured areas. In this work, we show that taking all image information (i.e. every single pixel) enables visual-inertial odometry even on areas with very low texture and smooth gradients, inherently interpolating and estimating the scene with no texture based on its informative surrounding. This information propagation is only possible as we estimate all states and their uncertainties (robot pose, extrinsic sensor calibration, and scene depth) jointly in a fully dense filter framework. Our complexity reduction approach enables real-time execution despite the large size of the state vector. Compared to our previous basic feasibility study on this topic, this work includes higher order covariance propagation and improved state handling for a significant performance gain, thorough comparisons to state-of-the-art algorithms, larger mapping components with uncertainty, self-calibration capability, and real-data tests.

At present, most of the visual Simultaneous Localization and Mapping (SLAM) systems rely on the surrounding point features to achieve acceptable localization and mapping. However, the number of point features is insufficient in the low-texture environments, so the performance of these SLAM systems will be significantly reduced. In this research, a PLISLAM system that integrates point features, line features and Inertial Measurement Unit (IMU) is proposed to implement high-precision positioning and mapping for dynamic vehicles. Specifically, a state-of-the-art SLAM scheme ORBSLAM3 is built at first. Then, its theoretical formulation is derived step by step to handle the environmental line features and the Bundle Adjustment (BA) is integrated to optimize the data. Finally, the system performance is verified through the EuRoC dataset, the results demonstrate its accuracy could be improved by adding the line features especially in scenes with rich line features.

The application of unmanned vehicles in civilian and military fields is increasingly widespread. Traditionally, unmanned vehicles primarily rely on Global Positioning Systems (GPSs) for positioning; however, GPS signals can be limited or completely lost in conditions such as building obstructions, indoor environments, or electronic interference. In addition, countries are actively developing GPS jamming and deception technologies for military applications, making precise positioning and navigation of unmanned vehicles in GPS-denied or constrained environments a critical issue that needs to be addressed. In this work, authors propose a method based on Visual–Inertial Odometry (VIO), integrating the extended Kalman filter (EKF), an Inertial Measurement Unit (IMU), optical flow, and feature matching to achieve drone localization in GPS-denied environments. The proposed method uses the heading angle and acceleration data obtained from the IMU as the state prediction for the EKF, and estimates relative displacement using optical flow. It further corrects the optical flow calculation errors through IMU rotation compensation, enhancing the robustness of visual odometry. Additionally, when re-selecting feature points for optical flow, it combines a KAZE feature matching technique for global position correction, reducing drift errors caused by long-duration flight. The authors also employ an adaptive noise adjustment strategy that dynamically adjusts the internal state and measurement noise matrices of the EKF based on the rate of change in heading angle and feature matching reliability, allowing the drone to maintain stable positioning in various flight conditions. According to the simulation results, the proposed method is able to effectively estimate the flight trajectory of drones without GPS. Compared to results that rely solely on optical flow or feature matching, it significantly reduces cumulative errors. This makes it suitable for urban environments, forest areas, and military applications where GPS signals are limited, providing a reliable solution for autonomous navigation and positioning of drones.

Although vehicular global positioning system (GPS) has been widely applied in many traffic scenarios, it is far from achieving lane-level positioning due to its low accuracy. Existing cooperative positioning (CP) methods have improved vehicular positioning accuracy to varying degrees, which still have challenges in further improving the system’s robustness and security. In this article, we propose a novel framework of blockchain-enabled Internet of Vehicles (IoV) with CP for improving vehicular GPS positioning accuracy, system robustness, and security. First, a self-positioning correction scheme for the intelligent vehicles is proposed to improve their positioning accuracy, which uses the multitraffic signs as benchmarks to correct the vehicular position (given by GPS) by deep neural network (DNN) algorithm. We further design a multi-intelligent vehicle positioning error sharing model to reduce GPS positioning error of common vehicles (CoVs) in the same segment or area. In addition, to realize information sharing between vehicles and ensure system security, the connections among intelligent vehicles, CoVs, and roadside units are built by proposing a blockchain-enabled architecture that includes IoV subsystem and blockchain subsystem, where the corresponding mechanism of the information choosing, information sharing, and penalty is designed. Extensive simulation results show the accuracy, robustness, and security of our proposal in terms of vehicular positioning, information transferring, and sharing.

Most of the existing visual SLAM schemes rely solely on point or line features to estimate camera trajectory. In some scenes such as texture missing or motion blurring, it’s difficult to find a sufficient number of reliable features, resulting in low positioning accuracy. To extract more features, an RPL-SLAM solution is proposed to extract point features and line features respectively. Ulteriorly, the depth information of RGBD image is used to restore the 3D information of point and line features, improving the accuracy of camera track positioning. RPL-SLAM scheme mainly includes three modules: tracking, local mapping and loop detection. Tracking module extends the application of line feature on the basis of point feature extraction and matching. A SLD line segment extraction algorithm which can eliminate the micro segments and a DBM segment matching algorithm based on word bag are proposed respectively. These two algorithms improve the matching efficiency while ensuring the matching accuracy, and effectively track and locate the camera of each frame. In the local mapping and loop detection module, the Plucker coordinates are applied to express the spatial line and define the re-projection error of the straight line, so that the back-end optimization error model of point-line fusion is unified to solve the instability problem in optimization. RPL-SLAM is tested on TUM RGBD and ICL-NUIM data set respectively, and compared with ORB-SLAM2. The result shows that RPL-SLAM can effectively improve the accuracy of pose estimation and map reconstruction while maintaining real-time performance by fusing point-line features with depth images.

The mobility status of self and surrounding vehicles provides important information to various tasks in autonomous driving (AD) and intelligent transportation system (ITS). Accordingly, a precise, stable, and robust mobility tracking framework is essential. Compared with self-tracking that relies only on mobility observations from onboard sensors [e.g., global positioning system (GPS), inertial measurement unit (IMU), and camera], cooperative tracking markedly increases the precision and reliability of the mobility information by integrating observations from roadside units (RSUs) and nearby vehicles through vehicle-to-everything (V2X) communications in the Internet of Vehicles (IoV). Nevertheless, cooperative tracking can be quite vulnerable if there are malicious users sending bogus observations in the cooperative network. In this article, we present a malicious user detection framework, which includes two sequential detection algorithms and a secure mobility data exchange and fusion model to detect and remove bogus mobility information and integrate proposed detection algorithms with previous data fusion algorithms, which secures the cooperative mobility tracking in AD, ITS. Simulations validate the effectiveness and robustness of the proposed framework under different types of attacks.