Fast Point-Line Visual Inertial Odometry based on Improved EDLines Algorithm

Most of the existing visual SLAM schemes rely solely on point or line features to estimate camera trajectory. In some scenes such as texture missing or motion blurring, it’s difficult to find a sufficient number of reliable features, resulting in low positioning accuracy. To extract more features, an RPL-SLAM solution is proposed to extract point features and line features respectively. Ulteriorly, the depth information of RGBD image is used to restore the 3D information of point and line features, improving the accuracy of camera track positioning. RPL-SLAM scheme mainly includes three modules: tracking, local mapping and loop detection. Tracking module extends the application of line feature on the basis of point feature extraction and matching. A SLD line segment extraction algorithm which can eliminate the micro segments and a DBM segment matching algorithm based on word bag are proposed respectively. These two algorithms improve the matching efficiency while ensuring the matching accuracy, and effectively track and locate the camera of each frame. In the local mapping and loop detection module, the Plucker coordinates are applied to express the spatial line and define the re-projection error of the straight line, so that the back-end optimization error model of point-line fusion is unified to solve the instability problem in optimization. RPL-SLAM is tested on TUM RGBD and ICL-NUIM data set respectively, and compared with ORB-SLAM2. The result shows that RPL-SLAM can effectively improve the accuracy of pose estimation and map reconstruction while maintaining real-time performance by fusing point-line features with depth images.

PurposeThis paper aims to enable the unmanned aerial vehicles to inspect the surface condition of wind turbine in close range when the global positioning system signal is not reliable, and further improve its intelligence. So a visual-inertial odometry with point and line features is developed.Design/methodology/approachVisual front-end combining point and line features, as well as its purification strategies, are first presented to improve the robustness of feature tracking in low-textured scene and rapidity of segment detector. Additionally, the inertial measurement is integrated between keyframes as constrain to reduce tracking error existed in visual-only system. Second, the graph-based visual-inertial back-end is constructed. To parameterize line features effectively, the infinite line representation not sensitive to outdoor light is employed, in which Plücker and Cayley are selected for line re-projection and nonlinear optimization. Furthermore, Jacobians of the line re-projection errors are analytically derived for better accuracy.FindingsExperiments are performed in various scenes of the wind farm. The results demonstrate that the tight-coupled visual-inertial odometry with point and line features is more precise on all the samples than conventional algorithms in complex wind farm environments. Additionally, the constructed line feature map can be used in the following research for autonomous navigation.Originality/valueThe proposed visual-inertial odometry works robustly in strong electromagnetic interference, low-textured and illumination-change wind farm.

Generally speaking, visual SLAM systems just based on point features have poor robustness in low-textured scenes, which limits their application. To improve the robustness and accuracy of simultaneous location and mapping in various environments, a multi-scene adaptive visual SLAM system based on RGB-D camera is proposed. We not only use point features in our system, but also introduce line features that are abundant indoors. They are less sensitive to lighting variation and more stable than point features in low-textured scenes. The SLAM system combining point features and line features contains several parts: visual odometry, local mapping, loop closing, full BA and mapping. The experimental results on datasets demonstrate that the performance of our proposed SLAM system is better than state-of-the-art point-based method in various indoor scenes.

In indoor low-texture environments, the point feature-based visual SLAM system has poor robustness and low trajectory accuracy. Therefore, we propose a visual inertial SLAM algorithm based on point-line feature fusion. Firstly, in order to improve the quality of the extracted line segment, a line segment extraction algorithm with adaptive threshold value is proposed. By constructing the adjacent matrix of the line segment and judging the direction of the line segment, it can decide whether to merge or eliminate other line segments. At the same time, geometric constraint line feature matching is considered to improve the efficiency of processing line features. Compared with the traditional algorithm, the processing efficiency of our proposed method is greatly improved. Then, point, line, and inertial data are effectively fused in a sliding window to achieve high-accuracy pose estimation. Finally, experiments on the EuRoC dataset show that the proposed PLI-VINS performs better than the traditional visual inertial SLAM system using point features and point line features.

Image registration is an important basis of image processing, which is of great significance in image mosaicking, target recognition, and change detection. Aiming at the automatic registration problem of multi-angle optical images for ground scenes, a registration method combining point features and line features to register images is proposed. Firstly, the LSD (Line Segment Detector) algorithm is used to extract line features of images. The obtained line segments whose length are less than a given threshold are eliminated by a visual significant algorithm. Then, an affine transform model obtained by estimating a Gaussian mixture model (GMM) is applied to the image to be matched. Lastly, Harris point features are utilized in fine matching to overcome shortages of methods based on line features. In experiments, the proposed algorithm is compared with popular feature-based registration algorithms. The results indicate that the proposed algorithm in this work has obvious advantages in terms of registration accuracy and reliability for optical images acquired at different angles.

The point feature, whose dynamic robustness has been widely studied, dominates in the field of visual simultaneous localization and mapping (SLAM) or visual-inertial odometry (VIO). When discussing a dynamic scene, line features are not given enough attention. This paper proposes a pre-processing step for VIO to reduce the influence of lines upon dynamic objects on system robustness and merges it into a state-of-the-art optimization-based VIO pipeline. First, it is determined whether the line feature is a potential dynamic line based upon the result of the semantic segmentation, optical flow and re-projection error. Then, instead of filtering them out, the information matrixes of these line features in the optimization function is adjusted by a weight-based method according to their tracked size. A simulated challenged visual-inertial dataset is used to evaluate the proposed algorithm against other state-of-the-art methods. The results shows that proposed method can increase robustness to dynamic scenes and make the line-based map neater and more intuitive through avoiding the drawing of dynamic line features during the mapping procedure.

This paper presents a novel approach for estimating the ego-motion of a vehicle in dynamic and unknown environments using tightly-coupled inertial and visual sensors. To improve the accuracy and robustness, we exploit the combination of point and line features to aid navigation. The mathematical framework is based on trifocal geometry among image triplets, which is simple and unified for point and line features. For the fusion algorithm design, we employ the Extended Kalman Filter (EKF) for error state prediction and covariance propagation, and the Sigma Point Kalman Filter (SPKF) for robust measurement updating in the presence of high nonlinearities. The outdoor and indoor experiments show that the combination of point and line features improves the estimation accuracy and robustness compared to the algorithm using point features alone.

The precise measuring of vehicle location has been a critical task in enhancing the autonomous driving in terms of intelligent decision making and safe transportation. Internet of Vehicles ( IoV ) is an important infrastructure in support of autonomous driving, allowing real-time road information exchanging and sharing for localizing vehicles. Global positioning System ( GPS ) is widely used in the traditional IoV system. GPS is unable to meet the key application requirements of autonomous driving due to meter level error and signal deterioration. In this article, we propose a novel solution, named Semi-Direct Monocular Visual-Inertial Odometry using Point and Line Features ( SDMPL-VIO ) for precise vehicle localization. Our SDMPL-VIO model takes advantage of a low-cost Inertial Measurement Unit ( IMU ) and monocular camera, using them as the sensor to acquire the surrounding environmental information. Visual-Inertial Odometry ( VIO ), taking into account both point and line features, is proposed, which is able to deal with both weak texture and dynamic environment. We use a semi-direct method to deal with keyframes and non-keyframes, respectively. Dual sliding window mechanisms can effectively fuse point-line and IMU information. To evaluate our SDMPL-VIO system model, we conduct extensive experiments on both an indoor dataset (i.e., EuRoC) and an outdoor dataset (i.e., KITTI) from the real-world applications, respectively. The experimental results show that the accuracy of SDMPL-VIO proposed by us is better than the mainstream VIO system at present. Especially in the weak texture of the datasets, fast-moving datasets, and other challenging datasets, SDMPL-VIO has a relatively high robustness.

Point-cloud-based technique plays a very significant role in 3D model restoration. In the archaeological application of stone tools, the scale drawing, which is hand-drawn from measured stone tools, is traditionally used. In the scale drawing creation, a base drawing which consists outline and ridge lines is initially drawn from geometric features of shape. After that other lines are extracted from knowledge of making stone tools and are added to the base drawing. It requires special knowledge to extract feature lines from stone tools so that scale drawing is time-consuming. Therefore, if the base drawing is automatically extracted, the working hours are reduced. To overcome this issue, this paper proposes a feature line extraction method using the Mahalanobis distance metric. First, the points on outline are extracted from a point cloud. Then, the surface variation is calculated with a various number of neighbors and thus the potential feature points are detected by the analysis of its surface variation. After that, the potential feature points are thinned towards the highest variation points by using Laplacian smoothing. Then, the thinned feature points are shrunk to the potential feature points. Finally, a feature line is extracted by connecting the nearest thinned feature points locating in the Mahalanobis distance field. To verify our method, the extracted feature lines are compared to the ground truth of base drawing drawn by archaeological illustrators. Our method is applied to stone tools, and we confirm the effectiveness of our method.

The detection of feature lines is important for representing and understanding geometric features of 3D models. In this paper, we introduce a new and robust method for extracting feature lines from unorganized point clouds. We use a one-dimensional truncated Fourier series for detecting feature points. Each point and its neighbors are approximated along the principal directions by using the truncated Fourier series, and the curvature of the point is computed from the approximated curves. The Fourier coefficients are computed by Fast Fourier Transform (FFT). We apply low-pass filtering to remove noise and to compute the curvature of the point robustly. For extracting feature points from the detected potential feature points, the potential feature points are thinned using a curvature weighted Laplacian-like smoothing method. The feature lines are constructed through growing extracted points and then projected onto the original point cloud. The efficiency and robustness of our approach is illustrated by several experimental results.

At present, most of the visual Simultaneous Localization and Mapping (SLAM) systems rely on the surrounding point features to achieve acceptable localization and mapping. However, the number of point features is insufficient in the low-texture environments, so the performance of these SLAM systems will be significantly reduced. In this research, a PLISLAM system that integrates point features, line features and Inertial Measurement Unit (IMU) is proposed to implement high-precision positioning and mapping for dynamic vehicles. Specifically, a state-of-the-art SLAM scheme ORBSLAM3 is built at first. Then, its theoretical formulation is derived step by step to handle the environmental line features and the Bundle Adjustment (BA) is integrated to optimize the data. Finally, the system performance is verified through the EuRoC dataset, the results demonstrate its accuracy could be improved by adding the line features especially in scenes with rich line features.

A method of feature line extraction from triangular mesh model is presented. Firstly, initial feature point is manually specified based on feature points extraction from triangular mesh model. Then begin with initial feature point, Principal Component Analysis (PCA) method is applied to analyze primary direction of special feature region of feature points. Search the feature point which is furthest from the center of mass in the primary direction and it is chosen as subsequent feature point. Iteratively, record the feature point sequence in order. Finally, use three order non-uniform B-spline curve to fit and construct feature line. The experimental results demonstrate that the proposed method in this paper is efficient.

For the visual-inertial system, the key frame loss always emerges in the rotation with the weak texture and strong illumination, which seriously affects the camera positioning accuracy. To solve this problem, a tight-coupling-based visual fusion scheme leveraging line features is proposed in our work, called VinsFusion-Line. It first deeply integrates line features, inertial measurement unit(IMU) pre-integration, and stereo visual observations through a tightly-coupled framework. And the infinitely extended line features provide more geometric and structural information, compared with the base point and line feature tracing methods. Hence, the positioning accuracy is improved greatly. In order to further make the calculation of 3D spatial line features faster and more efficient, Plücker coordinates and straight lines are used to represent the features. In addition, the camera state is optimized by a multi-source cost function that embeds both IMU pre-integration residuals and line feature reprojection errors. This design enables tight-coupled fusion of IMU and binocular visual information, rather than relying solely on point features as in existing methods. Finally, experiments on the public data show that under the strong light and weak texture, the binocular VinsFusion-Line system with infinite extension line in our work has better positioning accuracy than the single-purpose Vins-Fusion system.

Visual odometry algorithms tend to degrade when facing low-textured scenes —from e.g. human-made environments—, where it is often difficult to find a sufficient number of point features. Alternative geometrical visual cues, such as lines, which can often be found within these scenarios, can become particularly useful. Moreover, these scenarios typically present structural regularities, such as parallelism or orthogonality, and hold the Manhattan World assumption. Under these premises, in this work, we introduce MSC-VO, an RGB-D -based visual odometry approach that combines both point and line features and leverages, if exist, those structural regularities and the Manhattan axes of the scene. Within our approach, these structural constraints are initially used to estimate accurately the 3D position of the extracted lines. These constraints are also combined next with the estimated Manhattan axes and the reprojection errors of points and lines to refine the camera pose by means of local map optimization. Such a combination enables our approach to operate even in the absence of the aforementioned constraints, allowing the method to work for a wider variety of scenarios. Furthermore, we propose a novel multi-view Manhattan axes estimation procedure that mainly relies on line features. MSC-VO is assessed using several public datasets, outperforming other state-of-the-art solutions, and comparing favourably even with some SLAM methods.

In the γ radioactive environment, high-energy photons induce degradation of the image sensor, which effects feature detection and tracking in the Visual Inertial Odometry (VIO) algorithm and deteriorates its localization performance. To address this issue, in this work, we propose a monocular VIO method using edge-based point features. To mitigate the effects of radiation noise, firstly, in the image preprocessing module, the median filter is used for real-time image denoising. Secondly, in the data association module, both Shi-Tomasi and edge-based point features are detected. The edge-based point feature is the endpoint or corner point in the salient edge map, which is more robust to radiation noise. Then, the bi-directional motion parallaxes and the RANdom SAmple Consensus (RANSAC) method are exploited to reject outliers. Finally, the point features measurements and Inertial Measurement Unit (IMU) pre-integration measurements are added into a tightly-coupled sliding window optimization VIO framework for localization estimation. The proposed method is verified by synthetic and real γ radioactive environment datasets. The experimental results show that the proposed method achieves more accurate and robust localization than the state-of-the-art VIO approaches in the γ radioactive environments.