A SLAM System Based on RGBD Image and Point-Line Feature

In mainstream simultaneous localization and mapping (SLAM) algorithms, feature points are commonly utilized to represent image features. However, the quantity and quality of these feature points are contingent upon the environmental texture, lighting conditions, and motion speed. Although existing algorithms enhance adaptability by extracting point-line features simultaneously, the presence of trivial short lines resulting from environmental noise and object occlusion can adversely affect system robustness. Therefore, in this study, we propose a line feature fusion strategy along with a model incorporating an adaptive length suppression parameter for line features. A new line feature residual model is defined, and the mathematical analytical form of line feature Jacobian matrix is derived in detail. Additionally, the point features are organized into a lattice structure and utilized to construct a global pointcloud map in a dedicated thread, aiming to enhance the semantic comprehension of environmental information. Finally, our algorithm is compared against state-of-the-art algorithms on the publicly available datasets TUM RGB-D and ICL-NUIM. Through quantitative trajectory error analysis and qualitative trajectory effect and mapping quality analysis, the final results indicate that the algorithm proposed in this paper achieves superior positioning accuracy and mapping quality, enabling robust 3D reconstruction of indoor scenes.

In indoor low-texture environments, the point feature-based visual SLAM system has poor robustness and low trajectory accuracy. Therefore, we propose a visual inertial SLAM algorithm based on point-line feature fusion. Firstly, in order to improve the quality of the extracted line segment, a line segment extraction algorithm with adaptive threshold value is proposed. By constructing the adjacent matrix of the line segment and judging the direction of the line segment, it can decide whether to merge or eliminate other line segments. At the same time, geometric constraint line feature matching is considered to improve the efficiency of processing line features. Compared with the traditional algorithm, the processing efficiency of our proposed method is greatly improved. Then, point, line, and inertial data are effectively fused in a sliding window to achieve high-accuracy pose estimation. Finally, experiments on the EuRoC dataset show that the proposed PLI-VINS performs better than the traditional visual inertial SLAM system using point features and point line features.

Most mainstream visual-inertial SLAM systems rely on the point feature method of front-end images for motion estimation and localization. However, the SLAM system based on the point feature method, in the case of weakly textured environments, may not be able to estimate the motion due to the lack of point features. For this reason, the introduction of line features has received extensive attention. Line features in the environment can improve the localization accuracy of SLAM, but the extraction and matching of line features are relatively time-consuming and cannot satisfy real-time requirements. Based on the EDLines algorithm, we propose a fast visual-inertial odometry that fuses point and line features. The improved EDLines algorithm is used to process line features. By adjusting the internal parameters of the algorithm, the extraction time of line features is decreased while the extraction effect is the same. And the line features screening strategy is proposed to filter out invalid short line segments and merge adjacent line segments. We use the more effective line features after screening for graph matching and back-end optimization, which reduces the calculation amount of line features matching and ensures the accuracy. The proposed algorithm has been tested on the public datasets, and the comparison results with PL-VIO and VINS-Mono show that the algorithm in this paper improved real-time performance and localization accuracy.

This paper presents a novel approach for estimating the ego-motion of a vehicle in dynamic and unknown environments using tightly-coupled inertial and visual sensors. To improve the accuracy and robustness, we exploit the combination of point and line features to aid navigation. The mathematical framework is based on trifocal geometry among image triplets, which is simple and unified for point and line features. For the fusion algorithm design, we employ the Extended Kalman Filter (EKF) for error state prediction and covariance propagation, and the Sigma Point Kalman Filter (SPKF) for robust measurement updating in the presence of high nonlinearities. The outdoor and indoor experiments show that the combination of point and line features improves the estimation accuracy and robustness compared to the algorithm using point features alone.

This paper proposes an enhanced visual simultaneous localization and mapping (vSLAM) algorithm tailored for mobile robots operating in indoor dynamic scenes. By incorporating point-line features and leveraging the Manhattan world model, the proposed PLM-SLAM framework significantly improves localization accuracy and map consistency. This algorithm optimizes the line features detected by the Line Segment Detector (LSD) through merging and pruning strategies, ensuring real-time performance. Subsequently, dynamic point-line features are rejected based on Lucas–Kanade (LK) optical flow, geometric constraints, and depth information, minimizing the impact of dynamic objects. The Manhattan world model is then utilized to reduce rotational estimation errors and optimize pose estimation. High-precision line feature matching and loop closure detection mechanisms further enhance the robustness and accuracy of the system. Experimental results demonstrate the superior performance of PLM-SLAM, particularly in high-dynamic indoor environments, outperforming existing state-of-the-art methods.

At present, most of the visual Simultaneous Localization and Mapping (SLAM) systems rely on the surrounding point features to achieve acceptable localization and mapping. However, the number of point features is insufficient in the low-texture environments, so the performance of these SLAM systems will be significantly reduced. In this research, a PLISLAM system that integrates point features, line features and Inertial Measurement Unit (IMU) is proposed to implement high-precision positioning and mapping for dynamic vehicles. Specifically, a state-of-the-art SLAM scheme ORBSLAM3 is built at first. Then, its theoretical formulation is derived step by step to handle the environmental line features and the Bundle Adjustment (BA) is integrated to optimize the data. Finally, the system performance is verified through the EuRoC dataset, the results demonstrate its accuracy could be improved by adding the line features especially in scenes with rich line features.

A method of feature line extraction from triangular mesh model is presented. Firstly, initial feature point is manually specified based on feature points extraction from triangular mesh model. Then begin with initial feature point, Principal Component Analysis (PCA) method is applied to analyze primary direction of special feature region of feature points. Search the feature point which is furthest from the center of mass in the primary direction and it is chosen as subsequent feature point. Iteratively, record the feature point sequence in order. Finally, use three order non-uniform B-spline curve to fit and construct feature line. The experimental results demonstrate that the proposed method in this paper is efficient.

LiDAR-based odometry and mapping is used in many robotic applications to retrieve the robot's position in an unknown environment and allows for autonomous operation in GPS-denied (e.g., indoor) environments. With a 3D LiDAR sensor, highly accurate localization becomes possible, which enables high quality 3D reconstruction of the environment. In this letter we extend the well-known LOAM framework by leveraging prior knowledge about a reference object in the environment to further improve the localization accuracy. This requires a known 3D model of the reference object and its known position in a global coordinate frame. Instead of only relying on the point features in the mapping module of LOAM, we also include mesh features extracted from the 3D triangular mesh of the reference object in the optimization problem. For fast correspondence computation of mesh features, we use the Axis-Aligned-Bounding-Box-Tree (AABB) structure. Essentially, our approach not only makes use of the previously built map for absolute localization in the environment, but also takes the relative position to the reference object into account, effectively reducing long-term drift. To validate the proposed concept, we generated datasets using the Gazebo simulation environment in exemplary visual inspection scenarios of an airplane inside a hangar and the Eiffel Tower. An actuated 3D LiDAR sensor is mounted via a 1-DoF gimbal on a UAV capturing 360° scans. We benchmark our approach against the state-of-the-art open-source LOAM framework. The results show that the proposed joint optimization using both point and mesh features yields a significant reduction in Absolute Pose Error (APE) and therefore improves the map and 3D reconstruction quality during long-term operations.

Point-cloud-based technique plays a very significant role in 3D model restoration. In the archaeological application of stone tools, the scale drawing, which is hand-drawn from measured stone tools, is traditionally used. In the scale drawing creation, a base drawing which consists outline and ridge lines is initially drawn from geometric features of shape. After that other lines are extracted from knowledge of making stone tools and are added to the base drawing. It requires special knowledge to extract feature lines from stone tools so that scale drawing is time-consuming. Therefore, if the base drawing is automatically extracted, the working hours are reduced. To overcome this issue, this paper proposes a feature line extraction method using the Mahalanobis distance metric. First, the points on outline are extracted from a point cloud. Then, the surface variation is calculated with a various number of neighbors and thus the potential feature points are detected by the analysis of its surface variation. After that, the potential feature points are thinned towards the highest variation points by using Laplacian smoothing. Then, the thinned feature points are shrunk to the potential feature points. Finally, a feature line is extracted by connecting the nearest thinned feature points locating in the Mahalanobis distance field. To verify our method, the extracted feature lines are compared to the ground truth of base drawing drawn by archaeological illustrators. Our method is applied to stone tools, and we confirm the effectiveness of our method.

The detection of feature lines is important for representing and understanding geometric features of 3D models. In this paper, we introduce a new and robust method for extracting feature lines from unorganized point clouds. We use a one-dimensional truncated Fourier series for detecting feature points. Each point and its neighbors are approximated along the principal directions by using the truncated Fourier series, and the curvature of the point is computed from the approximated curves. The Fourier coefficients are computed by Fast Fourier Transform (FFT). We apply low-pass filtering to remove noise and to compute the curvature of the point robustly. For extracting feature points from the detected potential feature points, the potential feature points are thinned using a curvature weighted Laplacian-like smoothing method. The feature lines are constructed through growing extracted points and then projected onto the original point cloud. The efficiency and robustness of our approach is illustrated by several experimental results.

Accurate and robust localization of roadheaders is essential for intelligent control in underground coal mining, where visual perception systems face significant challenges such as low illumination, abrupt lighting changes, dust interference, and complex backgrounds. This paper proposes a stereo vision-based localization method for roadheaders that integrates point and line features to enhance pose estimation accuracy and robustness under complex lighting conditions. To address image degradation caused by the coexistence of low-light and overexposed regions, an adaptive multi-region image enhancement approach is developed. The method dynamically segments the image into high-, medium-, and low-illumination regions based on average brightness and standard deviation, applying region-specific enhancement strategies such as gamma correction, CLAHE, and contrast adjustment. A gradient-based weighted fusion mechanism and bilateral filtering are introduced to improve image quality and feature extraction stability. Line features are extracted using EDLines and represented via Plücker coordinates. A joint point-line optimization model with a dynamic weighting scheme is constructed to improve localization robustness. Experiments on public datasets including EuRoc MAV and KITTI demonstrate that the proposed method achieves an absolute pose error (APE) of 0.10 m and root mean square error (RMSE) of 0.13 m, outperforming existing methods such as PL-SLAM and PL-EDLines-SLAM. Further validation in a dust- and glare-simulated underground environment yields an APE of 0.15 m and RMSE of 0.17 m, confirming the method’s environmental adaptability and practical applicability. This study provides a reliable visual localization solution for underground roadheaders, supporting downstream tasks such as path planning, tracking control, and autonomous operation.

For the visual-inertial system, the key frame loss always emerges in the rotation with the weak texture and strong illumination, which seriously affects the camera positioning accuracy. To solve this problem, a tight-coupling-based visual fusion scheme leveraging line features is proposed in our work, called VinsFusion-Line. It first deeply integrates line features, inertial measurement unit(IMU) pre-integration, and stereo visual observations through a tightly-coupled framework. And the infinitely extended line features provide more geometric and structural information, compared with the base point and line feature tracing methods. Hence, the positioning accuracy is improved greatly. In order to further make the calculation of 3D spatial line features faster and more efficient, Plücker coordinates and straight lines are used to represent the features. In addition, the camera state is optimized by a multi-source cost function that embeds both IMU pre-integration residuals and line feature reprojection errors. This design enables tight-coupled fusion of IMU and binocular visual information, rather than relying solely on point features as in existing methods. Finally, experiments on the public data show that under the strong light and weak texture, the binocular VinsFusion-Line system with infinite extension line in our work has better positioning accuracy than the single-purpose Vins-Fusion system.

Image registration is an important basis of image processing, which is of great significance in image mosaicking, target recognition, and change detection. Aiming at the automatic registration problem of multi-angle optical images for ground scenes, a registration method combining point features and line features to register images is proposed. Firstly, the LSD (Line Segment Detector) algorithm is used to extract line features of images. The obtained line segments whose length are less than a given threshold are eliminated by a visual significant algorithm. Then, an affine transform model obtained by estimating a Gaussian mixture model (GMM) is applied to the image to be matched. Lastly, Harris point features are utilized in fine matching to overcome shortages of methods based on line features. In experiments, the proposed algorithm is compared with popular feature-based registration algorithms. The results indicate that the proposed algorithm in this work has obvious advantages in terms of registration accuracy and reliability for optical images acquired at different angles.

Visual Simultaneous Localization and Mapping (SLAM) technologies based on point features achieve high positioning accuracy and complete map construction. However, despite their time efficiency and accuracy, such SLAM systems are prone to instability and even failure in poor texture environments. In this paper, line features are integrated with point features to enhance the robustness and reliability of stereo SLAM systems in poor texture environments. Firstly, method Edge Drawing lines (EDlines) is applied to reduce the line feature detection time. Meanwhile, the proposed method improves the reliability of features by eliminating outliers of line features based on the entropy scale and geometric constraints. Furthermore, this paper proposes a novel Bags of Word (BoW) model combining the point and line features to improve the accuracy and robustness of loop detection used in SLAM. The proposed PL-BoW technique achieves this by taking into account the co-occurrence information and spatial proximity of visual words. Experiments using the KITTI and EuRoC datasets demonstrate that the proposed stereo Point and EDlines SLAM (PEL-SLAM) achieves high accuracy consistently, including in challenging environments difficult to sense accurately. The processing time of the proposed method is reduced by 9.9% and 4.5% when compared to the Point and Line SLAM (PL-SLAM) and Point and stereo Point and Line based Visual Odometry (sPLVO) methods, respectively.

Generally speaking, visual SLAM systems just based on point features have poor robustness in low-textured scenes, which limits their application. To improve the robustness and accuracy of simultaneous location and mapping in various environments, a multi-scene adaptive visual SLAM system based on RGB-D camera is proposed. We not only use point features in our system, but also introduce line features that are abundant indoors. They are less sensitive to lighting variation and more stable than point features in low-textured scenes. The SLAM system combining point features and line features contains several parts: visual odometry, local mapping, loop closing, full BA and mapping. The experimental results on datasets demonstrate that the performance of our proposed SLAM system is better than state-of-the-art point-based method in various indoor scenes.