Monocular vision SLAM based on key feature points selection

This paper presents a sensor fusion strategy applied for Simultaneous Localization and Mapping (SLAM) in dynamic environments. The designed approach consists of two features: (i) the first one is a fusion module which synthesizes line segments obtained from laser rangefinder and line features extracted from monocular camera. This policy eliminates any pseudo segments that appear from any momentary pause of dynamic objects in laser data. (ii) The second characteristic is a modified multi-sensor point estimation fusion SLAM (MPEF-SLAM) that incorporates two individual Extended Kalman Filter (EKF) based SLAM algorithms: monocular and laser SLAM. The error of the localization in fused SLAM is reduced compared with those of individual SLAM. Additionally, a new data association technique based on the homography transformation matrix is developed for monocular SLAM. This data association method relaxes the pleonastic computation. The experimental results validate the performance of the proposed sensor fusion and data association method.

The SLAM (Simultaneous Localization and Mapping) problem is one of the essential challenges for the current robotics. Our main objective in this work is to develop a real-time visual SLAM system using monocular omnidirectional vision. Our approach is based on the Extended Kalman Filter (EKF). We use the Spherical Camera Model to obtain geometric information from the images. This model is integrated in the EKF-based SLAM through the linearization of the direct and the inverse projections. We introduce a new computation of the descriptor patch for catadioptric omnidirectional cameras which aims to reach rotation and scale invariance. We perform experiments with omnidirectional images comparing this new approach with the conventional one. The experimentation confirms that our approach works better with omnidirectional cameras since features last longer and constructed maps are bigger.

A novel scale-invariant feature transform (SIFT) algorithm is proposed for soccer target recognition application in a robot soccer game. First, the method of generating scale space is given, extreme points are detected. This gives the precise positioning of the extraction step and the SIFT feature points. Based on the gradient and direction of the feature point neighboring pixels, a description of the key points of the vector is generated. Finally, the matching method based on feature vectors is extracted from SIFT feature points and implemented on the image of the football in a soccer game. By employing the proposed SIFT algorithm for football and stadium key feature points extraction and matching, significant increase can be achieved in the robot soccer ability to identify and locate the football.

At present, most of the visual Simultaneous Localization and Mapping (SLAM) systems rely on the surrounding point features to achieve acceptable localization and mapping. However, the number of point features is insufficient in the low-texture environments, so the performance of these SLAM systems will be significantly reduced. In this research, a PLISLAM system that integrates point features, line features and Inertial Measurement Unit (IMU) is proposed to implement high-precision positioning and mapping for dynamic vehicles. Specifically, a state-of-the-art SLAM scheme ORBSLAM3 is built at first. Then, its theoretical formulation is derived step by step to handle the environmental line features and the Bundle Adjustment (BA) is integrated to optimize the data. Finally, the system performance is verified through the EuRoC dataset, the results demonstrate its accuracy could be improved by adding the line features especially in scenes with rich line features.

Chapter 3.4 - Monocular SLAM with Undelayed Initialization for an Indoor Robot

The autonomous navigation capability of USVs in complex water environments relies on Simultaneous Localization and Mapping (SLAM) technology. However, water surface perturbations and reflections generate numerous dynamic invalid feature points, causing low localization accuracy and poor robustness in VSLAM. To this end, we propose a monocular VSLAM algorithm considering dynamic invalid features of the water surface for USV, named WS-SLAM. Firstly, the ORBSLAM3 framework is improved by adding the PP-LiteSeg semantic segmentation thread to distinguish between dynamic invalid feature points of the water surface and feature points on shore. Secondly, semantic information is used to generate a mask to cull dynamic invalid feature points of the water surface, reducing the impact of water current perturbation on visual localization. Then, further culling of potentially residual dynamic feature points on shore is performed in conjunction with the epipolar geometric constraint. Finally, the camera position is estimated using only static feature points. The experimental results based on the public USVInland dataset show that WSSLAM improves the positioning accuracy by an average of 43.47% in straight line sailing compared with ORBSLAM3. It can effectively inhibit the position drift caused by the dynamic invalid features of the water surface and has higher positioning accuracy and stronger robustness.

Simultaneous localization and mapping (SLAM) becomes popular in the autonomous robot research community. SLAM requests a mobile robot, placed at an unknown location in an unknown environment, to incrementally build a consistent map of this environment while simultaneously determining its location within this map. The odometer data is helpful but the noisy error accumulates. Assistant with measurements from the other sensors, SLAM improves the accuracy by filtering. The extended Kalman filter is the most used method to feedback the error. In this thesis, a monocular vision based SLAM for a wheeled robot is proposed. A single web camera is the only measurement input to correct the odometer data. Extract features and match them based on the scale invariant feature transform (SIFT). Absence of the feature depth information is the key issue for the monocular vision in SLAM. The inverse depth parametrization method solves the problem by present the position in inverse coordinate with some reasonable assumptions. Convergence of the feature position is more easily achieved by the EKF due to the linearity. Combining the odometer data with the inverse depth method, this system provides an accurate map by tests of the indoor image sequence.

As unmanned aerial vehicles are used in more environments, flexible navigation strategies are required to ensure safe and reliable operation. Operation in the presence of degraded or denied GPS signal is critical in many environments, particularly indoors, in urban canyons, and hostile areas. Two techniques, laser-based simultaneous localization and mapping (SLAM) and monocular visual SLAM, in conjunction with inertial navigation, have attracted considerable attention in the research community. This paper presents an integrated navigation system combining both visual SLAM and laser SLAM with an EKF-based inertial navigation system. The monocular visual SLAM system has fully correlated vehicle and feature states. The laser SLAM system is based on a Monte Carlo scan-to-map matching, and leverages the visual data to reduce ambiguities in the pose matching. The system is validated in full 6 degree of freedom simulation, and in flight test. A key feature of the work is that the system is validated with a controller in the navigation loop.

Visual Simultaneous Localization and Mapping (SLAM) technologies based on point features achieve high positioning accuracy and complete map construction. However, despite their time efficiency and accuracy, such SLAM systems are prone to instability and even failure in poor texture environments. In this paper, line features are integrated with point features to enhance the robustness and reliability of stereo SLAM systems in poor texture environments. Firstly, method Edge Drawing lines (EDlines) is applied to reduce the line feature detection time. Meanwhile, the proposed method improves the reliability of features by eliminating outliers of line features based on the entropy scale and geometric constraints. Furthermore, this paper proposes a novel Bags of Word (BoW) model combining the point and line features to improve the accuracy and robustness of loop detection used in SLAM. The proposed PL-BoW technique achieves this by taking into account the co-occurrence information and spatial proximity of visual words. Experiments using the KITTI and EuRoC datasets demonstrate that the proposed stereo Point and EDlines SLAM (PEL-SLAM) achieves high accuracy consistently, including in challenging environments difficult to sense accurately. The processing time of the proposed method is reduced by 9.9% and 4.5% when compared to the Point and Line SLAM (PL-SLAM) and Point and stereo Point and Line based Visual Odometry (sPLVO) methods, respectively.

Simultaneous localization and mapping (SLAM) is an important problem to solve in robotics theory in order to build truly autonomous mobile robots. This work presents a novel method for implementing a SLAM system based on a single camera sensor. The SLAM with a single camera, or monocular SLAM, is probably one of the most complex SLAM variants. In this case, a single camera, which is freely moving through its environment, represents the sole sensor input to the system. The sensors have a large impact on the algorithm used for SLAM. Cameras are used more frequently, because they provide a lot of information and are well adapted for embedded systems: they are light, cheap and power-saving. Nevertheless, and unlike range sensors, which provide range and angular information, a camera is a projective sensor providing only angular measurements of image features. Therefore, depth information (range) cannot be obtained in a single step. In this case, special techniques for feature system-initialization are needed in order to enable the use of angular sensors (as cameras) in SLAM systems. The main contribution of this work is to present a novel and robust scheme for incorporating and measuring visual features in filtering-based monocular SLAM systems. The proposed method is based in a two-step technique, which is intended to exploit all the information available in angular measurements. Unlike previous schemes, the values of parameters used by the initialization technique are derived directly from the sensor characteristics, thus simplifying the tuning of the system. The experimental results show that the proposed method surpasses the performance of previous schemes.

Simultaneous Localization And Mapping (SLAM) has become a crucial aspect in the fields of autonomous driving and robotics. One crucial component of visual SLAM is the Field-of-View (FoV) of the camera, as a larger FoV allows for a wider range of surrounding elements and features to be perceived. However, when the FoV of the camera reaches the negative half-plane, traditional methods for representing image feature points using <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$[u,v,1]^T$</tex-math> </inline-formula> become ineffective. While the panoramic FoV is advantageous for loop closure, its benefits are not easily realized under large-attitude-angle differences where loop-closure frames cannot be easily matched by existing methods. As loop closure on wide-FoV panoramic data further comes with a large number of outliers, traditional outlier rejection methods are not directly applicable. To address these issues, we propose LF-VISLAM, a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b> isual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> nertial <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SLAM</b> framework for cameras with extremely <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> arge <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</b> oV with loop closure. A three-dimensional vector with unit length is introduced to effectively represent feature points even on the negative half-plane. The attitude information of the SLAM system is leveraged to guide the feature point detection of the loop closure. Additionally, a new outlier rejection method based on the unit length representation is integrated into the loop closure module. We collect the PALVIO dataset using a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</b> anoramic <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> nnular <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b> ens (PAL) system with an entire FoV of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$360^\circ{\times}(40^\circ{\sim}120^\circ)$</tex-math> </inline-formula> and an Inertial Measurement Unit (IMU) for <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b> isual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> nertial <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</b> dometry (VIO) to address the lack of panoramic SLAM datasets. Experiments on the established PALVIO and public datasets show that the proposed LF-VISLAM outperforms state-of-the-art SLAM methods. Our code will be open-sourced at https://github.com/flysoaryun/LF-VISLAM. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Motivated by the challenges of handling large-FoV cameras in SLAM applications, this paper proposes LF-VISLAM, a novel SLAM framework that uses a large-FoV camera and IMU sensors. Our framework is equipped with a loop closure thread that can use attitude information to eliminate accumulated errors. We have made algorithmic adjustments and optimizations to the negative half-plane features to better adapt to cameras with large FoV. Experimental evaluations demonstrate that LF-VISLAM significantly outperforms traditional SLAM methods. Additionally, the code will be open-sourced, providing easy access for research and implementation. Overall, LF-VISLAM is a promising solution to improve the performance of SLAM in challenging environments of autonomous driving and robotics.

To solve the accurate positioning problem of mobile robots, simultaneous localization and mapping (SLAM) or visual odometry (VO) based on visual information are widely used. However, most visual SLAM or VO cannot meet the accuracy requirements in dynamic indoor environments. This paper proposes a robust visual odometry based on deep learning to eliminate feature points matching error. However, when a camera and dynamic objects are in relative motion, the frames of camera will produce ghosting, especially in high-dynamic environments, which bring additional positioning error; in view of this problem, a novel method based on the average optical flow value of the dynamic region is proposed to identify feature points of the ghosting, and then the feature points of the ghosting and dynamic region are removed. After the remaining feature points are matched, we use a non-linear optimization method to calculate the pose. The proposed algorithm is tested on TUM RGB-D dataset, and the results show that our VO improves the positioning accuracy than other robust SLAM or VO and is strongly robust especially in high-dynamic environments.

Recently, Simultaneous Localization and Mapping (SLAM) has achieved good performance in a certain environment. However, in a complex indoor environment, the tracking of the position of the mobile robot in Visual SLAM may get lost in some scenes with hardly any features. This paper presents a mobile robot localization method using an asynchronous data fusion formed by a wheel odometer and a visual odometer in the Extended Kalman Filter (EKF) based on the Semantic SLAM system. The first approach uses Semantic SLAM based on ORB-SLAM2 with YOLOv3, to get rid of the dynamic object while localizing and mapping, and build a global point cloud map where keyframes information is stored. Then, wheel odometer information and visual odometer information acquire after using a global point cloud map. Finally, the asynchronous data fusion for mobile robot localization, composed of visual odometer and wheel odometer, is realized at a frequency of about 50Hz in EKF. The experimental results on the real robot validate the method in accuracy and robustness by comparing it with ground truth from the motion capture measurement system in indoor environment.

The robot simultaneous localization and mapping (SLAM) is a very important and useful technology in the robotic field. However, the environmental map constructed by the traditional visual SLAM method contains little semantic information, which cannot satisfy the needs of complex applications. The semantic map can deal with this problem efficiently, which has become a research hot spot. This paper proposed an improved deep residual network- (ResNet-) based semantic SLAM method for monocular vision robots. In the proposed approach, an improved image matching algorithm based on feature points is presented, to enhance the anti-interference ability of the algorithm. Then, the robust feature point extraction method is adopted in the front-end module of the SLAM system, which can effectively reduce the probability of camera tracking loss. In addition, the improved key frame insertion method is introduced in the visual SLAM system to enhance the stability of the system during the turning and moving of the robot. Furthermore, an improved ResNet model is proposed to extract the semantic information of the environment to complete the construction of the semantic map of the environment. Finally, various experiments are conducted and the results show that the proposed method is effective.

Robust hand tracking via novel multi-cue integration