Novel approach for mobile robot localization using monocular vision

Location is an elemental problem for mobile robotics due the importance of determining a position of the robot in space. This knowledge along with the environment map are basic information for robot mobility. In this paper, a new approach for navigation and location of mobile robots on topological maps using classification with reject option in attributes obtained from a Structural Co-occurrence Matrix (SCM) is proposed. Furthermore, we compare our approach with others state-of-the-art extractors, such as Statistical Moments, Gray-Level Co-occurrence Matrix (GLCM) and Local Binary Patterns (LBP). Structural Co-Occurrence Matrix was evaluated with the Average, Gaussian, Laplacian and Sobel filters. Regarding to classifiers, Bayesian classifier, Multilayer Perceptron (MLP) and Support Vector Machines (SVM) were analyzed. The descriptors Scale Invariant Feature Transform (SIFT) and Speed Up Robust Features (SURF) were also used. According to results, SCM was the fastest feature extractor with 0.117 s and accuracy of 100% in navigation test, showing the relevance of our approach in the mobile robot localization.

This paper proposes a novel appearance-based place recognition approach for mobile robot localization. The multi-channel Gabor filters is used to extract the global texture features of the scene images which are associated with the corresponding locations, and then these texture features are fed to support vector machine classifiers to determine the location of mobile robot. The algorithm has been tested on our designed and developed autonomous mobile robot CASIA-I. The experiment results indicate that the algorithm can reach up to 91.11% correct location rate, be robust to the various illumination and contrast, and satisfy the real-time self-localization demand of mobile robot.

This paper describes an approach for mobile robot localization using a visual word based place recognition approach. In our approach we exploit the benefits of a stereo camera system for place recognition. Visual words computed from SIFT features are combined with VIP (viewpoint invariant patches) features that use depth information from the stereo setup. The approach was evaluated under the ImageCLEF@ICPR 2010 competition. The results achieved on the competition datasets are published in this paper.

In this paper, we present a novel 2-D laser-based global localization approach for mobile robots, which is composed of geometrical relationship construction, a new structural unit encoding scheme (SUES), and an extended multiple hypothesis tracking (MHT) algorithm. Different from existing methods, we construct a 3-D "directional endpoint" feature encapsulating both the endpoint and the direction of a line segment; on this basis, a novel and efficient online structural unit encoding scheme (SUES) is proposed to describe the geometric relationship between the two directional endpoint features with some robustness to dynamic disturbances. Note that SUES presented in this paper is different from the bag-of-words scheme in two aspects: 1) SUES quantizes the geometrical relationship without offline training for vocabulary and 2) SUES is independent of the quality of the vocabulary. By factoring the global localization problem into a discrete pose estimation problem, the MHT is extended on the basis of SUES and odometry information to gradually restore the global robot pose. Different from the classical MHT framework, the extended MHT takes the independent observation results as the a priori, while the likelihood term is composed of consecutive candidate poses and the odometry information. Evaluations are carried out by using both publicly available data sets and self-recorded data sets. Comparative experimental results with respect to the adaptive Monte Carlo localization are presented to show the superior performance of the proposed approach in terms of success ratio and efficiency.

Simultaneous localization and mapping (SLAM) becomes popular in the autonomous robot research community. SLAM requests a mobile robot, placed at an unknown location in an unknown environment, to incrementally build a consistent map of this environment while simultaneously determining its location within this map. The odometer data is helpful but the noisy error accumulates. Assistant with measurements from the other sensors, SLAM improves the accuracy by filtering. The extended Kalman filter is the most used method to feedback the error. In this thesis, a monocular vision based SLAM for a wheeled robot is proposed. A single web camera is the only measurement input to correct the odometer data. Extract features and match them based on the scale invariant feature transform (SIFT). Absence of the feature depth information is the key issue for the monocular vision in SLAM. The inverse depth parametrization method solves the problem by present the position in inverse coordinate with some reasonable assumptions. Convergence of the feature position is more easily achieved by the EKF due to the linearity. Combining the odometer data with the inverse depth method, this system provides an accurate map by tests of the indoor image sequence.

In recent years, pose measurement based on vision system has become a research focus. As pose measurement based on monocular vision is simple and stable with low cost and high calculation speed, it has been widely applied in reality. This paper summarizes the pose measurement based on monocular vision, and emphasizes on pose measurement based on characteristic points, lines and advanced geometry features, with analysing each advantages and disadvantages. Ultimately, it makes a brief analysis on the future of pose measurement based on monocular vision in order to provide references on research on pose measurement.

This paper proposes a method to build a semantic map for robot navigation by using a laser sensor and monocular vision. Laser data is used to construct an environmental grid map, which can guide the robot forward without collision. In this paper, we assume the indoor environment is one of the floors of the building. By using the monocular vision, the mobile robot can automatically search the doorplate of each office room and recognize the room number, which is used as an environmental semantic identifier. We assign the semantic label to the grid map so that the constructed map includes not only the spatial information but also the semantic information. The robot traverses the target floor and completes the construction of the semantic map automatically. The experimental result has shown the robot can be navigated to any given address in the target floor to complete the visit task by using our proposed semantic map.

In this paper, we propose a method that combines the monocular vision with the binocular vision to process a single texture image that the feature points difficult to extract and restore the depth information of the inclined plane objects. Through integration of the two approaches, we combine the advantages of monocular vision in terms of the recovery of the basic structure of objects in the image with the advantages of binocular vision in terms of the accurate depth estimation after correctly determining the corresponding feature points, obtain high precision and accuracy depth estimates. Experiments show that application of this integration can obtain high-accuracy depth map. Moreover, a high quality 3-D reconstruction is also obtained from the accuracy depth map.

This thesis presents a recent research on the problem of environmental modeling for both localization and map building for wheel-based, differential driven, fully autonomous and self-contained mobile robots. The robots behave in an indoor office environment. They have a multi-sensor setup where the encoders are used for odometry and two exteroperceptive sensors, a 360° laser scanner and a monocular vision system, are employed to perceive the surrounding. The whole approach is feature based meaning that instead of directly using the raw data from the sensor features are firstly extracted. This allows the filtering of noise from the sensors and permits taking account of the dynamics in the environment. Furthermore, a properly chosen feature extraction has the characteristic of better isolating informative patterns. When describing these features care has to be taken that the uncertainty from the measurements is taken into account. The representation of the environment is crucial for mobile robot navigation. The model defines which perception capabilities are required and also which navigation technique is allowed to be used. The presented environmental model is both metric and topological. By coherently combining the two paradigms the advantages of both methods are added in order to face the drawbacks of a single approach. The capabilities of the hybrid approach are exploited to model an indoor office environment where metric information is used locally in structures (rooms, offices), which are naturally defined by the environment itself while the topology of the whole environment is resumed separately thus avoiding the need of global metric consistency. The hybrid model permits the use of two different and complementary approaches for localization, map building and planning. This combination permits the grouping of all the characteristics which enables the following goals to be met: Precision, robustness and practicability. Metric approaches are, per definition, precise. The use of an Extended Kalman Filter (EKF) permits to have a precision which is just bounded by the quality of the sensor data. Topological approaches can easily handle large environments because they do not heavily rely on dead reckoning. Global consistency can, therefore, be maintained for large environments. Consistent mapping, which handle large environments, is achieved by choosing a topological localization approach, based on a Partially Observable Markov Decision Process (POMDP), which is extended to simultaneous localization and map building. The theory can be mathematically proven by making some assumptions. However, as stated during the whole work, at the end the robot itself has to show how good the theory is when used in the real world. For this extensive experimentation for a total of more than 9 km is performed with fully autonomous self-contained robots. These experiments are then carefully analyzed. With the metric approach precision with error bounds of about 1 cm and less than 1 degree is further confirmed by ground truth measurements with a mean error of less than 1 cm. The topological approach is successfully tested by simultaneous localization and map building where the automatically created maps turned out to work better than the a priori maps. Relocation and closing the loop are also successfully tested.

Deep learning for 6D pose estimation of objects — A case study for autonomous driving

In this paper, a novel method is proposed about visual servoing with a monocular wheeled mobile robot system, which can deal with dynamic visual targets. For the existing methods, the mobile robot can reach the desired pose correctly where the feature points are fixed. However, the mobile robot cannot reach the desired pose when the feature points have been moved. By adding a fixed camera in the scene for monitoring the movement of the feature points, the proposed approach can still work well when feature points are moved. From the images of the monitor camera, the POSIT algorithm is utilized to calculate the movement of the feature points. Likewise, we can calculate the relationship between the relevant coordinate systems with respect to the feature points. By coordinate transformation, the translation vector and rotation matrix between the current pose and the desired pose of mobile robot can be obtained by real-time image feedback. Finally, by utilizing the polar coordinate representation, a motion controller is adopted to pilot the mobile robot reach the desired pose. The feasibility of the proposed method is investigated by simulation results.

The 3-D reconstruction is of great importance for measuring, inspecting, monitoring, or understanding surface characteristics. Previous active target-based vision methods provide the reconstruction with more than three images for monocular vision, usually using extra external monocular, stereo, or multiview vision to expand the measurement range. This article aims to achieve the reconstruction from the initial image of monocular vision without precalibration. The subpixel centers on the target with groups of three collinear concentric circles in the radial pattern are obtained in the region of interest (ROI) by histogram of oriented gradient (HOG)–support vector machine (SVM), grouped by electromagnetic (EM), and extracted by the Zernike moment (ZM). The intrinsic parameters of the projection of the single image are modeled by the two projections of the general-direction infinity points generated from the collinear centers of the radial pattern. Moreover, the ratio of the cross ratios of CSI and CST features with infinity points is proposed to determine moderate images with small errors for reconstruction. The mutual benchmark model is presented to enlarge the effective measurement range without extra external observation instruments. Moreover, the reconstruction is modeled and further refined in SE(3) to achieve the orthogonality, accuracy, and large measurement range, which are proved by simulation and experimental verification.

Accurate and reliable localization is a base requirement for a mobile robot. Landmark localization is a very common technique to do this. The arrangement of landmarks is important for reliable and accurate estimation of localization data. In this paper the optimized arrangement of artificial landmarks is discussed and a solution for building an accurate and reliable localization system based on combining artificial and natural landmarks is presented. Landmark placement is done by definition and optimization of a confidence level describing the expected localization estimation error. Areas in the environment which are ill conditioned by natural landmarks are furnished with additional artificial landmarks. The proposed system has been tested in a simulation and with a mobile robot in a real office environment.

Accurate localization for underwater robot is a big challenge in the mobile robot community. We use acoustic and visual-based approaches to solve the problem. In this paper, we focuses on the artificial landmark visual-based localization. The robot carries the camera to localize its position by calculating the camera's viewpoint through looking at the landmarks. The position of landmarks is known in a world-centered coordinate system (WCCS). A method is presented in this paper to recover the camera's viewpoint with minimum three feature points and a single image from one camera. Two steps are used in this paper: first step is to calculate the feature points' 3D coordinates in a camera centered coordinate system (CCCS); second step is to obtain a closed-form solution through the geometric transformations to map the 3D points from CCCS to WCCS. The algorithm is robust and efficient. It uses the fewest feature points required so far to deal with the same problem

In this paper, a novel framework for 3D face reconstruction from a single 2D face image was proposed. We focus on generating 3D face model without expensive devices and complicated calculation. First, we preprocess 2D face image, including illumination compensation, face detection and feature point extraction. Then, the method is based on a 3D morphable face model that encodes shape and texture in terms of model parameters. The prior 3D face model is a linear combination of “eigenheads” obtained by applying PCA to a training set of laser-scanned 3D faces. To account for pose and illumination variations, the algorithm simulates the process of image formation in 3D face and it estimates 3D face and texture of faces from a single image. As opposed to previous literature, our method can locate facial feature points automatically in a 2D facial image. Moreover, we also show that our proposed method for pose estimation can be successfully applied to 3D face reconstruction. In our experiment results, the method proposed has a satisfied performance regarding calculating time and the reconstructing photorealistic 3D faces.