Efficient 7D aerial pose estimation

Aim. Determination of the elements of external spatial orientation of the surveying systems at the moment of image acquisition is the fundamental task in photogrammetry. Principally, this problem is solving in two ways. The first way is direct positioning and measuring of directions of camera optical axis in the geodetic space with the help of GNSS/INS equipment. The second way is the analytical solution of the problem using a set of reference information (often such information is a set of ground control points whose geodetic positions are known with sufficient accuracy and which are reliably recognised on aerial images of the photogrammetric block). The authors consider the task of providing reference and control information using the second approach, which has a number of advantages in terms of reliability and accuracy of determining the unknown image exterior orientation parameters. It is proposed to obtain additional images of ground control points by the method of their auxiliary aerial photography using an unmanned aerial vehicle (UAV) on a larger scale compared to the scale of the images of the photogrammetric block. The aim of the presented work is the implementation of the method of creating reference points and experimental confirmation of its effectiveness for photogrammetric processing. Methods and results. For the entire realization of the potential of the analytical way to determine the elements of external orientation of images, it is necessary to have a certain number of ground control points (GCP) and to keep the defined scheme of their location on the photogrammetric block. As the main source of input data authors use UAV aerial images of the terrain, which are obtained separately from the block of aerial survey, and have a better geometric resolution and which clearly depict the control reference points. Application of such auxiliary images gives the possibility of automated transferring of the position of ground control point into images of the main photogrammetric block. In our interpretation, these images of ground control points and their surroundings on the ground are called "control reference images". The basis of the work is to develop a method for obtaining the auxiliary control reference images and transferring of position of GCP depicted on them into aerial or space images of terrain by means of computer stereo matching. To achieve this goal, we have developed a processing method for the creation of control reference images of aerial image or a series of auxiliary multi-scale aerial images obtained by a drone from different heights above the reference point. The operator identifies and measures the GCP once on the auxiliary aerial image of the highest resolution. Then there is an automatic stereo matching of the control reference image in the whole series of auxiliary images in succession with a decrease in the resolution and, ultimately, directly with the aerial images of photogrammetric block. On this stage there are no recognition/cursor targeting by the human operator, and therefore there are no discrepancies, errors or mistakes related to it. In addition, if to apply fairly large size of control reference images, the proposed method can be used on a low-texture terrain, and therefore deal in many cases without the physical marking of points measured by GNSS method. And this is a way to simplify and reduce the cost of photogrammetric technology. The action of the developed method has been verified experimentally to provide the control reference information of the block of archival aerial images of the low-texture terrain. The results of the experimental approbation of the proposed method give grounds to assert that the method makes it possible to perform geodetic reference of photogrammetric projects more efficiently due to the refusal of the physical marking of the area before aerial survey. The proposed method can also be used to obtain the information for checking the quality of photogrammetric survey for provision of check points. The authors argue that the use of additional equipment - UAV of semi-professional class to obtain control reference images is economically feasible. Scientific novelty and practical relevance. The results of approbation of the "control reference image" method with obtaining stereo pairs of aerial images with vertical placement of the base are presented for the first time. There was implemented the study of the properties of such stereo pairs of aerial images to obtain images of reference points. The effectiveness of including reference images in the main block of the digital aerial triangulation network created on UAV’s images is shown.

Online pose estimation and mapping in unknown environments is essential for most mobile robots. Especially autonomous unmanned aerial vehicles require good pose estimates at comparably high frequencies. In this paper, we propose an effective system for online pose and simultaneous map estimation designed for light-weight UAVs. Our system consists of two components: (1) real-time pose estimation combining RTK-GPS and IMU at 100 Hz and (2) an effective SLAM solution running at 10 Hz using image data from an omnidirectional multi-fisheye-camera system. The SLAM procedure combines spatial resection computed based on the map that is incrementally refined through bundle adjustment and combines the image data with raw GPS observations and IMU data on keyframes. The overall system yields a real-time, georeferenced pose at 100 Hz in GPS-friendly situations. Additionally, we obtain a precise pose and feature map at 10 Hz even in cases where the GPS is not observable or underconstrained. Our system has been implemented and thoroughly tested on a 5 kg copter and yields accurate and reliable pose estimation at high frequencies. We compare the point cloud obtained by our method with a model generated from georeferenced terrestrial laser scanner.

We envision a telepresence system that enhances remote work by facilitating both physical and immersive visual interactions between individuals. However, during robot teleoperation, communication often lacks realism, as users see the robot’s body rather than the remote individual. To address this, we propose a method for overlaying a digital human model onto a humanoid robot using XR visualization, enabling an immersive 3D telepresence experience. Our approach employs a learning-based method to estimate the 2D poses of the humanoid robot from head-worn stereo views, leveraging a newly collected dataset of full-body poses for humanoid robots. The stereo 2D poses and sparse inertial measurements from the remote operator are optimized to compute 3D poses over time. The digital human is localized from the perspective of a continuously moving observer, utilizing the estimated 3D pose of the humanoid robot. Our moving camera-based pose estimation method does not rely on any markers or external knowledge of the robot’s status, effectively overcoming challenges such as marker occlusion, calibration issues, and dependencies on headset tracking errors. We demonstrate the system in a remote physical training scenario, achieving real-time performance at 40 fps, which enables simultaneous immersive and physical interactions. Experimental results show that our learning-based 3D pose estimation method, which operates without prior knowledge of the robot, significantly outperforms alternative approaches requiring the robot’s global pose, particularly during rapid headset movements, achieving markerless digital human augmentation from head-worn views.

Head pose estimation methods can be broadly classified into learning-based methods and model-based methods. The learning based methods use machine learning techniques to directly predict the pose from image appearance, while the model-based methods link the 2D observation (e.g. facial landmarks) and 3D model through the projection model for pose estimation. However, both methods may have difficulty on images with very low quality (e.g. low resolution, occlusion, and noisy images). For example, there would be limited appearance information to generate accurate landmark detection on lowquality images for reliable face pose estimation. To tackle pose estimation on low-quality images, we propose to combine the learning and model based methods. Specifically, we first build the relationship between facial landmark locations and image appearance using the Restricted Boltzmann Machine (RBM) model. Then, we link the landmark locations and 3D model analytically using the projection model. By combining the RBM model with the projection model, without explicit landmark detection, we predict the head pose with a KL-divergence based method and a gradient-based method. Experimental results demonstrate the effectiveness of the proposed method.

Initiated and financed by the Canadian Space Agency, Micro-Rover Platform with Tooling Arm (MRPTA) project has targeted the development of a robotic system with remote control and autonomous navigation capabilities for testing a large variety of analogous planetary missions. Although developed for planetary exploration, the navigation system is highly suitable to such areas of application as construction and in mining. Autonomous navigation system is capable of moving the platform to predefined position(s)/orientation(s) while continuously mapping the terrain, assessing its traversability and choosing most appropriate path to follow. Designed according to the JAUS (Joint Architecture for Unmanned Systems) framework, the system consists of several interconnected components. Following components are of particular importance: Pose Estimator (PE), Terrain Evaluator (TE), Map Manager (MP), and Path Planner (PP). The PE uses a minimal sensor configuration consisting of an azimuth gyroscope, inclinometer and wheel odometry. The PE provides reliable pose estimates in spite of pronounced slippage by employing extensive use of fused data. The TE uses a nodding laser scanner which continuously sweeps the area in front of the platform and constructs a “traversability grid” (TG) of the surrounding area. Multiple TGs are constantly constructed along the platform motion path are combined into a terrain map maintained by the MP. The PP uses this map to compute a motion path Obtained path is executed by the motion controller and remains current until it is contradicted by the most recent map – at which point it is re-planned. This paper addresses integration issues, lessons learned and also outlines the possible applications for construction and mining.

This paper has two contributions in the context of line based camera pose estimation, 1) We propose a purely geometric approach to establish correspondence between 3D line segments in a given model and 2D line segments detected in an image, 2) We eliminate a degenerate case due to the type of rotation representation in arguably the best line based pose estimation method currently available. For establishing line correspondences we perform exhaustive search on the space of camera pose values till we obtain a pose (position and rotation) which is geometrically consistent with the given set of 2D, 3D lines. For this highly complex search we design a strategy which performs precomputations on the 3D model using separate set of constraints on position and rotation values. During runtime, the set of different rotation values are ranked independently and combined with each position values in the order of their ranking. Then successive geometric constraints which are much simpler when compared to computing reprojection error are used to eliminate incorrect pose values. We show that the ranking of rotation values reduces the number of trials needed by a huge factor and the simple geometric constraints avoid the need for computing the reprojection error in most cases. Though the execution time for the current MATLAB implementation is far from real time requirement, our method can be accelerated significantly by exploiting simplicity and parallelizability of the operations we employ. For eliminating the degenerate case in the state of art pose estimation method, we reformulate the rotation representation. We use unit quaternions instead of CGR parameters used by the method.

Locating the rotator cable during subacromial arthroscopy: bursal- and articular-sided anatomy.

Pose detection of objects is an important topic in object-level mapping and indoor localization. In the past, pose estimation methods were performed either with the help of artificial markers or natural features found on the object. However, due to the fact that the markers can only be utilized in controlled environment experiments, the application of marker-based approaches is very limited. Furthermore, methods that depend on the object's natural visual features require texture on the object and lack robustness to illumination and camera viewpoint variations. With the advent of Deep Learning (DL), the classical pose estimation methods have been outperformed. The DL-based pose estimation can detect deep features of the object and exhibits higher robustness to many distortions and variabilities caused by the changes in the illumination and viewpoint conditions. However, the massive training data set requirement is the main challenge with most DL-based methods. The training set is often a real set of images that have been manually labeled or annotated. In addition, such methods face problems related to the degradation of their predicted accuracy in the presence of uncertainties due to the symmetrical structure of many objects. To address the aforementioned issues, a novel and very fast method for generating synthetic data, as well as a contour-based technique for accurate pose estimation (that can handle pose ambiguities for a symmetrical object) are proposed in this paper. The tests that are conducted in multiple indoor scenarios demonstrate not only the effectiveness of the synthetic data generation but also exhibit, in many cases, the very high accuracy of the proposed pose estimation method.

Vehicle pose estimation with light detection and ranging (LiDAR) is essential in the perception technology of autonomous driving. However, because of incomplete observation measurements and sparsity of the LiDAR point cloud, it is challenging to achieve satisfactory pose extraction based on 3D LiDAR with the existing pose estimation methods. In addition, the demand for real-time performance further increases the difficulty of the pose estimation task. In this paper, we propose a novel vehicle pose estimation method based on the convex hull. The extracted 3D cluster is reduced to the convex hull, reducing the subsequent computation burden while preserving essential contour information. Subsequently, a novel criterion based on the minimum occlusion area is developed for the search-based algorithm, enabling accurate pose estimation. Additionally, this criterion renders the proposed algorithm particularly well-suited for obstacle avoidance. The proposed algorithm is validated on the KITTI dataset and a manually labeled dataset acquired at an industrial park. The results demonstrate that our proposed method can achieve better accuracy than the classical pose estimation method while maintaining real-time speed.

Vehicle pose estimation with LIDAR plays a crucial role in autonomous driving systems. It serves as the fundamental basis for functions such as tracking, path planning, and decision-making. However, the majority of current vehicle pose estimation techniques struggle to produce satisfactory results when faced with incomplete observation measurements, such as L-shaped point cloud clusters without side contours or those including side-view mirrors. In addition, the requirement for real-time results further increases the difficulty of the pose estimation task. In this paper, we present a vehicle Pose Estimation method with Heuristic L-shape fitting and grid-based Particle Filter (PE-HL-PF). We design a geometric shape classifier module to divide clusters into symmetrical and asymmetrical ones according to their shape features. Furthermore, a contour-based heuristic L-shape fitting module is introduced for asymmetrical clusters, and a structure-aware grid-based particle filter is used to estimate the pose of symmetrical clusters. PE-HL-PF first utilizes a heuristic asymmetrical module that selects dominant contours fitting orientation in a heuristic manner, thereby avoiding the need for a complex traversal search. Additionally, a symmetrical module based on particle filtering is incorporated to enhance the stability of orientation estimation. This method achieves significant improvements in both the runtime efficiency and pose estimation accuracy of incomplete point clouds. Compared with state-of-the-art pose estimation methods, our PE-HL-PF demonstrates a notable performance improvement. Our method can estimate the pose of thousands of objects in less than 1 millisecond, a significant improvement over previous methods. The results of experiments performed on the KITTI dataset validate the effectiveness of our approach.

Pose estimation is one of the vital issues in mobile robot navigation. Odometry data can be fused with absolute position measurements to provide better and more reliable pose estimation. This paper deals with the determination of better relative localization of a two wheeled differential drive robot by means of odometry by considering the influence of parameters namely weight, velocity, wheel perimeter and tyre width. Experiments have been conducted based on central composite rotatable design matrix. A mathematical model has been developed for the robot using Response Surface Methodology (RSM) with the help of MINITAB software. An optimum condition for minimum odometry error was obtained by using Excel (XL) Solver.

STag: A stable fiducial marker system

In this paper, a robust pose (i.e., position and orientation) estimation algorithm using two-views captured by a calibrated monocular camera is presented. A collection of pose hypotheses is obtained when more than the minimum number of feature points required to uniquely identify a pose are available in both the images. The pose hypotheses - unit quaternion and unit translation vectors - lie on the S3 and S2 manifolds in the Euclidean 4-space and 3-space, respectively. Probability density function (pdf) of the rotation and translation pose hypotheses is evaluated by gridding the unit spheres where a robust, coarse pose estimate is identified at the mode of the pdf. Further, a "refining" pdf of the geodesic distance from the coarse pose estimate is constructed for the hypotheses within a grid containing the coarse estimate. A refined pose estimate is obtained by averaging the low-noise hypotheses in the neighbourhood of the mode of refining pdf. Pose estimation results of the proposed method are compared with RANSAC and nonlinear mean-shift (NMS) algorithms using the Oxford Corridor sequence and the robustness to feature outliers, image noise rejection, and scalability to number of features is analyzed using the synthetic data experiments. Processing time comparison with the RANSAC and NMS algorithms indicate that the deterministic time requirement of the proposed and NMS algorithms is amenable to a variety of visual servo control applications.

Light detection and ranging (LiDAR) sensors provide accurate 3-D point clouds for noncooperative spacecraft pose estimation. Several robust methods, such as iterative closest point, exist to perform a local refinement of the pose starting from an initial estimate. However, finding the initial pose of the spacecraft is a global optimization problem, which is challenging to solve in real time. This is especially true on space hardware with limited computing power. In addition, many spacecrafts have a shape with multiple symmetries, making an unambiguous initial pose estimation impossible. This work introduces a convolutional-neural-network-based pose estimation method, accounting for potential symmetries of the target satellite. The point clouds are projected to a 2-D depth image before being processed by the network. To generate a sufficient amount of training data, a LiDAR simulator integrating multiple effects such as reflections or laser beam divergence is developed. While being trained solely on synthetic point clouds, the pose estimation method shows to be precise, efficient, and reliable when evaluated on real point clouds taken at a hardware-in-the-loop rendezvous test facility. A runtime evaluation on potential space computing hardware is also performed to demonstrate the applicability of the method to real-time onboard pose estimation.

Comunicació presentada a la 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), celebrada els dies 30 de maig a 3 de juny de 2017 a Washington DC, EUA.