Dynamic Vehicle Pose Estimation with Heuristic L-Shape Fitting and Grid-Based Particle Filter

Vehicle pose estimation with light detection and ranging (LiDAR) is essential in the perception technology of autonomous driving. However, because of incomplete observation measurements and sparsity of the LiDAR point cloud, it is challenging to achieve satisfactory pose extraction based on 3D LiDAR with the existing pose estimation methods. In addition, the demand for real-time performance further increases the difficulty of the pose estimation task. In this paper, we propose a novel vehicle pose estimation method based on the convex hull. The extracted 3D cluster is reduced to the convex hull, reducing the subsequent computation burden while preserving essential contour information. Subsequently, a novel criterion based on the minimum occlusion area is developed for the search-based algorithm, enabling accurate pose estimation. Additionally, this criterion renders the proposed algorithm particularly well-suited for obstacle avoidance. The proposed algorithm is validated on the KITTI dataset and a manually labeled dataset acquired at an industrial park. The results demonstrate that our proposed method can achieve better accuracy than the classical pose estimation method while maintaining real-time speed.

Introduction. An important part of an automotive unmanned vehicle (UV) control system is the environment analysis module. This module is based on various types of sensors, e.g. video cameras, lidars and radars. The development of computer and video technologies makes it possible to implement an environment analysis module using a single video camera as a sensor. This approach is expected to reduce the cost of the entire module. The main task in video image processing is to analyse the environment as a 3D scene. The 3D trajectory of an object, which takes into account its dimensions, angle of view and movement vector, as well as the vehicle pose in a video image, provides sufficient information for assessing the real interaction of objects. A basis for constructing a 3D trajectory is vehicle pose estimation.Aim. To develop an automatic method for estimating vehicle pose based on video data analysis from a single video camera.Materials and methods. An automatic method for vehicle pose estimation from a video image was proposed based on a cascade approach. The method includes vehicle detection, key points determination, segmentation and vehicle pose estimation. Vehicle detection and determination of its key points were resolved via a neural network. The segmentation of a vehicle video image and its mask preparation were implemented by transforming it into a polar coordinate system and searching for the outer contour using graph theory.Results. The estimation of vehicle pose was implemented by matching the Fourier image of vehicle mask signatures and the templates obtained based on 3D models. The correctness of the obtained vehicle pose and angle of view estimation was confirmed by experiments based on the proposed method. The vehicle pose estimation had an accuracy of 89 % on an open Carvana image dataset.Conclusion. A new approach for vehicle pose estimation was proposed, involving the transition from end-to-end learning of neural networks to resolve several problems at once, e.g., localization, classification, segmentation, and angle of view, towards cascade analysis of information. The accuracy level of end-to-end learning requires large sets of representative data, which complicates the scalability of solutions for road environments in Russia. The proposed method makes it possible to estimate the vehicle pose with a high accuracy level, at the same time as involving no large costs for manual data annotation and training.

Light detection and ranging (LiDAR) sensors provide accurate 3-D point clouds for noncooperative spacecraft pose estimation. Several robust methods, such as iterative closest point, exist to perform a local refinement of the pose starting from an initial estimate. However, finding the initial pose of the spacecraft is a global optimization problem, which is challenging to solve in real time. This is especially true on space hardware with limited computing power. In addition, many spacecrafts have a shape with multiple symmetries, making an unambiguous initial pose estimation impossible. This work introduces a convolutional-neural-network-based pose estimation method, accounting for potential symmetries of the target satellite. The point clouds are projected to a 2-D depth image before being processed by the network. To generate a sufficient amount of training data, a LiDAR simulator integrating multiple effects such as reflections or laser beam divergence is developed. While being trained solely on synthetic point clouds, the pose estimation method shows to be precise, efficient, and reliable when evaluated on real point clouds taken at a hardware-in-the-loop rendezvous test facility. A runtime evaluation on potential space computing hardware is also performed to demonstrate the applicability of the method to real-time onboard pose estimation.

PurposeMany metal workpieces have the characteristics of less texture, symmetry and reflectivity, which presents a challenge to existing pose estimation methods. The purpose of this paper is to propose a pose estimation method for grasping metal workpieces by industrial robots.Design/methodology/approachDual-hypothesis robust point matching registration network (RPM-Net) is proposed to estimate pose from point cloud. The proposed method uses the Point Cloud Library (PCL) to segment workpiece point cloud from scenes and a trained-well robust point matching registration network to estimate pose through dual-hypothesis point cloud registration.FindingsIn the experiment section, an experimental platform is built, which contains a six-axis industrial robot, a binocular structured-light sensor. A data set that contains three subsets is set up on the experimental platform. After training with the emulation data set, the dual-hypothesis RPM-Net is tested on the experimental data set, and the success rates of the three real data sets are 94.0%, 92.0% and 96.0%, respectively.Originality/valueThe contributions are as follows: first, dual-hypothesis RPM-Net is proposed which can realize the pose estimation of discrete and less-textured metal workpieces from point cloud, and second, a method of making training data sets is proposed using only CAD models with the visualization algorithm of the PCL.

We consider a category-level perception problem, where one is given 3D sensor data picturing an object of a given category (e.g., a car), and has to reconstruct the pose and shape of the object despite intra-class variability (i.e., different car models have different shapes). We consider an active shape model, where—for an object category— we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape estimation are formulated as a non-convex optimization. Our first contribution is to provide the first certifiably optimal solver for pose and shape estimation. In particular, we show that rotation estimation can be decoupled from the estimation of the object translation and shape, and we demonstrate that (i) the optimal object rotation can be computed via a tight (small-size) semidefinite relaxation, and (ii) the translation and shape parameters can be computed in closed-form given the rotation. Our second contribution is to add an outlier rejection layer to our solver, hence making it robust to a large number of misdetections. Towards this goal, we wrap our optimal solver in a robust estimation scheme based on graduated non-convexity. To further enhance robustness to outliers, we also develop the first graph-theoretic formulation to prune outliers in category-level perception, which removes outliers via convex hull and maximum clique computations; the resulting approach is robust to 70 − 90% outliers. Our third contribution is an extensive experimental evaluation. Besides providing an ablation study on a simulated dataset and on the PASCAL3D+ dataset, we combine our solver with a deep-learned keypoint detector, and show that the resulting approach improves over the state of the art in vehicle pose estimation in the ApolloScape driving datasets.

Six dimensional pose prediction of objects for robotic manipulation has received much attention in industrial applications. Despite considerable research on improving the performance of pose estimation, the problem remains challenging for industrial objects because of their textureless and mostly homogeneous properties in the presence of clutter and occlusion. This article proposes deep learning-based pose estimation by jointly detecting, segmenting and pose predicting. The binary classification branch as an attention module is suggested to improve the accuracy of instance segmentation. Using the instance information, the initial pose estimation network is designed by fusing the depth information into the grayscale image to strengthen the geometric features. Then, to obtain a highly accurate pose, an iterative network is constructed with point clouds as inputs to refine the initial pose. The networks are implemented to predict the pose of textureless object on the synthetic and real scenes. Experimental results indicate the pose estimation method is efficient and robust to pose prediction of textureless objects in cluttered and occluded scenes.

Aiming at the difficulty and low efficiency of pose estimation in the process of automatic drug picking, a pose estimation method combining plane segmentation and key point matching is proposed. First, a point cloud hierarchical clustering algorithm is used to segment the plane area, and the improved secondary key point matching method is used to identify and locate the pill box in each thread, which is convenient for multi-thread processing. Then the point cloud plane normal vector estimation method based on least squares is used to realize the posture reconstruction of the pill box. 600 images with viewpoint change, rotation transformation and scale change were collected for testing, the experimental results show that the pose estimation speed of the medicine box can reach 11 frames/s, the positioning deviation is less than 5 mm, and the pose estimation deviation is within 2°. So the method is efficient and can accurately identify the pose of the medicine box.

A pose (relative attitude and position) estimation approach to support proximity operations such as autonomous landing of unmanned air vehicles or spacecraft on-orbit servicing is investigated in this paper. An algorithm is developed based on a feature detection capability using the point cloud data that a flash lidar (light detection and ranging) produces. The feature refers to an edge, a corner, or a distinct geometrical shape of an object. Once features are detected, orthogonal triads that are attached to the object are constructed using them. Pose estimation is then performed based on the orientation of the triads with respect to the lidar sensor frame. Since the algorithm employs detected features, it allows pose estimation of an object from a single set of point cloud data. In contrast, the conventional iterative closest point based algorithms typically require two sets of point cloud data. The estimated pose thus represents the changes in pose between the two data sets, leaving the pose of the object with respect to the sensor still to be determined. This featurebased pose estimation capability performs directly an initial pose acquisition with respect to the sensor. The algorithm may also help reduce the number of data points required for pose estimation since a small, selected feature of an object within the sensor field of view can be employed to estimate the pose. Employing the algorithm developed, pose estimation tests were conducted and their preliminary results are presented, which were obtained using a flash lidar in a laboratory testing environment.

Robotic pollination represents a pivotal component of smart agriculture, with foundational architectures for target recognition, path planning, and motion control having been progressively established. However, developing an efficient and robust pollination system that integrates perception, decision-making, and execution within real-world scenarios remains confronted with complex challenges. This study systematically reviews recent advancements in the field and distills the core technical issues of greenhouse robotic pollination into three primary domains: target detection and pose estimation, end-effector design, and pollination strategies combined with motion control. Focusing on the visual perception of flowers, actuator architecture, and operational tactics, this review synthesizes existing academic findings to evaluate the state-of-the-art in flower detection and pose estimation, characterize diverse end-effector designs, and analyze the evolutionary trajectory of motion control techniques. Specifically, the analysis encompasses the impact of detection algorithms on recognition accuracy and robustness, the structural classification and performance attributes of pollination mechanisms, and the optimization of control strategies. Furthermore, the study categorizes global research backgrounds, technical methodologies, and paradigmatic system cases, offering a critical evaluation of experiences in constructing automated pollination systems. Despite these advances, current robotic pollination technologies for peppers (chili) face significant bottlenecks characterized by immature methods for precise flower detection and pose estimation, the need for optimized specialized end-effector designs, and insufficient robustness in decision-making systems under dynamic environmental conditions. To address these issues, future development should prioritize constructing diverse, large-scale flower image and pose datasets while developing detection algorithms adaptable to complex environments to achieve high-precision identification. Additionally, implementing this system requires a hierarchical architecture where perception drives adaptive actuation. Deep learning models must localize flower targets and assess maturity in real-time, feeding coordinates to path planners that generate collision-free trajectories through foliage. These trajectories are executed via multimodal motion control, synchronizing the rigid manipulator with soft end-effectors. By embedding tactile feedback into the machine learning loop, the system creates a unified sensorimotor framework. This enables dynamic force modulation based on physical resistance, ensuring precise, non-destructive pollination tailored to chili plants.

Vehicle pose estimation from a monocular image is a crucial and challenging problem in computer vision and robotics. Generally, current benchmark methods mostly combine expensive LIDAR sensors and stereo RGB images for 6 degrees of freedom(6DoF) pose estimation algorithms; however, single RGB image-based approaches suffer from a dramatically decreased performance. In this paper, we propose a method to reduce the gap by reformulating the monocular 6DoF vehicle pose estimation problem as a depth-aware imbalance learning. We integrate classification networks with regression networks to solve the imbalance. To address the problem of inaccurate prediction depth with a monocular image, we further design a depth-aware structure, which extracts a more reliable feature of location and improves 3D surroundings perception. As a result, our method surpasses the state-of-the-art methods, even with straightforward end-to-end training and limited imbalanced data. The experimental results on the challenging Apollocar3d [1] dataset show that the method outperforms the state-of-the-art significantly, achieving improvements on the standard metric up to 37.4%.

An unsupervised point cloud object retrieval and pose estimation method, called PCRP, is proposed in this work. It is assumed that there exists a gallery point cloud set that contains point cloud objects with given pose orientation information. PCRP attempts to register the unknown point cloud object with those in the gallery set so as to achieve content-based object retrieval and pose estimation jointly, where the point cloud registration task is built upon an enhanced version of the unsupervised R-PointHop method. Experiments on the ModelNet40 dataset demonstrate the superior performance of PCRP in comparison with traditional and learning based methods.

Pose detection of objects is an important topic in object-level mapping and indoor localization. In the past, pose estimation methods were performed either with the help of artificial markers or natural features found on the object. However, due to the fact that the markers can only be utilized in controlled environment experiments, the application of marker-based approaches is very limited. Furthermore, methods that depend on the object's natural visual features require texture on the object and lack robustness to illumination and camera viewpoint variations. With the advent of Deep Learning (DL), the classical pose estimation methods have been outperformed. The DL-based pose estimation can detect deep features of the object and exhibits higher robustness to many distortions and variabilities caused by the changes in the illumination and viewpoint conditions. However, the massive training data set requirement is the main challenge with most DL-based methods. The training set is often a real set of images that have been manually labeled or annotated. In addition, such methods face problems related to the degradation of their predicted accuracy in the presence of uncertainties due to the symmetrical structure of many objects. To address the aforementioned issues, a novel and very fast method for generating synthetic data, as well as a contour-based technique for accurate pose estimation (that can handle pose ambiguities for a symmetrical object) are proposed in this paper. The tests that are conducted in multiple indoor scenarios demonstrate not only the effectiveness of the synthetic data generation but also exhibit, in many cases, the very high accuracy of the proposed pose estimation method.

This paper presents the extension of our previous work on moment invariants with respect to centroid point (obtained from frontal mugs hot images) for thermal-based face identification system. In previous work, seeded region growing method was applied for background filtering. Sequentially, the system decomposes a background filtered thermal image into 4 thermal regions via 3-valued threshold method. In the extension of the pre-processes, we employed anis tropic diffusion between these two steps. This step aids in decreasing the effect of noise. Furthermore, previously we only considered frontal mugs hot images as registered images, therefore, in this paper we include mid-profiles and left and right profile in the registered dataset. This is to overcome for images with angular pose that exceeds 45 degrees to the left and right. Due to multiple angular pose available in the registered dataset, customized and simple pose estimation is introduced in this paper. The pose estimation utilizes information previously obtained during the pre-processes, thus avoiding additional steps that is required if current available pose estimation methods were to be employed. This system employs minimum distance measurement method for classification purposes. The encouraging performance of this system is represented by a cumulative match characteristic curve (also known as the CMC curve) at the end of this paper.

This paper presents a cost-effective framework for indoor localization that integrates spherical panoramas, smartphone cameras, and Building Information Modeling (BIM). We propose two methods for panorama pose estimation: Panorama Pose Estimation via Attribute Matching (PPE-AM) and Panorama Pose Estimation via Genetic Algorithm (PPE-GA). PPE-AM provides an initial pose estimate by matching attributes extracted from the panorama with those rendered from the BIM. PPE-GA refines this estimate using a genetic algorithm that aligns semantic and depth maps. Our approach eliminates the need for pre-mapping and on-site data collection, offering a practical alternative to expensive solutions such as spherical panorama localization based on point cloud or 3D line maps. Experimental results in a typical indoor environment demonstrate the system’s accuracy, with PPE-AM achieving a mean 3D error of 0.74 meters, and PPE-GA further improving it to 0.47 meters. Smartphone Pose Estimation (SPE), leveraging the localized panoramas, achieved a mean 3D error of 0.92 meters. The proposed framework offers a robust and scalable solution for accurate indoor localization, with potential applications in augmented reality, facility management, and digital twin technologies.

Deep learning method for 6D object pose estimation based on RGB image and depth (RGB-D) has been successfully applied to robot grasping. The fusion of RGB and depth is one of the most important difficulties. Previous works on the fusion of these two features are mostly concatenated together without considering the different contributions of the two types of features to pose estimation. We propose a selective embedding with gated fusion structure called SEGate, which can adjust the weights of RGB and depth features adaptively. Furthermore, we aggregate the local features of point clouds according to the distance between them. More specifically, the close point clouds contribute a lot to local features, while the distant point clouds contribute a little. Experiments show that our approach achieves the state-of-art performance in both LineMOD and YCB-Video datasets. Meanwhile, our approach is more robust to the pose estimation of occluded objects.