A Lightweight Two-End Feature Fusion Network for Object 6D Pose Estimation

An unsupervised point cloud object retrieval and pose estimation method, called PCRP, is proposed in this work. It is assumed that there exists a gallery point cloud set that contains point cloud objects with given pose orientation information. PCRP attempts to register the unknown point cloud object with those in the gallery set so as to achieve content-based object retrieval and pose estimation jointly, where the point cloud registration task is built upon an enhanced version of the unsupervised R-PointHop method. Experiments on the ModelNet40 dataset demonstrate the superior performance of PCRP in comparison with traditional and learning based methods.

Joint object recognition and pose estimation using multiple-anchor triplet learning of canonical plane

Bucket pose estimation is a key technology for the intelligent mining excavator to realize automatic excavation and loading. The existing estimation methods mainly include two categories. The sensor measurement methods install sensors to detect the bucket pose, which suffer from large cumulative error and short service life caused by strong vibrations. The traditional registration methods estimate the bucket pose by registering its point clouds with the iterative closest point method, which easily fall into a local optimum. Recently, point cloud registration networks have been studied for object pose estimation based on balanced training samples. However, since the bucket speed changes obviously during excavation, the number of bucket samples at different poses is imbalanced. As a result, point cloud registration networks will easily forget the bucket features at some poses which have small samples, also called nondominant pose features, during learning. This will reduce the accuracy of bucket pose estimation. To handle the forgetting problem caused by the sample imbalance, we propose a novel memory-augmented registration network (MARNet) for bucket pose estimation. The MARNet consists of an encoder, a dual-pose-memory module (DPMM), and a decoder. The encoder extracts the global pose features from the input point clouds. The DPMM learns and memorizes the prototypical pose features covering all the samples by using the memory units at the training phase. Then, nondominant pose features are augmented by retrieving the stored prototypical pose features at the testing phase. At last, the decoder calculates the final poses. Our method can effectively alleviate the forgetting problem and strengthen the generalization by using the DPMM. The experimental results demonstrate that our method increases the estimation accuracy of rotation and translation by at least 1.49° and 0.021 m, respectively, for the bucket poses with small samples. This contributes to the development of intelligent mining excavators.

The next generation of service robots is to offer services relying heavily on visually guided manipulation, besides navigation, such services as errand, logistics, appliance, home keeping, etc. For the successful introduction of these services, it is critical to establish the consumer-level dependability in 3D object recognition and pose estimation in a natural setting where a large variation of environment, e.g., perspective, texture, form factor, illumination, occlusion, etc., is common. To address this problem, we propose an approach of the two-layered particle filter to the dependability in 3D object recognition and pose estimation. In the upper layer, a set of object pose candidates is identified and maintained in the search space as a set of super-particles, each of which is assigned a probability of the true pose and evolved in time along with the accumulation of further evidences. To define the object pose candidates, first, we acquire initially weak evidences quickly and interpret them in terms of possible object poses in space. These interpretations serve as the region of interest for detailed investigation by which the pose probabilities are computed for individual interpretations based on the likelihood and unlikelihood of various features available in the corresponding regions of interest. During the process of probability computation, we select the object pose candidates to be used as super-particles in the upper layer. In the lower level, the pose uncertainties associated with the individual candidates are represented as particles that are subject to the propagation in time. Finally, the experimental results support the strength of the proposed approach in the real environment in terms of its dependability in 3D object recognition and pose estimation.

In this work, we propose a new image feature extraction network for 6D pose estimation tasks. The majority of existing 6D pose estimation networks are based on the RGBD input, which is the fusion of images and point clouds. They are mainly working on keypoint extraction, point cloud feature extraction, and other tasks, however, image features, on the other hand, mainly rely on CNN networks that are currently advanced. With the development of Vision Transformer in the field of computer vision, the results on ImageNet demonstrate its benefit of extracting global features, and we see new directions that the Transformers can bring to the area of image feature extraction. ICTC is a Transformer paired with a CNN network that fuses the retrieved global features into each pixel and then does local feature extraction. This provides greater robustness to 6D pose estimation In complex situations such as shifting lighting conditions and object occlusion. Our method achieves 96% accuracy using the average ADD metric on the LineMOD dataset, demonstrating our method's effectiveness.

The 6D pose estimation of objects plays a crucial role in robot sorting technology. To simultaneously improve the accuracy and efficiency of pose estimation during the grasping process, this study proposes an innovative deep learning pose estimation method based on PVN3D. Specifically, this study deeply optimized the backbone network of the image extraction model, using dense connections and learnable grouped convolutions to improve model accuracy and reduce model complexity. At the same time, the core idea of convolutional neural networks is integrated into the point cloud feature extraction process, which uses dynamic convolution techniques to process point cloud features, making the processing of point cloud data more efficient and flexible. In addition, by introducing a parameter-free attention mechanism, the accuracy of the model has been further improved. Through extensive experimental verification on the publicly available LineMOD, YCB video datasets and other three core datasets under the BOP benchmark, the method proposed in this paper demonstrates significant advantages in both computational efficiency and estimation accuracy. It not only significantly improves the accuracy of pose estimation but also significantly enhances real-time performance.

6D pose estimation is an important branch in the field of vision measurement, and is widely used in the fields of robotics, autonomous driving and reality augmentation. The latest research trend in 6D pose estimation is to train a deep neural network to directly predict the 2D projection position of the 3D keypoint from the image, establish the corresponding relationship, and finally use the perspective-n-point (PnP) algorithm to perform pose estimation. The current challenge of pose estimation is that when objects are textureless, occluded or scene-clutterd, the detection accuracy is reduced, and most of the existing algorithm models are large and cannot accommodate real-time requirements. In this paper, we introduce a densely connected feature pyramid network (DFPN) that can efficiently integrate and utilize features. We combine the cross-stage partial network (CSPNet) with DFPN to design a new network for 6D pose estimation, DFPN-6D, a new approach for 6D object pose estimation. DFPN-6D can efficiently and accurately handle objects with textureless, occluded and scene clutter and estimate their full 6D poses in a single shot. Furthermore, we propose a new confidence calculation method and loss function for object pose estimation, which can fully consider spatial information. Finally, we propose a novel augmentation method for direct 6D pose estimation approaches to improve performance and generalization ability in the case of occlusion, which is called 6D augmentation. Our approach achieves a new state-of-the-art accuracy of 98.06 and 87.09 in terms of the ADD(-S) metric on the Linemod dataset and Occluded-Linemod dataset, and our method also achieves the best result in terms of the different metric on the MULT-I dataset, BIN-P dataset and T-LESS dataset, respectively, while still running end-to-end at over 65 FPS. The experimental results demonstrate that our algorithm is robust to textureless materials and occlusion while running more efficiently than other methods. We also deploy our proposed method to a real robot to grasp and manipulate objects based on the estimated pose.

A pose (relative attitude and position) estimation approach to support proximity operations such as autonomous landing of unmanned air vehicles or spacecraft on-orbit servicing is investigated in this paper. An algorithm is developed based on a feature detection capability using the point cloud data that a flash lidar (light detection and ranging) produces. The feature refers to an edge, a corner, or a distinct geometrical shape of an object. Once features are detected, orthogonal triads that are attached to the object are constructed using them. Pose estimation is then performed based on the orientation of the triads with respect to the lidar sensor frame. Since the algorithm employs detected features, it allows pose estimation of an object from a single set of point cloud data. In contrast, the conventional iterative closest point based algorithms typically require two sets of point cloud data. The estimated pose thus represents the changes in pose between the two data sets, leaving the pose of the object with respect to the sensor still to be determined. This featurebased pose estimation capability performs directly an initial pose acquisition with respect to the sensor. The algorithm may also help reduce the number of data points required for pose estimation since a small, selected feature of an object within the sensor field of view can be employed to estimate the pose. Employing the algorithm developed, pose estimation tests were conducted and their preliminary results are presented, which were obtained using a flash lidar in a laboratory testing environment.

In this paper, a novel global point cloud descriptor is proposed for reliable object recognition and pose estimation, which can be effectively applied to robot grasping operation. The viewpoint feature histogram (VFH) is widely used in three-dimensional (3D) object recognition and pose estimation in real scene obtained by depth sensor because of its recognition performance and computational efficiency. However, when the object has a mirrored structure, it is often difficult to distinguish the mirrored poses relative to the viewpoint using VFH. In order to solve this difficulty, this study presents an improved feature descriptor named orthogonal viewpoint feature histogram (OVFH), which contains two components: a surface shape component and an improved viewpoint direction component. The improved viewpoint component is calculated by the orthogonal vector of the viewpoint direction, which is obtained based on the reference frame estimated for the entire point cloud. The evaluation of OVFH using a publicly available data set indicates that it enhances the ability to distinguish between mirrored poses while ensuring object recognition performance. The proposed method uses OVFH to recognize and register objects in the database and obtains precise poses by using the iterative closest point (ICP) algorithm. The experimental results show that the proposed approach can be effectively applied to guide the robot to grasp objects with mirrored poses.

This paper presents an appearance-based method for object pose recognition using single camera image. The basic idea of our method is to find the correlation between object image and its pose, and use it for object recognition and pose estimation. Canonical correlation analysis is introduced to derive such correlation and build a compact appearance model. In the approach, we first obtain a pair of training data set, i.e., object images and their pose parameters. The appearance model is given as the subspace spanned by the canonical vectors that maximize the correlation between images and poses. Pose parameters of currently observed image is predicted by finding the regression coefficient in this subspace. We also introduce the kernel methods to cope with the non-linearity lies in training data set. Experiments are conducted on object pose estimation and vehicle type classification problem. Performance of our appearance models is discussed through the comparison with conventional subspace method

Purpose Accurate 6D object pose estimation is essential for various robotic tasks. Uncertain pose estimates can lead to task failures; however, a certain degree of error in the pose estimates is often acceptable. This paper aims to enable the robots to make informed decisions by quantifying errors in the object pose estimate and acceptable errors for task success. Design/methodology/approach In this paper, the authors introduce a framework for evaluating robotic task success under object pose uncertainty, representing both the estimated error space of the object pose and the acceptable error space for task success using multi-modal non-parametric probability distributions. The proposed framework pre-computes the acceptable error space for task success using dynamic simulations and subsequently integrates the pre-computed acceptable error space over the estimated error space of the object pose to predict the likelihood of the task succes. Findings The authors evaluated the proposed framework on two mobile manipulation tasks. Their results show that by representing the estimated and the acceptable error space using multi-modal non-parametric distributions, the authors achieve higher task success rates and fewer failures. Research limitations/implications Their proposed framework is generic and can be applied to a wide range of robotic tasks requiring object pose estimation. Hence, given the recent advancements in object pose uncertainty estimation and dynamic simulations, the proposed framework, in conjunction with these advancements, has the potential to enable robots to make reliable and informed decisions under pose uncertainty. Originality/value Unlike related works that model both acceptable error space and estimated error space using parametric uni-modal distributions, the authors model them as multi-modal distributions which is often the case in the real world.

Object recognition and pose estimation are critical components in autonomous robot manipulation systems, playing a crucial role in enabling robots to interact effectively with the environment. During actual execution, the robot must recognize the object in the current scene, estimate its pose, and then select a feasible grasp pose from the pre-defined grasp configurations. While most existing methods primarily focus on pose estimation, they often neglect the graspability and reachability aspects. This oversight can lead to inefficiencies and failures during execution. In this study, we introduce an innovative graspability-aware object pose estimation framework. Our proposed approach not only estimates the poses of multiple objects in clustered scenes but also identifies graspable areas. This enables the system to concentrate its efforts on specific points or regions of an object that are suitable for grasping. It leverages both depth and color images to extract geometric and appearance features. To effectively combine these diverse features, we have developed an adaptive fusion module. In addition, the fused features are further enhanced through a graspability-aware feature enhancement module. The key innovation of our method lies in improving the discriminability and robustness of the features used for object pose estimation. We have achieved state-of-the-art results on public datasets when compared to several baseline methods. In real robot experiments conducted on a Franka Emika robot arm equipped with an Intel Realsense camera and a two-finger gripper, we consistently achieved high success rates, even in cluttered scenes.

Coarse pose registration is a crucial step in 6DOF pose estimation. Assuming the objects to be estimated are regular and their 3D models are known, this paper presents local geometric features (LGFs) to estimate the poses of objects in point cloud. Coordinate frames describing the object's pose are defined according to geometric information. Pose estimation are performed by calculating the transformation between the frames in the scene point cloud and the reference model. Two kinds of LGFs are proposed: one is the corner edge feature (CEF) which exploits corner and radiating edges, and the other is the circle normal feature (CNF) which exploits circle center and normal vector of the plane where the circle is located. Included angle sequence of adjacent edges and circle radius, which are invariant to SE (3) transformation, are selected to identify CEF and CNF, respectively. Simulation is performed to verify the feasibility of the proposed features.

6D object pose estimation from a single RGB-D image is a fundamental problem in computer vision and robot manipulation. Despite recent advancements, existing methods still suffer several limitations. First of all, the object shape representation extracted from the depth map is often less expressive because the object point cloud parsed from the depth map is highly incomplete due to the object self-occlusion and noisy due to the sensor artifacts. This shape representation issue further intensifies when lacking sufficient labeled data for model training, which unfortunately is another typical problem for object pose estimation considering the heavy annotation cost for real-world pose labeling. In this study, we propose to tackle the above issues in a unified way. First, we enhance the object shape representation from the partial point cloud with a novel canonical shape reconstruction module, in which an implicit canonical frame is established by incorporating the SE(3) equivariance, achieving implicit feature alignment of the partial point cloud inputs, leading to robust shape recovery. Second, based on the enhanced object representation, we further utilize the de-canonicalized and pose-dependent completed object shape as the training signal, and develop a novel weakly-supervised learning framework to leverage both labeled synthetic data and unlabeled real data to train the pose estimation model in a label-efficient way. Extensive experiments on three widely used benchmarks demonstrate the effectiveness, and superiority of our framework over state-of-the-art methods.

Generalization is very important for pose estimation, especially for 3D pose estimation where small changes in the 2D images could trigger structural changes in the 3D space. To achieve generalization, the system needs to have the capability of detecting estimation errors by double-checking the projection coherence between the 3D and 2D spaces and adapting its network inference process based on this feedback. Current pose estimation is one-time feed-forward and lacks the capability to gather feedback and adapt the inference outcome. To address this problem, we propose to explore the concept of progressive inference where the network learns an observer to continuously detect the prediction error based on constraints matching, as well as an adjuster to refine its inference outcome based on these constraints errors. Within the context of 3D hand pose estimation, we find that this observer-adjuster design is relatively unstable since the observer is operating in the 2D image domain while the adjuster is operating in the 3D domain. To address this issue, we propose to construct two sets of observers-adjusters with complementary constraints from different perspectives. They operate in a dynamic sequential manner controlled by a decision network to progressively improve the 3D pose estimation. We refer to this method as Cross-Constrained Progressive Inference (CCPI). Our extensive experimental results on FreiHAND and HO-3D benchmark datasets demonstrate that the proposed CCPI method is able to significantly improve the generalization capability and performance of 3D hand pose estimation.