Patch-Based Pose Inference with a Mixture of Density Estimators

Correspondence-based six-degree-of-freedom (6-DoF) pose estimation remains a mainstream solution for 3D point cloud registration. However, the heavy outliers pose great challenges to this problem. In this paper, we propose a random sample consensus (RANSAC) variant based on sampling locally and hypothesis globally (SLHG) for 6-DoF pose estimation and 3D point cloud registration. The key novelties are efficient sampling by guiding the sampling process locally and accurate pose estimation by generating hypotheses with global information. SLHG first generates a correspondence subset via compatibility clustering on the initial set. Second, locally guided graph sampling is performed. Third, 6-DoF hypotheses are generated by incorporating global information with a voting scheme. The best hypothesis serves as the estimation result by repeating the second and third steps. Extensive experiments on four popular datasets and comparisons with state-of-the-art methods confirm that: SLHG manages to 1) achieve accurate registrations with a few iterations, and 2) yield better accuracy performance than most competitors.

Recent studies have demonstrated that high-level semantics in data can be captured using sparse representation. In this paper, we propose an approach to human body pose estimation in static images based on sparse representation. Given a visual input, the objective is to estimate 3D human body pose using feature space information and geometrical information of the pose space. On the assumption that each data point and its neighbors are likely to reside on a locally linear patch of the underlying manifold, our method learns the sparse representation of the new input using both feature and pose space information and then estimates the corresponding 3D pose by a linear combination of the bases of the pose dictionary. Two strategies for dictionary construction are presented: (i) constructing the dictionary by randomly selecting the frames of a sequence and (ii) selecting specific frames of a sequence as dictionary atoms. We analyzed the effect of each strategy on the accuracy of pose estimation. Extensive experiments on datasets of various human activities show that our proposed method outperforms state-of-the-art methods.

3D hand pose estimation can provide basic information about gestures, which has an important significance in the fields of Human-Machine Interaction (HMI) and Virtual Reality (VR). In recent years, 3D hand pose estimation from a single depth image has made great research achievements due to the development of depth cameras. However, 3D hand pose estimation from a single RGB image is still a highly challenging problem. In this work, we propose a novel four-stage cascaded hierarchical CNN (4CHNet), which leverages hierarchical network to decompose hand pose estimation into finger pose estimation and palm pose estimation, extracts separately finger features and palm features, and finally fuses them to estimate 3D hand pose. Compared with direct estimation methods, the hand feature information extracted by the hierarchical network is more representative. Furthermore, concatenating various stages of the network for end-to-end training can make each stage mutually beneficial and progress. The experimental results on two public datasets demonstrate that our 4CHNet can significantly improve the accuracy of 3D hand pose estimation from a single RGB image.

Marker-based 3D motion capture systems, widely used for gait analysis, are accurate but have disadvantages such as cost and accessibility. Whereas markerless pose estimation has emerged as a convenient and cost-effective alternative for gait analysis, challenges remain in achieving optimal accuracy. Given the limited research on the effects of camera location and orientation on data collection accuracy, this study investigates how camera placement affects gait assessment accuracy utilizing five smartphones. This study aimed to explore the differences in data collection accuracy between marker-based systems and pose estimation, as well as to assess the impact of camera location and orientation on accuracy in pose estimation. The results showed that the differences in joint angles between pose estimation and marker-based systems are below 5°, an acceptable level for gait analysis, with a strong correlation between the two datasets supporting the effectiveness of pose estimation in gait analysis. In addition, hip and knee angles were accurately measured at the front diagonal of the subject and ankle angle at the lateral side. This research highlights the significance of careful camera placement for reliable gait analysis using pose estimation, serving as a concise reference to guide future efforts in enhancing the quantitative accuracy of gait analysis.

For robot-assisted assembly of complex optical systems, the alignment is facilitated by an accurate pose estimation of its components. However, wavefront-based pose estimation is typically ill-conditioned due to the inherent geometry of conventional industrially manufactured optical components. Therefore, we propose a novel approach in this paper to increase wavefront-based pose estimation accuracy via the design of freeform optics. For this purpose, an optimization problem is derived that parameterizes the component's surfaces by a predetermined freeform surface model. To show the efficacy of our approach, we provide simulation results to compare the pose estimation accuracy for a variety of optical designs. As an application example for the resulting improved pose estimation, a hand-eye calibration of a wavefront sensor is performed. This calibration originates from the field of robotics and represents the identification of a sensor coordinate system with respect to a global reference frame. For quantitative evaluation, the calibrating results are first presented with the aid of simulation data. Finally, the practical feasibility is demonstrated using a conventional industrial robot and additively manufactured freeform lenses.

Joint object recognition and pose estimation using multiple-anchor triplet learning of canonical plane

We propose and demonstrate an efficient pose estimation method, specifically small tilt angle variation, for visual servo control system design using a monocular camera, eliminating the need for iterative computations. This paper addresses the challenges associated with accurately estimating the relative pose between a chaser, equipped with a monocular camera, and a target that provides visual cues, particularly in scenarios where the captured image shows only minimal variations in the target's tilt angle. Traditional approaches often rely on direct feedback methods from image data or designs for pose estimation and control separately, necessitating computationally expensive iterations for pose estimation or complex implementation of visual servoing. To address these challenges, we adopt an extended Kalman filter, known as a robust nonlinear observer, and propose a non‐iterative approach that integrates motion dynamics models and control inputs into pose estimation from image data. This approach enables efficient integration of pose estimation and control solely relying on a monocular camera, reducing computational costs and improving robustness. Experimental validation using the ball‐and‐plate system demonstrates the effectiveness of our proposed method. In our setup, the camera is deliberately positioned directly above the pose estimation target, creating a scenario with minimal image variation in the tilt angle. Despite this condition, our approach achieves efficient and accurate pose estimation, demonstrating its feasibility for high‐speed pose estimation and control without iterative computations.

SkeletonPose: Exploiting human skeleton constraint for 3D human pose estimation

UVMO: Deep unsupervised visual reconstruction-based multimodal-assisted odometry

The 6D pose estimation of objects plays a crucial role in robot sorting technology. To simultaneously improve the accuracy and efficiency of pose estimation during the grasping process, this study proposes an innovative deep learning pose estimation method based on PVN3D. Specifically, this study deeply optimized the backbone network of the image extraction model, using dense connections and learnable grouped convolutions to improve model accuracy and reduce model complexity. At the same time, the core idea of convolutional neural networks is integrated into the point cloud feature extraction process, which uses dynamic convolution techniques to process point cloud features, making the processing of point cloud data more efficient and flexible. In addition, by introducing a parameter-free attention mechanism, the accuracy of the model has been further improved. Through extensive experimental verification on the publicly available LineMOD, YCB video datasets and other three core datasets under the BOP benchmark, the method proposed in this paper demonstrates significant advantages in both computational efficiency and estimation accuracy. It not only significantly improves the accuracy of pose estimation but also significantly enhances real-time performance.

This paper explores clothing virtual display technology based on human pose estimation and applies it to the Unity3D platform. This technology combines human pose estimation and virtual character-driven approaches to achieve rapid real-time clothing try-on and display effects. Firstly, key point detection of the human body in input images or videos is performed using human pose estimation techniques. This accurately infers the positions and connectivity relationships of human body keypoints, capturing subtle changes in human pose. Kalman filtering techniques are applied to enhance the stability and accuracy of pose estimation. Finally, we apply virtual character-driven technology to the Unity3D platform, developing a real-time dressing application that incorporates the estimated human pose results into virtual character models, enabling real-time clothing changes. By animating virtual characters, dynamic clothing display effects can be achieved, allowing users to instantly view and experience the effects of different clothing styles in various poses.

Perception of head pose is useful for many face-related tasks such as face recognition, gaze estimation, and emotion analysis. In this paper, we propose a novel random forest based method for estimating head pose angles from single face images. In order to improve the effectiveness of the constructed head pose predictor, we introduce feature weighting and tree screening into the random forest training process. In this way, the features with more discriminative power are more likely to be chosen for constructing trees, and each of the trees in the obtained random forest usually has high pose estimation accuracy, while the diversity or generalization ability of the forest is not deteriorated. The proposed method has been evaluated on four public databases as well as a surveillance dataset collected by ourselves. The results show that the proposed method can achieve state-of-the-art pose estimation accuracy. Moreover, we investigate the impact of pose angle sampling intervals and heterogeneous face images on the effectiveness of trained head pose predictors.

IntroductionInertial measurement units have been proposed for automated pose estimationand exercise monitoring in clinical settings. However, many existing methodsassume an extensive calibration procedure, which may not be realizable inclinical practice. In this study, an inertial measurement unit-based poseestimation method using extended Kalman filter and kinematic chain modelingis adapted for lower body pose estimation during clinical mobility testssuch as the single leg squat, and the sensitivity to parameter calibrationis investigated.MethodsThe sensitivity of pose estimation accuracy to each of the kinematic modeland sensor placement parameters was analyzed. Sensitivity analysis resultssuggested that accurate extraction of inertial measurement unit orientationon the body is a key factor in improving the accuracy. Hence, a simplecalibration protocol was proposed to reach a better approximation forinertial measurement unit orientation.ResultsAfter applying the protocol, the ankle, knee, and hip joint angle errorsimproved to , and , without the need for any other calibration.ConclusionsOnly a small subset of kinematic and sensor parameters contributesignificantly to pose estimation accuracy when using body worn inertialsensors. A simple calibration procedure identifying the inertial measurementunit orientation on the body can provide good pose estimationperformance.

In this paper, we propose a method for estimating the camera pose using a classification neural network based on the idea of OCR. Some terrestrial robots can exhibit high maneuverability by freely tilting their upper bodies. When estimating the posture of such robots, posture estimation using IMU and gyroscopic sensors as in the case of drones is affected by the noise generated by the unevenness of the ground, making posture estimation difficult. Pose estimation using deep learning from camera images is one solution to these problems, and various studies have been conducted in the past. However, the accuracy of pose estimation using only inference by deep learning with camera images is extremely poor and is not practical. In order to solve this problem, this paper proposes a classification neural network based on the idea of OCR, which can ensure high inference accuracy in the pose estimation task.

We observe that human poses exhibit strong group-wise structural correlation and spatial coupling between keypoints due to the biological constraints of different body parts. This group-wise structural correlation can be explored to improve the accuracy and robustness of human pose estimation. In this work, we develop a self-constrained prediction-verification network to characterize and learn the structural correlation between keypoints during training. During the inference stage, the feedback information from the verification network allows us to perform further optimization of pose prediction, which significantly improves the performance of human pose estimation. Specifically, we partition the keypoints into groups according to the biological structure of human body. Within each group, the keypoints are further partitioned into two subsets, high-accuracy proximal keypoints and low-accuracy distal keypoints. We develop a self-constrained prediction-verification network to perform forward and backward predictions between these keypoint subsets. One fundamental challenge in pose estimation, as well as in generic prediction tasks, is that there is no mechanism for us to verify if the obtained pose estimation or prediction results are accurate or not, since the ground truth is not available. Once successfully learned, the verification network serves as an accuracy verification module for the forward pose prediction. During the inference stage, it can be used to guide the local optimization of the pose estimation results of low-accuracy keypoints with the self-constrained loss on high-accuracy keypoints as the objective function. Our extensive experimental results on benchmark MS COCO and CrowdPose datasets demonstrate that the proposed method can significantly improve the pose estimation results.KeywordsHuman pose estimationSelf-constrainedStructural inferencePrediction optimization