Learning push recovery for a bipedal humanoid robot with Dynamical Movement Primitives

Push recovery is an essential requirement for a humanoid robot with the objective of safely performing tasks within a real dynamic environment. In this environment, the robot is susceptible to external disturbance that in some cases is inevitable, requiring push recovery strategies to avoid possible falls, damage in humans and the environment. In this paper, a novel push recovery approach to counteract disturbance from any direction and any walking phase is developed. It presents a pattern generator with the ability to be modified according to the push recovery strategy. The result is a humanoid robot that can maintain its balance in the presence of strong disturbance taking into account its magnitude and determining the best push recovery strategy. Push recovery experiments with different disturbance directions have been performed using a 20 DOF Darwin-OP robot. The adaptability and low computational cost of the whole scheme allows is incorporation into an embedded system.

Bipedal humanoid robots have complex dynamics and they are intrinsically unstable. Although, bipedal robots have a similar kinematic architecture to a human and they are the most appropriate type of robots to operate in human environments, developing a humanoid robot that has robust is a difficult task. This paper proposes an omnidirectional walking engine that takes into account the push recovery strategies. The walking engine has a hierarchical structure and tries to fade the complexities of the dynamic walking. In addition, it can adapt to another platform with few changes. We enhanced the Linear Inverted Pendulum Plus Flywheel Model to release the height constraint of the center of mass. This enhancement allows a more human-like motion and provides more stable walking. The proposed walking engine has been successfully tested on a real humanoid soccer robot. The experimental results show the performance of this controller to generate stable walking. The average speed of walking that we have achieved was 20cm/sec.

This paper addresses a significant challenge in achieving collaborative tasks; how can a robot or multiple robots, endowed with a library of pre-learned primitive movements, generate multiple simultaneous coordinated robotic movements, adapting and optimizing those in the library, to complete one collaborative task? This work can thus be seen as a follow-up to the work with a motion presented as dynamic movement primitive (DMP) that now considers collaborative tasks and the existence of multiple robots/manipulators. Specifically, we start with a simple task using one DMP and extend it to accommodate the coordinated execution of multiple DMPs in robots with multiple manipulators or-alternatively-multiple robots with a single manipulator. We investigate mechanisms to jointly optimize multiple DMPs to perform one task in a coordinated fashion. The joint trajectory is built from initial DMPs learned for a single manipulator, and its optimization must comply with task-specific constraints. We illustrate the application of our approach both in a simulated environment and in a simulated and real Baxter robot.

Forestry cranes are an important tool for safe and efficient timber harvesting with forestry machines. However, their complex manual control often led to inefficiencies and excessive energy usage, due to the many joysticks and buttons that must be used in a precise sequence to perform efficient movements. To address this, the industry is increasingly turning to partial automation, making manual control more intuitive for the operator and, consequently, achieving improvements in energy efficiency. This article introduces a novel approach to energy-optimal motion planning that can be used along with a feedback control system to automate crane motions, taking over portions of the operator’s work. Our method combines dynamic movement primitives (DMPs) and an energy-optimization algorithm. DMPs is a machine learning technique for motion planning based on human demonstrations, while the optimization algorithm exploits the crane’s redundancy to find energy-optimal trajectories. Simulation results show that DMPs can replicate human-like controlled motions with a 25% reduction in energy consumption. However, our energy optimization algorithm shows improvements of over 40%, providing substantial energy savings and a promising pathway towards environmentally friendly partially automated machines.

Bipedal walking in human environments is made difficult by the unevenness of the terrain and by external disturbances. Most approaches to bipedal walking in such environments either rely upon a precise model of the surface or special hardware designed for uneven terrain. In this paper, we present an alternative approach to stabilize the walking of an inexpensive, commercially-available, position-controlled humanoid robot in difficult environments. We use electrically compliant swing foot dynamics and onboard sensors to estimate the inclination of the local surface, and use a online learning algorithm to learn an adaptive surface model. Perturbations due to external disturbances or model errors are rejected by a hierarchical push recovery controller, which modulates three biomechanically motivated push recovery controllers according to the current estimated state. We use a physically realistic simulation with an articulated robot model and reinforcement learning algorithm to train the push recovery controller, and implement the learned controller on a commercial DARwIn-OP small humanoid robot. Experimental results show that this combined approach enables the robot to walk over unknown, uneven surfaces without falling down.

Active balancing in autonomous humanoid robots is a challenging task due to the complexity of combining a walking gait with dynamic balancing, vision and high-level behaviors. Humans not only walk successfully over even and uneven terrain, but can recover from the interaction of external forces such as impacts with obstacles and active pushes. While push recovery has been demonstrated successfully in expensive robots, it is more challenging with robots that are inexpensive, with limited power in actuators and less accurate sensing. This work describes a closed-loop control method that uses an accelerometer and gyroscope to allow an inexpensive humanoid robot to actively balance while walking and recover from pushes. An experiment is performed to test three hand-tuned closed-loop control configurations; using only a the gyroscope, only the accelerometer, and a combination of both sensors to recover from pushes. Experimental results show that the combination of gyroscope and accelerometer outperforms the other methods with 100% recovery from a light push and 70% recovery from a strong push.

Impact of Body Parameters on Dynamic Movement Primitives for Robot Control

Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

This paper describes a real-time data collection framework and an adaptive machining learning method for constructing a real-time energy prediction model for a machine tool. To effectively establish the energy consumption pattern of a machine tool over time, the energy prediction model is continuously updated with new measurement data to account for time-varying effects of the machine tool, such as tool wear and machine tool deterioration. In this work, a real-time data collection and processing framework is developed to retrieve raw data from a milling machine tool and its sensors and convert them into relevant input features. The extracted input features are then used to construct the energy prediction model using Gaussian Process (GP) regression. To update the GP regression model with real-time streaming data, we investigate the use of sparse representation of the covariance matrix to reduce the computational and storage demands of the GP regression. We compare computational efficiency of sparse GP to that of full GP regression model and show the effectiveness of the sparse GP regression model for tracking the variation in the energy consumption pattern of the target machine.

Particulate matter (PM) emissions, including soot, pose a significant environmental concern. Traditional emission control approaches, such as engine calibration, often struggle to maintain consistent performance across varying driving conditions, engine speeds, and loads. This inconsistency leads to variations in pollutant output, making it challenging for internal combustion engines to comply with stringent emission standards under real-world driving conditions. Physics-based soot models, grounded in complex mathematical equations, have been developed to capture the mechanisms of soot formation. However, these models often face accuracy limitations and require substantial computational resources, rendering them impractical for real-time emission control. In contrast, Machine Learning (ML) techniques offer a more versatile and efficient approach to soot prediction. This paper introduces a machine learning methodology for predicting Soot Particle Numbers (PN). The approach integrates both physics-based and black-box strategies to create a grey-box model for predicting soot emissions from a 1600 cc Gasoline Direct Injection (GDI) engine across diverse load and speed conditions. The physical model is based on a validated 1D engine model in GT-SUITE, while the black-box model is formulated using multiple ML approaches exclusively on measured data. Ultimately, the grey-box model is developed through ML techniques based on measured data and combustion-related parameters obtained from the 1D engine physical model in GT-SUITE. The grey-box model’s results are compared with those of the black-box model. The ML techniques employed in this study include Gaussian Process Regression (GPR), Support Vector Machine (SVM), Artificial Neural Network (ANN), and an Ensemble of Regression Trees (ERT). The Minimum Redundancy Maximum Relevance (MRMR) and RreliefF feature selection methods are used to identify the most crucial features in soot PN prediction. The results indicate that the grey-box GPR exhibits superior performance for both single and multi-pulse injection strategies, achieving a coefficient of determination (R2) of 0.99.

We present a novel computational approach for extracting localized signals from smooth background distributions. We focus on datasets that can be naturally presented as binned integer counts, demonstrating our procedure on the CERN open dataset with the Higgs boson signature, from the ATLAS collaboration at the Large Hadron Collider. Our approach is based on Gaussian Process (GP) regression — a powerful and flexible machine learning technique which has allowed us to model the background without specifying its functional form explicitly and separately measure the background and signal contributions in a robust and reproducible manner. Unlike functional fits, our GP-regression-based approach does not need to be constantly updated as more data becomes available. We discuss how to select the GP kernel type, considering trade-offs between kernel complexity and its ability to capture the features of the background distribution. We show that our GP framework can be used to detect the Higgs boson resonance in the data with more statistical significance than a polynomial fit specifically tailored to the dataset. Finally, we use Markov Chain Monte Carlo (MCMC) sampling to confirm the statistical significance of the extracted Higgs signature.

To achieve bipedal walking in real human environments, a bipedal robot should be capable of modifiable walking both on uneven terrain with different heights and on flat terrain. In this paper, a novel walking pattern generator based on a 3-D linear inverted pendulum model (LIPM) is proposed to achieve this objective. By adopting a zero moment point (ZMP) variation scheme in real time, it is possible to change the center-of-mass (COM) position and the velocity of the 3-D LIPM throughout the single support phase. Consequently, the proposed method offers the ability to generate a modifiable pattern for walking on uneven terrain without the necessity for any extra footsteps to adjust the COM motion. In addition, a control strategy for bipedal walking on uneven terrain with unknown height is developed. The torques and ground reaction force are measured through force-sensing resisters (FSRs) on each foot and the foot of the robot is modeled as three virtual spring-damper models for the disturbance compensation. The methods for generating the foot and vertical COM of 3-D LIPM trajectories are proposed to achieve modifiable bipedal walking on uneven terrain without any information regarding the height of the terrain. The effectiveness of the proposed method is confirmed through dynamic simulations.

Legged robots require fast and accurate representation of their surrounding terrain to achieve behaviors such as running, push recovery, continuous walking, backflips, while also utilizing on-board computational resources efficiently. The desired tasks can be achieved efficiently by representing the environment using planar regions. However, existing methods for planar region extraction are either too slow or require significant compute time on the Central Processing Unit (CPU). In this work we exploit key properties of depth images and Graphical Processing Unit (GPU) to estimate planar regions around the robot at very high frame rates of 150-200 Hz. The proposed algorithm uses a set of fully customizable and interchangeable set of kernel layers on the GPU to process the depth map in parallel and generate a locally connected graph structure, which is later separated into planar components using a basic depth-first search. We test the proposed algorithm on the Atlas robot while performing different walking behaviors on oriented cinder blocks, as well as in simulation with simulated sensor and robot. The algorithm is open-sourced for research on legged robots and other fields.

BackgroundThe analysis of gene expression from time series underpins many biological studies. Two basic forms of analysis recur for data of this type: removing inactive (quiet) genes from the study and determining which genes are differentially expressed. Often these analysis stages are applied disregarding the fact that the data is drawn from a time series. In this paper we propose a simple model for accounting for the underlying temporal nature of the data based on a Gaussian process.ResultsWe review Gaussian process (GP) regression for estimating the continuous trajectories underlying in gene expression time-series. We present a simple approach which can be used to filter quiet genes, or for the case of time series in the form of expression ratios, quantify differential expression. We assess via ROC curves the rankings produced by our regression framework and compare them to a recently proposed hierarchical Bayesian model for the analysis of gene expression time-series (BATS). We compare on both simulated and experimental data showing that the proposed approach considerably outperforms the current state of the art.ConclusionsGaussian processes offer an attractive trade-off between efficiency and usability for the analysis of microarray time series. The Gaussian process framework offers a natural way of handling biological replicates and missing values and provides confidence intervals along the estimated curves of gene expression. Therefore, we believe Gaussian processes should be a standard tool in the analysis of gene expression time series.