Control parallel double inverted pendulum by hierarchical reinforcement learning

The double inverted pendulum is a multivariable, nonlinear control system. The fuzzy controller of double inverted pendulum is studied in the paper. The mathematical model is gained by the Lagrange equation and the controllability of double inverted pendulum system in unstable equilibrium point is analyzed. On the base of above, the fuzzy controller for double inverted pendulum is designed by the fuzzy control theory. The dimensions and control rules of the fuzzy controller are depressed by designing a fusion function using the optimization control theory. Simulation and real-time control results prove that the fuzzy controller is effective and feasible.

Inverted pendulums are typical examples of unstable systems and have been used for the verification of designed control systems and/or control education in laboratories. In particular, the control of a double inverted pendulum has been known as a good example to show the power of the state space approach. Instead of these stabilizations of a pendulum at the upright state, the authors have studied a swing-up control of single pendulum from the stable hanging state to the upright position, which is a control to transfer the state of the system from one equilibrium state to the others. The control problem is a non-linear control problem and it can be a good example to show a power of non-linear control methods. In this paper a control method for the swing-up control of a double pendulum is proposed, and the experimental results show that the proposed method is effective for the control problem.

Deals with the modeling of a flexible inverted pendulum. The mathematical model of the controlled object used is a simplified one, which is approximated by rigid links and springs. The possibility of driving a simplified model of the flexible inverted pendulum using a double inverted pendulum with an elastic joint is studied. The experimental results show that the simplified model is sufficient to describe the flexible inverted pendulum, but with some modeling and identification errors. These uncertainty will excite the vibration. An H/sub infinity / control to stabilize the flexible inverted pendulum using the simplified model is presented. >

In this design, through the analysis of the physical model of the linear double inverted pendulum, the motion equation of the system is obtained by using the classical Newtonian mechanical formula, and the obtained equation is linearized at the stable point, and the state space model of the system is obtained. In order to realize the stable control of the trolley position and the secondary linkage in the linear inverted pendulum system, the linear quadratic optimal controller is used in this design to obtain an excellent output response. In order to design the optimal state feedback control matrix K, the shortcomings of weight matrices Q and R in general LQR control design that depend on experience selection and trial and error are overcome. This design attempts to apply PSO algorithm to LQR controller parameter optimization. PSO algorithm is applied to optimize Q parameters and R parameters of LQR weighting matrix of linear quadratic optimal controller, and K matrix is obtained. The matrix is applied to a fuzzy PID controller designed by the fusion algorithm to overcome the disadvantage that a single PSO cannot adapt itself. Finally, a comparative simulation study is carried out in the Simulink environment of Matlab software toolbox, which verifies the effectiveness of the fuzzy PID controller based on particle swarm optimization algorithm, laying a theoretical foundation for the practical application of the inverted pendulum system, and has certain application value.

Mathematical models and parameters with high accuracy are required for the swing-up control of a double inverted pendulum. This study proposes a mechanical structure of a double inverted pendulum having excellent matching characteristics with a mathematical model. The implementation of a parameter estimation method based on nonlinear optimization, which is essential for swing-up control, is also proposed. The proposed structure of a double inverted pendulum eliminates backlash using the direct drive of a BLDC motor, minimizes the unmodeled frictional force through bearing cleaning and oiling, minimizes unnecessary load torque using the double support of a timing pulley shaft and power transmission through a coupling, and constrains the pendulum rotation to one degree-of-freedom by adopting a dual-bearing joint. In addition, both 3D printing and CNC milling are appropriately used to satisfy the requirements for each mechanical part. The swing motion of the manufactured double inverted pendulum is measured using the lab-built light-weight rapid control prototyping 02 (LW-RCP02). The proposed parameter estimation method based on nonlinear optimization is implemented to match the response trajectory of the actual double inverted pendulum with that of a Simulink model implemented as an S-function. Finally, the effectiveness of the proposed double inverted pendulum design and parameter estimation method is confirmed by showing that the mathematical model that, based on the parameters estimated using the proposed method, accurately describes the response characteristics of the double inverted pendulum.

Control of a rotary inverted pendulum

The control of Inverted Pendulum (IP) is a hugely complex task. A great deal of nonlinearity is present inherently and as well as affected by the surrounding external conditions. The sliding mode controller (SMC) is very robust inherently. It is used in this paper to control the IP. This paper examines the designing of sliding mode controller (SMC) for a linear inverted pendulum (IP). The paper highlights the important features of the sliding mode and also throws ample lights on the designing guidelines. The paper puts special impetus on the mathematical modeling of the controller. The robustness of the design of SMC with proportional control is amply displayed with the help of simple mathematics. It gives rise to a controller which can control a highly nonlinear system like IP quite efficiently. The performance of the SMC is compared with fuzzy and PID controller. The edge this controller poses is the key aspect of this paper. External disturbances and internal inaccuracies are also introduced to the system to bring out the robustness of the controller to the fore. Background on sliding mode and the pendulum are provided. Simulation results are displayed in a vivid manner and explained suitably.

Two wheel mobile manipulator has the potential to become multi-skilled robot in the field of robotics engineering and it is already implemented by using inverted pendulum control. Centre of gravity position is controlled to achieve the balancing of robot. Due to the unbalanced nature and the nonholonomic property of the two-wheel mobile manipulator, balancing is challenging task. On the other hand sliding mode control is a good candidate to address the nonlinear/under actuated system. This paper presents the control methods for balancing of two-wheel mobile manipulator. This paper proposes a novel second order sliding mode controller using twisting algorithm, disturbance observer and dynamics of double inverted pendulum. Dynamics of double inverted pendulum is used to design the control equation of sliding mode controller. Twisting algorithm is used to design the control input. Disturbance observer is compensated model uncertainties and external disturbances. Posture of two wheel mobile manipulator can successfully stabilized by this novel controller.

An Inverted Pendulum (IP) is one of pendulum-cart laboratory setup consisted of a pole mounted on a cart. The system encompasses complex control problems and greatly challenges the researchers in designing a better and optimal controller. This paper aims to model the IP and to design a PID control that using Quadratic Optimal Tuning (QOT) scheme for stabilization and position control of a single link IP. Firstly, a nonlinear model of IP is derived using Newtonian method. Then an augment model that include PID scheme is formulated. Lastly, the gains of PID scheme is tuned using QOT. Matlab-Simulink environment is utilized to simulate the balancing and position control performance of IP. The results show that the stability of the system had been confirmed and the system had a comparable performance with the one which controlled by LQR controller.

Recently, some challenging tasks in multi-agent systems have been solved by some hierarchical reinforcement learning methods. Inspired by the intra-level and inter-level coordination in the human nervous system, we propose a novel value decomposition framework HAVEN based on hierarchical reinforcement learning for fully cooperative multi-agent problems. To address the instability arising from the concurrent optimization of policies between various levels and agents, we introduce the dual coordination mechanism of inter-level and inter-agent strategies by designing reward functions in a two-level hierarchy. HAVEN does not require domain knowledge and pre-training, and can be applied to any value decomposition variant. Our method achieves desirable results on different decentralized partially observable Markov decision process domains and outperforms other popular multi-agent hierarchical reinforcement learning algorithms.

This paper proposes a high-speed state transition control method for the I-PENTAR, which is an inverted pendulum type robot aiming at supporting humans. It is important for an inverted pendulum type robot to control its center of gravity during inverted pendulum control. However, robots aimed at supporting humans must carry loads. This changes the center of gravity of the robot arbitrarily, so these robots need robust control for center of gravity. In this paper, the wheels are controlled by the computed torque method during the standing up and sitting down motion. At the same time, the inclination angle is estimated by the disturbance observer, but control using the observer is performed only during inverted pendulum control. From the result of experiments, it was confirmed that I-PENTAR could realize quick and soft standing up and sitting down motion by the proposed method.

Control of a chain of two planar inverted pendulums is studied in this paper. The goal is to keep the pendulums at the upright position in spite of the free base point motion. First a piecewise continuous controller is developed which guarantees the global stability of the system in the sense of Lyapunov. This idealized controller, however, is not desirable from the implementation viewpoint. A continuous control law is then proposed. It is found that the continuous controller only guarantees the stability in a suitable sense. The effectiveness of the developed controllers is tested through computer simulations. The above inverted pendulums are then used to simulate human thorax and pelvic movement during walking. The simulation results are compared with those from gait measurements and found that the above pendulums are capable of producing displacements and torques for the human thorax and the pelvis during normal walking.

In this study, interval type2 fuzzy logic (IT2FL) and PID controller is designed for swing-up position control of double inverted pendulum (DIP) system. The double inverted pendulum system consists of two rigid bars connected by a revolute joint. Mass of the revolute joint is included in the dynamic model.Rigid bars in the system are assumed to experience planar motion. The pendulum system is connected to the base by means of a revolute joint. Torque provided through a motor mounted to the base is used for position control of the system. PID (Proportional-Derivative-Integral) and interval type2 fuzzy logic controllers are developed by using the same performance criteria for position control of double inverted pendulum system. IT2FL controller is similar with type1 fuzzy logic controller. IT2FL system provides soft decision boundaries, whereas a type-1 fuzzy logic system provides a hard decision boundary. Membership function in interval type2 fuzzy logic set as an area called Footprint of Uncertainty (FOU) which limited by two type1 membership function those are upper membership function (UMF) and lower membership function (LMF).System behaviour is obtained by computer simulation using developed controllers respectively. Computer simulation results are compared in order to evaluate applicability of developed controllers. MATLAB/Simulink software is used in computer simulations.

In this paper a model based fuzzy controller is designed and implemented. This algorithm is applied, by simulation, to the control of an inverted pendulum and compared with a LQR controller and an adaptive network based fuzzy controller. The model based fuzzy controller is also applied to the control of an experimental inverted pendulum.

Physical simulation is often proposed as a way to generate motion for interactive characters. A simulated character has the potential to adapt to changing terrain and disturbances in a realistic and robust manner. In this paper, we present a balancing control algorithm based on a simplified dynamic model, an inverted pendulum on a cart. The simplified model lacks the degrees of freedom found in a full human model, so we analyze a captured reference motion in a preprocessing step and use that information about human running patterns to supplement the balance algorithms provided by the inverted pendulum controller. At run-time, the controller plans a desired motion at every frame based on the current estimate of the pendulum state and a predicted pendulum trajectory. By tracking this time-varying trajectory, our controller creates a running character that dynamically balances, changes speed and makes turns. The initial controller can be optimized to further improve the motion quality with an objective function that minimizes the difference between a planned desired motion and a simulated motion. We demonstrate the power of this approach by generating running motions at a variety of speeds (3 m/s to 5m/s), following a curved path, and in the presence of disturbance forces and a skipping motion.