A Robust Reinforcement Learning Control Design Method for Nonlinear System with Partially Unknown Structure

The antilock braking systems are designed to maximize wheel traction by preventing the wheels from locking during braking, while also maintaining adequate vehicle steerability; however, the performance is often degraded under harsh road conditions. In this study, a robust hybrid control system is developed for the antilock braking systems. In this design method, the robust hybrid control system is comprised of a hybrid controller and a robust controller to achieve a desired H/sup /spl infin// control performance. The hybrid controller containing a fuzzy uncertainty observer is the principal controller, and the robust controller is designed to achieve the desired H/sup /spl infin// tracking performance. Finally, simulations are performed to demonstrate the effectiveness of the proposed robust hybrid control system for antilock braking control under various road conditions.

This paper discusses an application of Robust Model Matching (RMM) to the spill position control of a fuel injection pump. Since the fuel injection servo mechanism is soaked in fuel oil, the parameters of the plant vary due to differences in the viscosity coefficient caused by wide changes in temperature. The invariable fuel injection characteristic is required to meet the demands of the drivability, exhaust emissions and fuel consumption, even though the parameters change. The RMM is an idea which makes up a robust control system against parameter uncertainties.In this paper, we design a robust discrete-time control system based on the RMM: (1) For the low sensitivity, to make small pulse transfer functions from equivalent disturbances to controlled variables. (2) For the robust stability, to increase the margin of pulse transfer functions from conceptual inputs to control variables. The experimental results show that this robust compensator makes up a robust control system against the parameter variations due to the change in temperature. And it is confirmed that the idea of the RMM is effective for the fuel injection pump which includes the variations, identification errors, friction, hysteresis and delay time of the calculation.

The complexity in planning and control of robot compliance tasks mainly results from simultaneous control of both position and force and inevitable contact with environments. It is quite difficult to achieve accurate modeling of the interaction between the robot and the environment during contact. In addition, the interaction with the environment varies even for compliance tasks of the same kind. To deal with these phenomena, in this paper, we propose a reinforcement learning and robust control scheme for robot compliance tasks. A reinforcement learning mechanism is used to tackle variations among compliance tasks of the same kind. A robust compliance controller that guarantees system stability in the presence of modeling uncertainties and external disturbances is used to execute control commands sent from the reinforcement learning mechanism. Simulations based on deburring compliance tasks demonstrate the effectiveness of the proposed scheme.

For the speed control system of autonomous electric vehicle (AEV), challenge happens with how to determine an appropriate driving speed to satisfy the dynamic environment while resisting uncertainty and disturbance. Therefore, this paper proposes a robust optimal speed control approach based on hierarchical architecture for AEV through combining deep reinforcement learning (DRL) and robust control. In decision‐making layer, a deep maximum entropy proximal policy optimization (DMEPPO) algorithm is presented to obtain an optimal speed via dynamic environment information, heuristic target entropy and adaptive entropy constraint. In motion control layer, to track the learned optimal speed while resisting uncertainty and disturbance, a robust speed controller is designed by the linear matrix inequality (LMI). Finally, simulation experiment results show that the proposed robust optimal speed control scheme based on hierarchical architecture for AEV is feasible and effective.

In this paper, we overview the design and development of robust multivariable control design to highlight some of the pragmatic automotive engineering control issues with an emphasis on a non-trivial application of Hybrid Electric Vehicle (HEV) Torque Management. We shall outline that most existing torque management approaches are not model-based dynamical systems, or do not address key concepts of robust stability and performance. We shall also discuss some advantages that robust control systems could offer to a wider automotive control applications. In part II, we show representative simulation results verifying that robust control systems design could be a breakthrough to the production of hybrid electric vehicles by yielding significant improvements on the emission, fuel consumption, calibration time and cost. Our contribution is aimed at the fact that cutting-edge robust controllers could be broadly adopted to tackle increasingly challenging problems of the automotive industry.

Rotorcraft flight control systems present design challenges which often exceed those associated with fixed-wing aircraft. First, large variations in the response characteristics of rotorcraft result from the wide range of airspeeds of typical operation (hover to over 100 kts). Second, the assumption of vehicle rigidity often employed in the design of fixed-wing flight control systems is rarely justified in rotorcraft where rotor degrees of freedom can have a significant impact on the system performance and stability. Proposed herein is a methodology for the design of robust rotorcraft flight control systems utilizing Quantitative Feedback Theory (QFI'), a technique which accounts for variability in the dynamic response of the controlled element in the design of robust control systems. It was developed to address a Multiple-Input-Single-Output (MISO) design problem, but was extended to the address the Multiple-Input-Multiple-Output (MIMO) flight control system (FCS) design problem of a UH-60 Black Hawk Helicopter. Inherent conservatism in this design technique leads to limitations in its utility. A modified QFI' approach proved a viable method for designing robust MIMO flight control systems. An analysis of the two flight control system design methodologies is presented.

Robust control theory is used to design stable controllers in the presence of uncertainties. This provides powerful closed‐loop robustness guarantees, but can result in controllers that are conservative with regard to performance. Here we present an approach to learning a better controller through observing actual controlled behaviour. A neural network is placed in parallel with the robust controller and is trained through reinforcement learning to optimize performance over time. By analysing nonlinear and time‐varying aspects of a neural network via uncertainty models, a robust reinforcement learning procedure results that is guaranteed to remain stable even as the neural network is being trained. The behaviour of this procedure is demonstrated and analysed on two control tasks. Results show that at intermediate stages the system without robust constraints goes through a period of unstable behaviour that is avoided when the robust constraints are included. Copyright © 2001 John Wiley & Sons, Ltd.

Since the computed torque or inverse dynamics control techniques are based on a good understanding of the controlled system dynamics and even its environment; the objection for the real-time application is unrealizable for the unknown systems. To tackle this problem, this study developed a robust adaptive fuzzy sliding-mode control system based on the H/sup /spl infin// tracking design technique. The developed control system is comprised of a fuzzy controller and a robust controller. The fuzzy controller is utilized to approximate an ideal controller by the tuning algorithms based on the Lyapunov sense, and the robust controller is designed to achieve H/sup /spl infin// robust tracking performance. To investigate the effectiveness of the developed control system, it is applied to a wing rock motion control system. Simulation results demonstrate that the proposed control system can achieve favorable tracking performances with unknown system dynamics.

For field tasks, it is quite challenged to operate in a complex environment for the underwater robots, especially for those with multiple control surfaces due to different response and gain characteristics. To this end, this paper develops a highly integrated robotic dolphin followed by a robust motion control system. For better maneuverability and fault-tolerant capabilities, a newly-designed robotic dolphin is presented, owning a wide array of sensors and multiple control surfaces, in which passive flukes are particularly applied. On this basis, a robust motion control system is proposed, including a depth controller based on velocity-related allocation strategies and a heading controller based on clearance compensation. In detail, considering the degradation of motion performance caused by passive flukes, a sliding mode controller for gain uncertainty and an allocation-related parameter tuning strategy for inputs response characteristics are designed. Extensive simulations and aquatic experiments are conducted, and the obtained results demonstrate the satisfied maneuverability of the designed prototype and the effectiveness of the proposed methods. This study can lay a foundation for further development of robotic dolphins with a robust motion system to execute complex tasks in the field. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is inspired by the issue of robust motion control system for a newly-designed practical robotic dolphin that possesses a passive tail and redundant control surfaces. The traditional methods are usually susceptible to uncertainties in the passive tail gain, exhibiting degraded control performance. Moreover, control oscillations and slow convergence speed often occur caused by neglecting the characteristics of different control surfaces, including response patterns and clearance. This paper suggests a robust depth controller based on velocity-related allocation strategies and a robust heading controller based on clearance compensation. Specifically, an allocation-related parameter tuning strategy is given by considering inputs response characteristics, including response speed, saturations, and hydrodynamic force variation patterns. To guarantee fine regulations of heading control, a nonlinear disturbance observer (NDOB)-based clearance compensation is proposed. Extensive aquatic experiments on the newly-designed robotic dolphin verified the effectiveness of the proposed methods. It is envisioned that all these presented results can provide valuable engineering practice insights for industry practitioners.

This paper proposes a new robust feedforward tracking servo system for optical disk recording systems in the case of a sudden disturbance for the optical disk recording system. In optical recording systems, the tracking servo system must suppress tracking error below its tolerance. This paper designs the robust feedback control system by using the coprime factorization and disturbance observer. The proposed robust feedback control system suppresses the sudden disturbance caused by walking and running. The detecting signal of optical disk recording systems is only a tracking error. Hence, the feedforward controller of the proposed tracking control system is constructed based on both Zero Phase Error Tracking (ZPET) control theory and prediction of tracking error. The experimental results point out that the proposed tracking servo system has a quick and precise tracking response and keeps the residual tracking error below its tolerance. © 2006 Wiley Periodicals, Inc. Electr Eng Jpn, 156(4): 60– 68, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/eej.20257

In the paper, the impact of atmospheric turbulence on the Unmanned Aerial Vehicles (UAV) motion is studied. The models of Dryden turbulence are discussed, and the airplane models of lateral directional under the atmospheric turbulence are deducted, the application of linear matrix inequality (LMI) is used to design a kind of robust flight control system based on H∞ control theory. The effect of the roll angle control of Robust flight controller is compared to that of PID controller and several characteristics of the state points in full-flight envelope are selected to validate the robust flight control system. The analysis shows that the designed controller is better than PID controller to attenuate the disturbance, and the flight control system has good dynamic performance and satisfactory robustness.

Various linear and nonlinear controllers have been developed to improve the dynamic performance of DC-DC converters. Most controllers can only be designed on the basis of understanding the mathematical model of DC-DC converter, but the inherent nonlinear and time-varying characteristics of DC-DC switching converter make it difficult to complete the precise modeling, so the model-based control design is complex and the control performance is limited. In order to overcome the problem, this paper proposes a reinforcement learning (RL) controller based on the twin-delayed deep deterministic policy gradient (TD3) algorithm. This controller does not need the model of the switching converter. The converter will be regarded as a black box model, the policy approximation function (policy neural network) can be trained and learned by constructing a Markov decision process interacting with the black box model in the control system, and the optimal control action can be output. The RL controller is developed based on actor critic architecture, and a TD3 algorithm with higher learning efficiency is proposed to improve the control performance of the RL controller. The proposed RL controller based on TD3 algorithm is compared with the traditional PI controller. The simulation results show that the RL controller has better dynamic performance when the converter starts and the load step changes.

A new cooperative error was defined firstly and a new cooperative model of the vehicle was developed for the vehicle's steering antilock braking stability fields. A new two-level fuzzy robust cooperative control system was proposed. The architecture was composed with the execution layer and the cooperation layer. The uncertain parameters deteriorate the performance of the systems, so the yaw moment and the front wheel angle robust controllers in the execution layer based on the desired control index were designed to improve the respond, robustness and stability of the vehicle. The vehicle anti-lock brake robust controller was designed. In the cooperation layer, a new practical cooperative control law based on the optimal index and robustness was devised. In order to eliminate the conservation of the controllers, the fuzzy control method was introduced. Then the brake force distribution policy applicable to the complex work condition was presented .

This paper proposes a sliding mode control with a disturbance estimation for an underwater robot. The mobility performance of an underwater robot is influenced by modeling error, observation noise, and several disturbances such as ocean current and tidal current. Therefore, a robust control system is needed for precise motion control of an underwater robot. This paper uses a sliding mode control, which is one of the robust control methods. In a sliding mode control, chattering tends to occur, if the switching gain is set to a high value. On the other hand, it is desirable to set the switching gain high from the viewpoint of robustness. Therefore, there is a trade-off between the switching gain and robustness. In the proposed method, the disturbance is estimated in real-time, and this estimated value is added to the control input. Most of the disturbances are compensated by this estimated value, and the sliding mode control is used for the rest of the disturbances. As a result, the robust control system is achieved by using the proposed method, even if the switching gain is set to a low value. The validity of the proposed method was confirmed from the simulation and experimental results.

Hybrid Passive/Active Control Performance in Buildings Using Robust Control Design Approaches