Event-Triggered Nonlinear $$H_{\infty }$$ Control Design via an Improved Critic Learning Strategy

In this paper, the adaptive critic method is applied to solve a class of continuous-time event-based H ∞ control problems. The present control problem is transformed into a two-player zero-sum differential game and the adaptive critic structure is employed for coping with the event-based minimax optimization. Next, the event-based optimal feedback control and the time-based worst-case disturbance law are learned by training a critic neural network. In addition, the closed-loop system is constructed with stability analysis of the critic error dynamics and the sampled-data plant. The exclusion of the infamous Zeno behavior of the present event-based control design is also discussed. Finally, the control performance of the present approach is evaluated through simulation study.

Pure Strategy Equilibria in Symmetric Two-Player Zero-Sum Games

In this chapter, we focus on developing adaptive optimal regulators for a class of continuous-time nonlinear dynamical systems through an improved neural learning mechanism. The main objective lies in that establishing an additional stabilizing term to reinforce the traditional training process of the critic neural network, so that to reduce the requirement with respect to the initial stabilizing control, and therefore, bring in an obvious convenience to the adaptive-critic-based learning control implementation. It is exhibited that by employing the novel updating rule, the adaptive optimal control law can be obtained with an excellent approximation property. The closed-loop system is constructed and its stability issue is handled by considering the improved learning criterion. After that, we apply the adaptation-oriented approximate optimal control strategy to perform robust stabilization when including complex nonlinearity and uncertainty. By considering the dynamical uncertainties, it is proven that the developed near-optimal control law can achieve uniform ultimate boundedness of the closed-loop state vector, thereby guaranteeing a certain extent of robustness for the uncertain nonlinear plant. Simulation for a classical nonlinear system and experiment on an overhead crane are conducted to verify the efficient performance of the present design methods, especially the major role that the stabilizing term performed.

An undecidable statement regarding zero-sum games

For an optimal parametric linear quadratic (LQ) control problem, a design objective is to determine a controller of constrained structure such that the closed-loop system is asymptotically stable and an associated performance measure is optimized. In the presence of system uncertainty, the system obtained by parametric LQ design is further required to be robust in terms of maintaining the closed-loop stability with a guaranteed cost bound. This problem, referred to as robust optimal parametric LQ control with a guaranteed-cost bound, is addressed. A design method is proposed for finding an optimal controller that simultaneously guarantees robust stability and performance over a specified range of parameter variations. A versatile numerical algorithm is given for computing the robust optimal gains. The usefulness of the design method is demonstrated by the design of robust control of a VTOL helicopter. >

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.

Game solving is a relatively less explored area in general game playing. This paper introduces a translation from the Game Description Language GDL to Quantified Boolean Formulas (QBF) that lets us leverage QBF solvers to compute winning strategies in two-player games described in GDL. We implement this approach and measure the computation time needed by state-of-the-art QBF solvers on a range of two-player zero-sum turn-taking games. We introduce a variety of optimizations to the translation and evaluate them experimentally. Our empirical analysis establishes that our proposed approach is suitable for solving small games and can potentially help general game players evaluate endgame positions.

This work is concerned with the problem of designing stable invariant subspaces for the construction of discontinuity surfaces of variable structure control systems (VSCS). A new method of analyzing and designing the closed loop systems, such as the model following control systems and VSC systems with sliding mode, having desirable behavior in tracking or regulation is developed in this paper. The study has been motivated by the observation that a basic operator associated with the assignment of dynamics in feedback control system qualifies as a projector. Projector theory provides a neat method for analysis and design of VSCS to construct the sliding manifolds in class of differentiate manifolds. Reducibility and invariance properties of projections make them very attractive to decompose a dynamic system into lower order two subsystems decoupled from each other, so we get deep insight into problems in analysis and design tasks of control systems and more information about dynamics incorporated with "modes" and eigenvectors represent intrinsic properties of dynamical systems

Off-policy evaluation (OPE) is the problem of evaluating new policies using historical data obtained from a different policy. In the recent OPE context, most studies have focused on single-player cases, and not on multi-player cases. In this study, we propose OPE estimators constructed by the doubly robust and double reinforcement learning estimators in two-player zero-sum Markov games. The proposed estimators project exploitability that is often used as a metric for determining how close a policy profile (i.e., a tuple of policies) is to a Nash equilibrium in two-player zero-sum games. We prove the exploitability estimation error bounds for the proposed estimators. We then propose the methods to find the best candidate policy profile by selecting the policy profile that minimizes the estimated exploitability from a given policy profile class. We prove the regret bounds of the policy profiles selected by our methods. Finally, we demonstrate the effectiveness and performance of the proposed estimators through experiments.

In this paper, an event-triggered adaptive dynamic programming (ADP) algorithm is developed to solve the two-player zero-sum game problem of continuous-time nonlinear systems. First, a critic neural network is employed to approximate the optimal value function. Then, an event-triggered ADP method is proposed, which guarantees the stability of the closed-loop system. The developed method can save the amount of computation as the control law and disturbance law that update only when the pre-designed triggering condition is violated. Finally, its effectiveness is verified through simulation results.

This paper focuses on the problem of guaranteed cost control for Takagi-Sugeno (T-S) fuzzy systems with time-varying delayed state. A linear quadratic cost function is considered as a performance index of the closed-loop fuzzy system. Then, the guaranteed cost control of the closed-loop fuzzy system is discussed, and the sufficient conditions are provided for the construction of a guaranteed cost controller via state feedback and observer-based output feedback. When these conditions, which are given in terms of the feasibility of linear matrix inequalities (LMIs), are satisfied, the designed state feedback controller and observer-based controller gain matrices can be obtained via a convex optimization problem. Two illustrative examples are provided to demonstrate the effectiveness of the approaches proposed in this paper.

This paper studies the impulsive sequence design for finite-time set stabilization with probability one of impulsive probabilistic Boolean networks (IPBNs). Firstly, the equivalent algebraic form of IPBNs is obtained by using the semi-tensor product (STP) method. Secondly, the design of impulsive sequence is converted to the design of state feedback control. Thirdly, based on the concept of control-invariant subset with impulsive sequence (CISIS), a necessary and sufficient condition is put forward to solve the finite-time set stabilization with probability one. Finally, an example is given to demonstrate the obtained results.

This paper is concerned with the development of fully Autonomous Underwater Vehicles (AUVs) for deep ocean scientific and commercial applications. Specifically, it is regarding the design of automatic feedback control systems for AUV attitude control. The design method used in this paper formulates the AUV attitude control problem within the framework of the Optimal Control Theory. This approach determines an optimal control law that keeps the state of a dynamic system near a constant reference state while minimizing a quadratic cost functional. This study has developed four optimal control laws based on different performance criteria. First, the standard Linear Quadratic performance index was used to obtain a baseline control design. The LQR control law has been found to be undesirable because it resulted in a closed-loop system with fast and slow dynamics. In order to place the closed-loop eigenvalues close to each other, a pole placement algorithm has been used in connection with the LQR performance index. Two control laws have been designed based on this approach. Finally, a fourth design has been obtained by using a Hilbert Space Norm (also known as H/sub 2/ Norm or Root-Mean-Square) performance index. The main reason for designing a control law based on H/sub 2/ Norm performance index was to minimize the fuel consumption of the AUV for longer endurance. The stability and performance characteristics of the four optimal control laws have been analyzed via computer simulation and frequency response studies. All the closed-loop AUV systems exhibited overdamped response without any overshoot. Because the yaw and pitch channels have right hand zeros indicating the controllers have finite gain margins in these two channels. Hence, stability robustness of the AUV controllers can not be ignored and must be the subject of more detailed investigation in the future.

This paper investigates an adaptive fuzzy tracking control design problem for single-input and single-output uncertain nonstrict feedback nonlinear systems. For the cases of the states measurable and the states immeasurable, fuzzy logic systems are separately adopted to approximate the unknown nonlinear functions or model the uncertain nonlinear systems. In the unified framework of adaptive backstepping control design, both adaptive fuzzy state feedback and observer-based output feedback control design schemes are proposed. The stability of the closed-loop systems is proved by using Lyapunov function theory. The simulation examples are provided to confirm the effectiveness of the proposed control methods.

The μ analysis method is applied to a class of multivariable separable autonomous nonlinear feedback systems. The nonlinear element is replaced by its sinusoidal input describing function (SIDF) to develop a quasilinear nominal model for control analysis and design. Some properties of the ∞ -norm and structured singular values (SSVs) are shown to hold for the type of systems where the output is a function of control input-signal amplitude and frequency. The model uncertainty induced by the SDIF approximation is calculated and incorporated in the control design as an input multiplicative perturbation to the quasilinear nominal model. The μ analysis technique is then employed to study the stability and performance robustness of the nonlinear control system. Using small gain theorem, the condition for the absence of limit cycle is derived. The information obtained from this analysis is then used to design a bank of controllers using the H ∞ optimization-based μ synthesis technique. These controllers are designed such that the closed loop system is robust over a range of control-input amplitude. A simple scheme is then proposed to select the appropriate controller for the operating range. The simulation results illustrate that the proposed technique may be used to design a high performance robust controller for nonlinear systems and the absence of the limit cycle oscillation may be ensured.