Relatively Optimal Control for Continuous-Time Systems

Online learning is an important property of adaptive dynamic programming (ADP). Online observations contain plentiful dynamics information, and ADP algorithms can utilize them to learn the optimal control policy. This paper reviews the research of online ADP algorithms for the optimal control of continuous-time systems. With the intensive study, ADP has been developed towards model free and data efficient. After separately introducing the algorithms, we compare their performance on the same problem. This paper is desired to provide a comprehensive understanding of continuous-time online ADP algorithms.

A new design for pole assignment in complex plane regions is presented. Vectorial bounds for structured uncertainties such that the assigned poles remain in these regions are given. Application to the linear constrained regulator problem is achieved. This approach makes a link between the pole assignment procedure used for constrained control and sufficient conditions established to check the positions of eigenvalues in the complex plane for a given matrix.

Motivated by the fact that determining a feedback solution for the optimal control problem under constraints is a hard task we introduce the concept of relative optimality, roughly optimality for a specific (nominal) plant initial condition. We consider a generic discrete-time finite-horizon constrained optimal control problem for linear systems, and we seek for a state feedback (possibly dynamic) controller. As a fundamental requirement, we do not admit preactions or controller-state initialization based on the plant initial state and we assume our controller to be time-invariant. In particular, we do not consider controllers simply achieved by the feedforward and tracking of the optimal trajectory. A relatively optimal control is a stabilizing controller such that, if initialized at its zero state, produces the optimal (constrained) trajectory for the nominal initial condition of the plant. We show that one of such controllers is linear, dead-beat, and its order is equal to the length of the horizon minus the plant order, thus, of complexity which is known a priori. Some additional features such as the assignment of the compensator poles to achieve strong stabilization are proposed. We show that, by means of the proposed approach, we can face several problems such as optimal point-to-point operations, optimal impulse response and optimal tracking.

It has been shown that a generic r-tuple of m input p output dynamical systems is simultaneously stabilizable (pole assignable) if r<m+P and the McMillian degrees of the systems are not too different. This result improves the currently known upper bound: r/spl les/max(m, p) on the number of plants. In the case r/spl les/max(m, p), an upper bound on the degree of the simultaneous pole assigning compensator has also been derived, which improves the bound obtained by Ghosh and Byrnes (1983).

This paper investigates optimal active control schemes applied to a point absorber wave energy converter within a receding horizon fashion. A variational formulation of the power maximization problem is adapted to solve the optimal control problem. The optimal control method is shown to be of a bang-bang type for a power take-off mechanism that incorporates both linear dampers and active control elements. We also consider a direct transcription of the optimal control problem as a general nonlinear program. A variation of the projected gradient optimization scheme is formulated and shown to be feasible and computationally inexpensive compared to a standard NLP solver. Since the system model is bilinear and the cost function is non-convex quadratic, the resulting optimization problem is not a convex quadratic program. Results will be compared with an optimal command latching method to demonstrate the improvement in absorbed power. Time domain simulations are generated under irregular sea conditions.

In optimal control loops delays can occur, for example through transmission via digital communication channels. Such delays influence the state that is generated by the implemented control. We study the effect of a delay in the implementation of L2-norm minimal Neumann boundary controls for the wave equation. The optimal controls are computed as solutions of problems of exact optimal control, that is if they are implemented without delay, they steer the system to a position of rest in a given finite time T. We show that arbitrarily small delays δ> 0 can have a destabilizing effect in the sense that we can find initial states such that if the optimal control u is implemented in the form yx(t,1) = u(t − δ) for t>δ, the energy of the system state at the terminal time T is almost twice as big as the initial energy. We also show that for more regular initial states, the effect of a delay in the implementation of the optimal control is bounded above in the sense that for initial positions with derivatives of BV-regularity and initial velocities with BV-regularity, the terminal energy is bounded above by the delay δ multiplied with a factor that depends on the BV-norm of the initial data. We show that for more general hyperbolic optimal exact control problems the situation is similar. For systems that have arbitrarily large eigenvalues, we can find terminal times T and arbitrarily small time delays δ, such that at the time T + δ, in the optimal control loop with delay the norm of the state is twice as large as the corresponding norm for the initial state. Moreover, if the initial state satisfies an additional regularity condition, there is an upper bound for the effect of time delay of the order of the delay with a constant that depends on the initial state only.

In this paper a new design procedure is proposed that combines linear quadratic optimal control (LQ) with pole assignment. In LQ-problems the quadratic weights are usually found by trial and error to get good time behavior. In contrast the proposed method modifies a given state weighting matrix such that all closed-loop poles are located in a specified region of the complex s-plane. It is shown that this can be done with the smallest possible increase of the quadratic performance index under consideration. The procedure is extended to the design of robust control systems where the plant is described by a family of linear systems according to different operational conditions. The control of a F4E aircraft demonstrates the achievements of the new procedure.

This paper presents a now method of optimal digital PID feedback control for a linear multivariate system by using a, parameter optimization technique. The most difficult points for the design of optimal digital PID feedback control system by the modern control theory are : (1) the control structure of it is specified in advance, (2) unless all the state variables are fed back, the optimal control cannot be obtained by dynamic programming or the maximum principle. However, the optimal digital PID feedback gains are determined by using the parameter optimization technique in this paper. When there are some states which are not fed back in the control system, the optimal feedback gains depend on the initial states. First, the optimal solution is presented in the case where all the initial Btates are known. Next, it is extended to the case where the initial states are random variables and their statistical properties are known,

A EROSPACE engineering applications greatly stimulated the development of optimal control theory during the 1950s and 1960s, where the objective was to drive the system states in such a way that some defined cost was minimized. This turned out to have very useful applications in the design of regulators (where some steady state is to be maintained) and in tracking control strategies (where some predetermined state trajectory is to be followed). Among such applications was the problem of optimal flight trajectories for aircraft and space vehicles. Linear optimal control theory in particular has been very well documented and widely applied, where the plant that is controlled is assumed linear and the feedback controller is constrained to be linear with respect to its input. However, the availability of powerful low-cost microprocessors has spurred great advantages in the theory and applications of nonlinear control. The competitive era of rapid technological change, particularly in aerospace exploration, now demands stringent accuracy and cost requirements in nonlinear control systems. This has motivated the rapid development of nonlinear control theory for application to challenging, complex, dynamical real-world problems, particularly those that bear major practical significance in aerospace, marine, and defense industries. Infinite-time horizon nonlinear optimal control (ITHNOC) presents a viable option for synthesizing stabilizing controllers for nonlinear systems by making a state-input tradeoff, where the objective is to minimize the cost given by a performance index. The original theory of nonlinear optimal control dates from the 1960s. Various theoretical and practical aspects of the problem have been addressed in the literature over the decades since. In particular, the continuous-time nonlinear deterministic optimal control problem associated with autonomous (time-invariant) nonlinear regulator systems that are affine (linear) in the controls has been studied by many authors. The long-established theory of optimal control offers quite mature and well-documented techniques for solving this control-affine nonlinear optimization problem, based on dynamic programming or calculus of variations, but their application is generally a very tedious task. Bellman’s dynamic programming approach reduces to solving a nonlinear first-order partial differential equation (PDE), expressed by the Hamilton–Jacobi–Bellman (HJB) equation. The solution to the HJB equation gives the optimal performance/cost value (or storage) function and determines an optimal control in feedback form under some smoothness assumptions. Alternatively, in the classical calculus of variations, optimal control problems can be characterized locally in terms of the Hamiltonian dynamics arising from Pontryagin’s minimum principle. These are the characteristic equations of the HJB PDE, which result in a nonlinear, constrained two-point boundary value problem (TPBVP) that, in general, can only be solved by successive approximation of the optimal control input using iterative numerical techniques for each set of initial conditions. Numerically, even though the nonlinear TPBVP is somewhat easier to solve than the HJBPDE, control signals can only be determined offline and are thus best suited for feedforward control of plants for which the state trajectories are known a priori. Therefore, contrary to the dynamic programming approach, the resultant control law is not generally in feedback form. Open-loop control, however, is sensitive to random disturbances and requires that the initial state be on the optimal trajectory. In contrast, nonlinear optimal feedback has inherent robustness properties (inherent in the sense that it is obtained by ignoring uncertainty and disturbances). The potential difficulty with the HJB approach is that no efficient algorithm is available to solve the PDE when it is nonlinear and the problem dimension is high, making it impossible to derive exact expressions for optimal controls for most nontrivial problems of interest. The optimal can only be computed in special cases, such as linear dynamics and quadratic cost, or very low-dimensional systems. In particular, if the plant is linear time invariant (LTI) and the (infinite-time) performance index is quadratic, then the corresponding HJB equation for this infamous linear-quadratic regulator (LQR) problem reduces to an algebraic Riccati equation (ARE). Contrary to the well-developed and widely applied theory and computational tools for theRiccati equation (for example, see [1]), theHJB equation is difficult, if not impossible, to solve for most practical applications. The exact solution for the optimal control policies is very complex

The pole assignment problem for distributed parameter systems governed by analytic semigroups is discussed. Introducing unbounded feedback operators, we can solve the problem under a weaker restriction on the separation of the open-loop and the closed-loop poles. These types of feedback systems inherit the spectrum determined growth assumption of the open-loop systems. We first present a characteristic equation whose roots are the poles of the closed-loop system. Next we give a feedback element which realizes the desired pole assignment of the closed-loop system. We show an interesting result that an infinite number of the open-loop poles can be uniformly moved by means of a suitable linear feedback. We apply the results -to a heat system and a flexible structure for the cases of distributed controls and boundary controls.

A new result of the problem of pole assignment by output feedback is presented. A simple condition is given under which min (n, m + r − 1) closed-loop poles can be assigned arbitrarily by a gain output feedback, where n, mand r are the dimensions of the state, output and input vectors, respectively. Under the same condition, a new lower bound for the number of poles assignable by a fixed-order dynamic compensator is given which improves considerably on the results of Breach and Pearson (1970) and Ahmari and Vacroux (1973). A design algorithm and illustrative examples are also given.

This paper presents a self-tuning controller that minimises a cost function incorporating system input, output and set-point variations, and adapts it in such a way that the closed-loop poles are located on prespecified locations. The link that exists between the classical control strategy of pole assignment and the suboptimal strategy of the self-tuning controller is demonstrated and made use of to produce a generalised controller which possesses the major advantages of both. The controller combines the robustness of pole assignment with the ease of reference following provided by suboptimal self tuning.

The input-output decoupling problem of generalized state-space systems (GSS) via proportional state feedback is studied. The following three major issues are resolved: the necessary and sufficient conditions for the problem to have a solution, the general analytical expressions of the controller matrices, and the general analytical expressions of the diagonal elements of the closed-loop system. It is concluded that the approach presented seems to be a quite powerful tool for solving several feedback design problems for GSS systems. Using this method, problems such as decoupling and simultaneous pole assignment and decoupling via output feedback are successfully studied. >

Motivated by regulating/eliminating the population of herbivorous pests, we investigate a discrete-time plant-herbivore model with two different constant control strategies (removal versus reduction), and formulate the corresponding optimal control problems when its dynamics exhibits varied types of bi-stability and fluctuating environments. We provide basic analysis and identify the critical factors to characterize the optimal controls and the corresponding plant-herbivore dynamics such as the control upper bound (the effectiveness level of the implementation of control measures) and the initial conditions of the plant and herbivore. Our results show that optimal control could be easier when the model has simple dynamics such as stable equilibrium dynamics under constant environment or the model exhibits chaotic dynamics under fluctuating environments. Due to bistability, initial conditions are important for optimal controls. Regardless of with or without fluctuating environments, initial conditions taken from the near the boundary makes optimal control easier. In general, the pest is hard to be eliminated when the control upper bound is not large enough. However, as the control upper bound is increased or the initial conditions are chosen from near the boundary of the basin of attractions, the pest can be manageable regardless of the fluctuating environments.

MPC formulations with linear dynamics and quadratic objectives can be solved efficiently by using a primal-dual interior-point framework, with complexity proportional to the length of the horizon. An alternative, which is more able to exploit the similarity of the problems that are solved at each decision point of linear MPC, is to use an active-set approach, in which the MPC problem is viewed as a convex quadratic program that is parametrized by the initial state \(x_{0}\). Another alternative is to identify explicitly polyhedral regions of the space occupied by \(x_{0}\) within which the set of active constraints remains constant, and to pre-calculate solution operators on each of these regions. All these approaches are discussed here.