On the Relation Between the Minimum Principle and Dynamic Programming for Classical and Hybrid Control Systems

On the Minimum Principle and Dynamic Programming for Hybrid Systems

A Class of Linear Quadratic Gaussian Hybrid Optimal Control Problems with Realization–Independent Riccati Equations

This paper provides a theory and associated algorithm for the optimization of autonomous and controlled hybrid systems on Lie groups. First, a geometrical derivation of the Hybrid Minimum Principle (HMP) for hybrid systems whose state manifolds constitute a Lie group (G, *) which is left invariant under the controlled dynamics of the system is presented. Second, a geometrical algorithm is developed by employing the notion of exponential curves on Lie groups. The convergence analysis for the proposed algorithm is based on Lasalle Theory. Simulation results are provided at the end of the paper.

We consider a fairly general class of state-constrained nonlinear hybrid optimal control problems that are based on coordinatizing Sussmann's model. An event set generalizes the notion of a guard set, reset map, endpoint set as well as the switching set. We present a pseudospectral (PS) knotting method that discretizes the continuous-time variables of the problem. The discrete event conditions are imposed over the PS knots leading to a large, sparse, mixed-variable programming (MVP) problem. The Karush-Kuhn-Tucker conditions for the MVP are transformed in a manner that makes them closely resemble the discretized necessary conditions obtained from the hybrid minimum principle. A set of closure conditions are introduced to facilitate commuting the operations of dualization and discretization. An immediate consequence of this is a hybrid covector mapping theorem that provides an order-preserving transformation of the Lagrange multipliers associated with the discretized problem to the discretized covectors associated with the hybrid optimal control problem.

The 48-V diesel mild-hybrid electric vehicle equipped with an electrically heated catalyst is a cost-effective solution to reduce tailpipe emissions, while its supervisory control design is more challenging, due to cross-coupled energy and thermal subsystems. This article presents a distributed control design for the energy and thermal management system, aiming to optimize fuel economy and reduce catalyst warm-up time, by cooperation between electrical heating and engine load upshift. A control-oriented model of HEV and catalyst is developed, which captures the dynamics of the battery state of charge and catalyst temperature. The energy and thermal management problem is cast as a hybrid optimal control problem, to minimize entire trip fuel consumption under different switching (warm-up) times. An off-line solution method is designed to exploit the tradeoff between fuel and warm-up time based upon hybrid minimum principle and a nested shooting algorithm. For online implementation, a distributed supervisory control is proposed, which covers both cold-start and normal operations, and is compatible with the equivalent consumption minimization strategy. Both off-line and online methods are validated in simulation, using two driving cycles and five warm-up speeds.

Many interesting problems in numerical optimization are hybrid optimal control problems. Hybrid optimal control (HOC) problems include both continuous-valued variables and categorical variables in the problem formulation. For the types of problems envisioned here the categorical variables will specify the structure or sequence of events that qualitatively describes the trajectory or mission. Continuous variables are used in the modeling of the continuous dynamics. In this work, the mission planning problem is of interest. The problem is qualitatively different from the typical interception of multiple targets as the discrete variables now represent events like impulses, coast and thrust arcs that change the structure of the problem. In addition, the number of events in the categorical sequence is not fixed. For the dynamical assembly of events required for the solution of the mission planning problem, a scheme that defines events as modules consisting of parameters and constraints is presented. The method assembles the respective events one next to the other in time according to the given mission structure. For the generation of the initial guess, two new methods were developed that approximate optimal low-thrust trajectories. The first method, based on genetic algorithms (GA), handles the rendezvous constraints explicitly using a conditional penalty function. The second method, feasible region analysis (FRA), is based on GA and nonlinear programming (NLP), which allows taking advantage of the GA capabilities in finding a global optimum and NLP ability in handling constraints. A rendezvous problem with known solution is solved.

In this paper, by applying interval methods, a new framework for analysis and control of discrete-time hybrid systems with parameter uncertainty is proposed. In analysis and control of hybrid systems, there are problem formulations such that convex polyhedra are computed, but for high-dimensional systems, it is difficult to solve these problems within the practical computation time. In this paper, instead of computing convex polyhedra, an interval method, which is one of classical methods in verified numerical computation, is applied to analysis and control of hybrid systems. By applying an interval method, discrete-time piecewise systems with parameter uncertainty can be approximately transformed into a mixed logical dynamical model.

We deal with the optimal control problem for piecewise linear and hybrid systems by using a computational approach based on performance-driven reachability analysis. The idea consists of coupling a reach-set exploration algorithm, essentially based on a repetitive use of linear programming, to a quadratic programming solver which selectively drives the exploration. In particular, an upper bound on the optimal cost is continually updated during the procedure, and used as a criterion to discern non-optimal evolutions and to prevent their exploration. The result is an efficient strategy of branch-and-bound nature, which is especially attractive for solving long-horizon hybrid optimal control and scheduling problems.

This paper deals with optimization techniques for linear impulsive hybrid systems (LIHSs). We study the LQ (linear quadratic) impulsive hybrid optimal control problem (OCP) and apply the corresponding Pontryagin-type maximum principle (MP) (see). Our aim is to investigate the natural relationship between the above MP and the Bellman dynamic programming (DP) approach to the above hybrid OCP under consideration. We derive the associated Riccati-type formalism and discuss some related numerical schemes.

This chapter is devoted upon optimality, a topic in which the central result is the Pontryagin’s minimum principle. This important result is briefly approached from the classical calculus of variation. We show that the classical calculus of variation has some major limitations to modern control problems and motivate Pontryagin’s minimum principle. The Hamilton–Jacobi–Bellman method is discussed as an alternative approach to gain first-order necessary conditions for optimality. It is shown that both approaches correspond to each other under restrictive assumptions. The original Pontryagin’s minimum principle for continuous optimal control problems is not suitable for hybrid optimal control problems. However, a quite natural reformulation of the hybrid optimal control problem admits the classical theory for deduction of first-order necessary conditions in the sense of Pontryagin. The charm of this methodology is its comprehensible derivation.

A EROSPACE engineering applications greatly stimulated the development of optimal control theory during the 1950s and 1960s, where the objective was to drive the system states in such a way that some defined cost was minimized. This turned out to have very useful applications in the design of regulators (where some steady state is to be maintained) and in tracking control strategies (where some predetermined state trajectory is to be followed). Among such applications was the problem of optimal flight trajectories for aircraft and space vehicles. Linear optimal control theory in particular has been very well documented and widely applied, where the plant that is controlled is assumed linear and the feedback controller is constrained to be linear with respect to its input. However, the availability of powerful low-cost microprocessors has spurred great advantages in the theory and applications of nonlinear control. The competitive era of rapid technological change, particularly in aerospace exploration, now demands stringent accuracy and cost requirements in nonlinear control systems. This has motivated the rapid development of nonlinear control theory for application to challenging, complex, dynamical real-world problems, particularly those that bear major practical significance in aerospace, marine, and defense industries. Infinite-time horizon nonlinear optimal control (ITHNOC) presents a viable option for synthesizing stabilizing controllers for nonlinear systems by making a state-input tradeoff, where the objective is to minimize the cost given by a performance index. The original theory of nonlinear optimal control dates from the 1960s. Various theoretical and practical aspects of the problem have been addressed in the literature over the decades since. In particular, the continuous-time nonlinear deterministic optimal control problem associated with autonomous (time-invariant) nonlinear regulator systems that are affine (linear) in the controls has been studied by many authors. The long-established theory of optimal control offers quite mature and well-documented techniques for solving this control-affine nonlinear optimization problem, based on dynamic programming or calculus of variations, but their application is generally a very tedious task. Bellman’s dynamic programming approach reduces to solving a nonlinear first-order partial differential equation (PDE), expressed by the Hamilton–Jacobi–Bellman (HJB) equation. The solution to the HJB equation gives the optimal performance/cost value (or storage) function and determines an optimal control in feedback form under some smoothness assumptions. Alternatively, in the classical calculus of variations, optimal control problems can be characterized locally in terms of the Hamiltonian dynamics arising from Pontryagin’s minimum principle. These are the characteristic equations of the HJB PDE, which result in a nonlinear, constrained two-point boundary value problem (TPBVP) that, in general, can only be solved by successive approximation of the optimal control input using iterative numerical techniques for each set of initial conditions. Numerically, even though the nonlinear TPBVP is somewhat easier to solve than the HJBPDE, control signals can only be determined offline and are thus best suited for feedforward control of plants for which the state trajectories are known a priori. Therefore, contrary to the dynamic programming approach, the resultant control law is not generally in feedback form. Open-loop control, however, is sensitive to random disturbances and requires that the initial state be on the optimal trajectory. In contrast, nonlinear optimal feedback has inherent robustness properties (inherent in the sense that it is obtained by ignoring uncertainty and disturbances). The potential difficulty with the HJB approach is that no efficient algorithm is available to solve the PDE when it is nonlinear and the problem dimension is high, making it impossible to derive exact expressions for optimal controls for most nontrivial problems of interest. The optimal can only be computed in special cases, such as linear dynamics and quadratic cost, or very low-dimensional systems. In particular, if the plant is linear time invariant (LTI) and the (infinite-time) performance index is quadratic, then the corresponding HJB equation for this infamous linear-quadratic regulator (LQR) problem reduces to an algebraic Riccati equation (ARE). Contrary to the well-developed and widely applied theory and computational tools for theRiccati equation (for example, see [1]), theHJB equation is difficult, if not impossible, to solve for most practical applications. The exact solution for the optimal control policies is very complex

Nurse Rostering is the activity of assigning nurses to daily shifts in order to satisfy the cover requirements, taking into account the operational requirements and nurses' preferences. The problem is usually modeled as sets of hard and soft constraints with an objective function to minimize violations of soft constraints. The nurse rostering problem is known to be NP-hard. Many metaheuristics were used to tackle this problem. One of the frequently used heuristics is the Variable Neighbourhood Search (VNS). The VNS is usually used as a standalone method or in integration with another exact or heuristic method. In this paper, a new hybrid VNS and Dynamic Programming based heuristic approach is proposed to handle the nurse rostering problem. In the proposed approach, two perturbation mechanisms are adopted simultaneously. The proposed approach is tested on two different benchmark data sets. A comparison with state-of-the-art methods from literature revealed the competitive performance of the proposed approach.

Control Optimization of Discontinuous Hybrid Systems Using Embedding

Chapter 1 - Introduction and Motivation

INTRODUCTION We present a mathematical formulation of fixed-time health conscious charging in Li-Ion batteries. We then solve the resulting problem optimally using thePontryagin Minimum Principle (PMP).Typically Lithium-Ion batteries are charged using a constant-current-constant-voltage (CCCV) strategy that relies on voltage and current limitations set forth by battery manufacturers. More sophisticated strategies that focus on minimizing battery degradation have been proposed in the literature (1, 2). These methods rely on controlling harmful side reactions in a cell by using a descriptive physics-based battery model and optimal control theory.The dynamic limitations obtained from models-based battery management schemes allow more efficient utilization of cells through providing more information on the state of the system. This additional information makes it possible to push the cells to their operational extremes without causing them much damage. Although the results obtained from these sophisticated models are very reliable, the enormous computational power required prevents their online implementation. In this work, we use a simplified cell model suitable for obtaining analytical optimization results. PROBLEM FORMULATION We present the following problem: Neglecting the diffusion effects occurring inside a cell, the dynamics of the state of charge (SOC) are given byEqn [1] where xis SOC. Assuming small over-potentials, the Butler-Volmer kinetics of the cell is linearized around the cell reference potential, which leads to Eq. 2.Eqn [2]The reference potential can be approximated as an affine function of the SOC in certain operating regimes,Eqn [3]Harmful side reactions such as SEI layer growth are modeled using the Tafel equation, which is of the formEqn [4] Iside is assumed to be much smaller than Ibattery . The health-conscious charging problem can then be stated as the following dynamic optimization problem Eqn [5]which is subject to the dynamic constraint in Eq. 1. As shown in Eq. 5, we also penalize the deviation of x(T) from xf , the desired final SOC and β represents the relative cost of this deviation. We solve this problem using the Pontryagin Minimum Principle (PMP). We define the Hamiltonian asEqn [6]where λ is a time varying Lagrange multiplier associated with the constraint in Eq. 1. Applying the PMP to this problem gives the following optimality conditions:Eqn [7]We obtain a closed-form solution for the optimal charging current trajectory by solving Eq. 7 and 1 together. Using this solution, the optimal trajectory can be studied under various conditions, as shown in Figs. 1 and 2. These trajectories represent a significant departure from CCCV cycling in terms of both its shape and the resulting degradation. This provides insight into the importance of health-conscious battery management and the ways in which cycle shapes affects cycle life. Figure 1. Optimal SOC trajectories as a function of the final time. Figure 2. Optimal SOC trajectories as a function of . REFERENCES 1. K. A. Smith, C. D. Rahn, and C.Y. Wang, IEEE Transactions on Control Systems Technology, 654, (2010).2. R. Klein, N. A. Chaturvedi, J. Christensen, J. Ahmed, R. Findeisen, and A. Kojic, American Control Conference, 382 (2011).