Computational Optimal Control.

In this paper, an iterative computational method is proposed to handle constrained nonlinear quadratic optimal control problems. with terminal state constraints, saturation state and saturation control variables constraints. The method is based on replacing the original constrained nonlinear optimal control problem by a sequence of constrained linear time-varying quadratic optimal control problems. To this end, an iterative technique is used to replace the original constrained nonlinear dynamic system by a sequence of constrained linear time-varying systems. Then each of constrained linear time-varying quadratic optimal control problems is approximated by a quadratic programming problem by parameterizing each of the state variable by a finite length Legendre polynomials with unknown parameters. To show the effectiveness of the proposed method, simulation results of a constrained nonlinear optimal control problem are presented.

The centralized paradigm of a single controller and a single plant upon which modern control theory is built is no longer applicable to modern cyber-physical systems of interest, such as the power-grid, software defined networks or automated highways systems, as these are all large-scale and spatially distributed. Both the scale and the distributed nature of these systems has motivated the decentralization of control schemes into local sub-controllers that measure, exchange and act on locally available subsets of the globally available system information. This decentralization of control logic leads to different decision makers acting on asymmetric information sets, introduces the need for coordination between them, and perhaps not surprisingly makes the resulting optimal control problem much harder to solve. In fact, shortly after such questions were posed, it was realized that seemingly simple decentralized optimal control problems are computationally intractable to solve, with the Wistenhausen counterexample being a famous instance of this phenomenon. Spurred on by this perhaps discouraging result, a concerted 40 year effort to identify tractable classes of distributed optimal control problems culminated in the notion of quadratic invariance, which loosely states that if sub-controllers can exchange information with each other at least as quickly as the effect of their control actions propagates through the plant, then the resulting distributed optimal control problem admits a convex formulation. The identification of quadratic invariance as an appropriate means of convexifying distributed optimal control problems led to a renewed enthusiasm in the controller synthesis community, resulting in a rich set of results over the past decade. The contributions of this thesis can be seen as being a part of this broader family of results, with a particular focus on closing the gap between theory and practice by relaxing or removing assumptions made in the traditional distributed optimal control framework. Our contributions are to the foundational theory of distributed optimal control, and fall under three broad categories, namely controller synthesis, architecture design and system identification. We begin by providing two novel controller synthesis algorithms. The first is a solution to the distributed H∞ optimal control problem subject to delay constraints, and provides the only known exact characterization of delay-constrained distributed controllers satisfying an H∞ norm bound. The second is an explicit dynamic programming solution to a two player LQR state-feedback problem with varying delays. Accommodating varying delays represents an important first step in combining distributed optimal control theory with the area of Networked Control Systems that considers lossy channels in the feedback loop. Our next set of results are concerned with controller architecture design. When designing controllers for large-scale systems, the architectural aspects of the controller such as the placement of actuators, sensors, and the communication links between them can no longer be taken as given -- indeed the task of designing this architecture is now as important as the design of the control laws themselves. To address this task, we formulate the Regularization for Design (RFD) framework, which is a unifying computationally tractable approach, based on the model matching framework and atomic norm regularization, for the simultaneous co-design of a structured optimal controller and the architecture needed to implement it. Our final result is a contribution to distributed system identification. Traditional system identification techniques such as subspace identification are not computationally scalable, and destroy rather than leverage any a priori information about the system's interconnection structure. We argue that in the context of system identification, an essential building block of any scalable algorithm is the ability to estimate local dynamics within a large interconnected system. To that end we propose a promising heuristic for identifying the dynamics of a subsystem that is still connected to a large system. We exploit the fact that the transfer function of the local dynamics is low-order, but full-rank, while the transfer function of the global dynamics is high-order, but low-rank, to formulate this separation task as a nuclear norm minimization problem. Finally, we conclude with a brief discussion of future research directions, with a particular emphasis on how to incorporate the results of this thesis, and those of optimal control theory in general, into a broader theory of dynamics, control and optimization in layered architectures.

In this paper, the nonlinear optimal control problem is investigated for spacecraft formation flying with collision avoidance. Based on a nonlinear model of formation flying, two optimal tracking control laws are proposed to ensure the formation pattern converges to the predetermined configuration. The first optimal control law which combines Lyapunov optimizing control with a trajectory-following optimization technique is developed to solve the finite-time nonlinear optimal control problem. For the second controller, an extended [Formula: see text]-[Formula: see text] method is applied to design a closed-form feedback control scheme for nonlinear control problem of spacecraft formation flying with non-standard cost functions. By taking into account the flying safety, a repulsive control scheme is incorporated in the optimal controller to ensure collision avoidance and minimize the performance index. Finally, a numerical example is performed to demonstrate the effectiveness of the proposed approaches.

Most direct methods solve finite time horizon optimal control problems with nonlinear programming solver. In this paper, we propose a numerical method for solving nonlinear optimal control problem with state and control inequality constraints. This method used quasilinearization technique and Haar wavelet operational matrix to convert the nonlinear optimal control problem into a quadratic programming problem. The linear inequality constraints for trajectories variables are converted to quadratic programming constraint by using Haar wavelet collocation method. The proposed method has been applied to solve Optimal Control of Multi-Item Inventory Model. The accuracy of the states, controls and cost can be improved by increasing the Haar wavelet resolution.

In this study, the problem of finding an optimal controller for nonlinear systems with one input and a reference tracking signal is approached. With the problem's formulation, any desired signal can be tracked instantly with a closed-loop controller without the need for integral terms. Presentation lies at the heart of optimal control. This study, however, does not consider the integral term, allowing tracking and stability to occur naturally. It has a broad scope with a wide range of applications, namely when dealing with affine nonlinear systems, which provide geometric control unification with asymptotic stability in some cases. A common scenario that comes from optimal control involves the minimization of integral cost functionals. Issues like asymptotic stability or even tracking to the desired reference signal have always been the main limitations. In this study, the main theorem allows the solution of optimal control problems with no-integral terms, in other words tracking problems with input/state constraints, providing closed-loop controllers. A DC motor with a pendulum in upright position is an example of an application for which singular optimal control is tested in this study. The results confirm both asymptotic stability and optimal tracking with an accuracy of 95%. The main contributions of this study include an optimal closed-loop controller with no mixed initial/final conditions, input/state constraints, asymptotic stability guarantee, a strong connection with geometric tools and finally the possibility to generalize to systems with multiple inputs. As a conclusion, general nonlinear control systems can be included in the optimal control methodology presented in this study including input/state constraints. Due to the lack of integral terms, the problem can be solved in closed form by using an optimal closed-loop controller.

For discrete-time linear time-invariant systems with constraints on inputs and outputs, the constrained finite-time optimal controller can be obtained explicitly as a piecewise-affine function of the initial state via multi-parametric programming. By exploiting the properties of the value function, we present two algorithms that efficiently perform the online evaluation of the explicit optimal control law both in terms of storage demands and computational complexity. The algorithms are particularly effective when used for model-predictive control (MPC) where an open-loop constrained finite-time optimal control problem has to be solved at each sampling time.

The pretransmission parallel hybrid electric vehicle (HEV) with a single electric motor requires relatively little changes from existing powertrain configurations. This configuration, however, has a challenging drivability issue during engine-starts because the electric motor must simultaneously provide the demanded propulsion torque and start the engine. Depending on the propulsion power level, such engine-start process may require a trade-off between drivability and quick start. The goal of this study is to find theoretical performance limits and corresponding optimal control strategies that achieve the balance between these two conflicting goals. We first develop a simplified parallel HEV powertrain model to predict the engine, clutch, and vehicle dynamics. Assuming that the clutch torque can be accurately estimated and perfectly cancelled, an optimal engine-start control problem is formulated to minimize engine-start time while supplying the driver demanded torque. This nonlinear constrained optimal control problem is solved both analytically and numerically. For some special cases, the optimization problem can be solved analytically to obtain a closed-form solution. For the numerical method, dynamic programming (DP) is used, and both analytical and numerical solutions show that selecting a proper level of constant clutch pressure is the key to achieve near-optimal drivability performance. Furthermore, the DP control policy is found to be time-invariant, and thus can be implemented in the form of a full state feedback controller.

This paper investigates the robust optimal control problem of a class of continuous-time, partially linear, interconnected systems. In addition to the dynamic uncertainties resulted from the interconnected dynamic system, unknown bounded disturbances are taken into account throughout the learning process, wherein the system’s dynamics and the disturbances are assumed unknown. These challenges lead the collected online data to be imperfect. In this scenario, traditional data-driven control techniques, such as adaptive dynamic programming (ADP) and robust ADP, encounter a challenge in approximating the optimal control policy precisely due to imperfect data and computational errors. In this paper, a novel data-driven robust policy iteration method is proposed to simultaneously solve the robust optimal control problems. Without relying on the knowledge of the system’s dynamics, the external disturbances or the complete state, the implementation of the proposed method only needs to access the input and partial state information. Based on the small-gain theorem, the notions of strong unboundedness observability and input-to-output stability, it is guaranteed that the learned robust optimal control gain is stabilizing and that the solution of the closed-loop system is uniformly ultimately bounded despite the existence of dynamic uncertainties and unknown external disturbances. The simulation results reveal the efficiency and practicality of the proposed data-driven control method. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work is motivated by the use of reinforcement learning to improve the quality of designing adaptive optimal controllers for engineering applications. Adaptive dynamic programming methods, in particular policy iteration (PI), are widely used in solving optimal control problems. However, due to the iterative nature of PI, the approximated optimal control policy may be inaccurate and imprecise. Especially, when using imperfect system’s measurements instead of the modelling information. This can result in causing the learned control policy to deviate from the actual optimal policy. This becomes more challenging in the existence of dynamic uncertainties and unknown external disturbances which corrupt the measurements and result in imperfect data. This work investigates the conditions on the uncertainties such that the proposed novel data-driven PI algorithm is robust to system’s uncertainties, unknown external disturbances and imperfect measurements. The approximated robust optimal control policy performs robustly in the existences of imperfect data and uncertainties, and at the same time is close enough to the optimal control policy.

Optimal semistable control for continuous-time linear systems

This paper is concerned with the optimal output synchronization control problem for a class of heterogeneous complex dynamical networks (CDNs) with partially unknown system dynamics. By defining a new quadratic performance index including the information of state couplings, an optimal controller consisting of a feedback control and a compensated feedforward control is then designed. A necessary and sufficient condition for the optimal control minimizing the quadratic performance index is established. Especially, an algebraic Riccati equation (ARE) associated with a discount factor is obtained, and a novel online iteration algorithm by iteratively solving the ARE is proposed to compute the optimal controller gain for the case that the CDNs are with partially unknown system dynamics. Moreover, it is proved that the output synchronization errors converge to zero asymptotically by applying the optimal control laws. Finally, two simulation examples are provided to verify the effectiveness of the proposed control method.

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.

An optimal control method for distributed parameter systems is developed and applied to active vibration control of a plate. Piezoelectric material is used as an actuator. The structural dynamics are modeled using the space—time assumed mode expansion technique (a Rayleigh—Ritz method applied to both time and space variables). In particular, the generalized coordinates (both mechanical and electrical) and all generalized forces are described in terms of expansion functions. Using Hamilton's law of varying action along with the space—time assumed mode expansion results in algebraic equations of motion. These are then used as the constraint equations in the optimal controller design. Using space—time expansion, the usual variational optimal control problem is transformed into an equivalent algebraic problem. Optimal solutions are then obtained in a closed form and the solution is global within the time period considered. The solution procedure does not lead to a Riccati equation as is the case in the conventional optimal control solutions. The direct optimal control problem of a vibrating plate is illustrated through simulations where a parametric study is undertaken to check the controller performance. This study shows that the developed optimal control method is simple and the structure—control interaction is an inherent property of the method.

This article presents a unified approach to simultaneous solution of optimization and optimal control problems in batch distillation, operating under different modes of operation: variable, constant, or optimal reflux. The simplified, computationally efficient short‐cut method and a novel algorithm to solve the optimal control problems in batch distillation is the basis of this unified approach. The short‐cut method identifies the feasible region of operation essential for optimization and optimal control problems, and provides analytical partial derivatives of the model parameters crucial to the solution.The new algorithm for the solution of optimal combination of the maximum principle and NLP optimization techniques. It circumvents the problems associated with the maximum principle approach (iterative solution of a two‐point boundary value problem, undounded control variables, and inability to handle the simultaneous optimization and optimal control problem), and the coupled ODE discretization‐NLP optimization scheme for nonlinear models (higher system nonlinearities, multiplicity of solutions, sensitivity of convergence to initial guesses). This algorithm reduces the dimensionality of the problem, and the nature of the algorithm allows a common platform to optimal solutions of different operating conditions. This article also shows that different categories of the optimal control problems in batch distillation essentially involve the solution of the maximum distillate problem.

This paper builds up two equivalence theorems for different kinds of optimal control problems of internally controlled Schrodinger equations. The first one is an equivalence of the minimal norm and the minimal time control problems where the target sets are the origin of the state space. (The minimal time control problems are also called the first type of optimal time control problems.) The second one is an equivalence of three optimal control problems: optimal target control problems, optimal norm control problems, and the second type of optimal time control problems.

In this paper, we present an optimal control approach using Linear Matrix Inequalities (LMIs) for trajectory tracking control of a three-wheeled omnidirectional mobile robot in the presence of external disturbances on the robot's actuators and noise in the robot's sensor measurements. First, a state-space representation of the omnidirectional robot dynamics is derived using a point-mass dynamic model. Then, we propose an LMI-based full-state feedback H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller for the tracking problem. The robot's tracking performance with the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller is compared to its performance with a classical full-state feedback tracking controller in simulations with circular and bowtie-shaped reference trajectories. In order to evaluate our proposed controller in practice, we also implement the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal and classical controllers for these reference trajectories on a three-wheeled omnidirectional robot. The H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller guarantees stabilization of the robot motion and attenuates the effects of frictional disturbances and measurement noise on the robot's tracking performance. Using the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller, the robot is able to track the reference trajectories with up to a 47.8% and 45.8% decrease in the maximum pose and twist errors, respectively, over a full cycle of the trajectory compared to the classical controller. The simulation and experimental results show that our LMI-based H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller is robust to undesired effects of disturbances and noise on the dynamic behavior of the robot during trajectory tracking and can outperform the classical controller in attenuating their effects.