A computer implementable algorithm for the synthesis of an optimal controller for acyclic discrete event processes

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

In certain discrete event applications it may be desirable to find a particular controller, within the set of acceptable controllers, which extremises some quantitative performance measure. In this paper we propose a theory of optimal control to meet such design requirements for deterministic systems. The discrete event system (DES) is modelled by a regular language. Event and cost functions are defined which induce costs on controlled system behaviour. The event costs associated with the system behaviour can only be reduced, in general, by increasing the control costs. Thus it is nontrivial to find the optimal amount of control to use and consequently the formulation captures the fundamental tradeoff motivating classical optimal control. Results on the existence and computability of minimally restrictive optimal controllers are presented. Such controllers are also polynomially computable under certain assumptions. It is shown that this framework subsumes the prior graph-theoretic formulation. >

For over twenty years, hybrid systems, involving both a discrete-valued state component and a continuous-valued state component (evolving in either discrete or continuous time), have challenged control engineers interested in expanding the applications of control theory. In particular, the hybrid system representation has motivated the formulation of "fault-tolerant" optimal control problems for systems subject to abrupt changes in their structure, parameters or operating environment. One approach has been the development of optimal controllers for jump linear systems. Strong theoretical results are currently limited, however, to Markov and semi-Markov jump processes, when change in the system structure are perfectly observed. During the same two decades, theory and methods for the analysis of discrete event systems have been developed. Recently the control of discrete event systems has received significant attention. This paper is directed toward the application of jump systems control theory to the control of continuous variable processes driven by discrete event systems.

In this paper, we present an optimal control approach using Linear Matrix Inequalities (LMIs) for trajectory tracking control of a three-wheeled omnidirectional mobile robot in the presence of external disturbances on the robot's actuators and noise in the robot's sensor measurements. First, a state-space representation of the omnidirectional robot dynamics is derived using a point-mass dynamic model. Then, we propose an LMI-based full-state feedback H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller for the tracking problem. The robot's tracking performance with the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller is compared to its performance with a classical full-state feedback tracking controller in simulations with circular and bowtie-shaped reference trajectories. In order to evaluate our proposed controller in practice, we also implement the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal and classical controllers for these reference trajectories on a three-wheeled omnidirectional robot. The H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller guarantees stabilization of the robot motion and attenuates the effects of frictional disturbances and measurement noise on the robot's tracking performance. Using the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller, the robot is able to track the reference trajectories with up to a 47.8% and 45.8% decrease in the maximum pose and twist errors, respectively, over a full cycle of the trajectory compared to the classical controller. The simulation and experimental results show that our LMI-based H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> -optimal controller is robust to undesired effects of disturbances and noise on the dynamic behavior of the robot during trajectory tracking and can outperform the classical controller in attenuating their effects.

The centralized paradigm of a single controller and a single plant upon which modern control theory is built is no longer applicable to modern cyber-physical systems of interest, such as the power-grid, software defined networks or automated highways systems, as these are all large-scale and spatially distributed. Both the scale and the distributed nature of these systems has motivated the decentralization of control schemes into local sub-controllers that measure, exchange and act on locally available subsets of the globally available system information. This decentralization of control logic leads to different decision makers acting on asymmetric information sets, introduces the need for coordination between them, and perhaps not surprisingly makes the resulting optimal control problem much harder to solve. In fact, shortly after such questions were posed, it was realized that seemingly simple decentralized optimal control problems are computationally intractable to solve, with the Wistenhausen counterexample being a famous instance of this phenomenon. Spurred on by this perhaps discouraging result, a concerted 40 year effort to identify tractable classes of distributed optimal control problems culminated in the notion of quadratic invariance, which loosely states that if sub-controllers can exchange information with each other at least as quickly as the effect of their control actions propagates through the plant, then the resulting distributed optimal control problem admits a convex formulation. The identification of quadratic invariance as an appropriate means of convexifying distributed optimal control problems led to a renewed enthusiasm in the controller synthesis community, resulting in a rich set of results over the past decade. The contributions of this thesis can be seen as being a part of this broader family of results, with a particular focus on closing the gap between theory and practice by relaxing or removing assumptions made in the traditional distributed optimal control framework. Our contributions are to the foundational theory of distributed optimal control, and fall under three broad categories, namely controller synthesis, architecture design and system identification. We begin by providing two novel controller synthesis algorithms. The first is a solution to the distributed H∞ optimal control problem subject to delay constraints, and provides the only known exact characterization of delay-constrained distributed controllers satisfying an H∞ norm bound. The second is an explicit dynamic programming solution to a two player LQR state-feedback problem with varying delays. Accommodating varying delays represents an important first step in combining distributed optimal control theory with the area of Networked Control Systems that considers lossy channels in the feedback loop. Our next set of results are concerned with controller architecture design. When designing controllers for large-scale systems, the architectural aspects of the controller such as the placement of actuators, sensors, and the communication links between them can no longer be taken as given -- indeed the task of designing this architecture is now as important as the design of the control laws themselves. To address this task, we formulate the Regularization for Design (RFD) framework, which is a unifying computationally tractable approach, based on the model matching framework and atomic norm regularization, for the simultaneous co-design of a structured optimal controller and the architecture needed to implement it. Our final result is a contribution to distributed system identification. Traditional system identification techniques such as subspace identification are not computationally scalable, and destroy rather than leverage any a priori information about the system's interconnection structure. We argue that in the context of system identification, an essential building block of any scalable algorithm is the ability to estimate local dynamics within a large interconnected system. To that end we propose a promising heuristic for identifying the dynamics of a subsystem that is still connected to a large system. We exploit the fact that the transfer function of the local dynamics is low-order, but full-rank, while the transfer function of the global dynamics is high-order, but low-rank, to formulate this separation task as a nuclear norm minimization problem. Finally, we conclude with a brief discussion of future research directions, with a particular emphasis on how to incorporate the results of this thesis, and those of optimal control theory in general, into a broader theory of dynamics, control and optimization in layered architectures.

Infinite-time optimal controllers have been designed for a dispersion-type tubular reactor model within the framework of adaptive-critic-based neuro-controller design. For the reactor control problem, which is governed by two coupled nonlinear partial differential equations, an optimal controller synthesis scheme is presented using two sets of neural networks. One set of neural networks captures the relationship between the states and the control, whereas the other set of networks captures the relationship between the states and the co-states. This innovative approach solves the optimal controller in a feedback form. This methodology can be viewed as a practical computational tool in designing optimal controllers for distributed parameter systems in general.

In order to make it possible to effectively represent deterministic uncertainties and vagueness as well as the human subjective observation and judgement inherent to many real-world problems, especially those in biomedicine, we introduce, in this paper, fuzzy states and fuzzy events and generalize (crisp) discrete event systems (DES) to fuzzy DES. The largely graph-based current framework of the crisp DES is unsuitable for the expansion, and we have thus reformulated it using state vectors and event transition matrices which can be extended to fuzzy vectors and matrices by allowing their elements to take values between 0 and 1. To measure information related to fuzzy DES, we generalize the crisp DES observability. The new observability allows one to determine whether or not the system output observed is sufficient for decision making. Finally, we extend the optimal control of DES to fuzzy DES. The new fuzzy DES theory is consistent with the existing theory, both at the conceptual and the computation levels, in that the former contains the latter as a special case when the memberships must be either 0 or 1. Numerical examples are provided to illustrate the theoretical development.

In this paper we study suitability of using assignment based programming language for supervisory control of discrete event systems. We first show that the supervisory control problem in its general setting is undecidable. Next we show that the supervisory control problem for discrete event systems modeled by assignment based programs is decidable and develop techniques which reduce the supervisory control problem of such systems to that of solving arithmetic equations. We first apply these techniques for synthesis of static controllers.

Supervisory control of discrete event systems under total and partial observation

In this study, the problem of finding an optimal controller for nonlinear systems with one input and a reference tracking signal is approached. With the problem's formulation, any desired signal can be tracked instantly with a closed-loop controller without the need for integral terms. Presentation lies at the heart of optimal control. This study, however, does not consider the integral term, allowing tracking and stability to occur naturally. It has a broad scope with a wide range of applications, namely when dealing with affine nonlinear systems, which provide geometric control unification with asymptotic stability in some cases. A common scenario that comes from optimal control involves the minimization of integral cost functionals. Issues like asymptotic stability or even tracking to the desired reference signal have always been the main limitations. In this study, the main theorem allows the solution of optimal control problems with no-integral terms, in other words tracking problems with input/state constraints, providing closed-loop controllers. A DC motor with a pendulum in upright position is an example of an application for which singular optimal control is tested in this study. The results confirm both asymptotic stability and optimal tracking with an accuracy of 95%. The main contributions of this study include an optimal closed-loop controller with no mixed initial/final conditions, input/state constraints, asymptotic stability guarantee, a strong connection with geometric tools and finally the possibility to generalize to systems with multiple inputs. As a conclusion, general nonlinear control systems can be included in the optimal control methodology presented in this study including input/state constraints. Due to the lack of integral terms, the problem can be solved in closed form by using an optimal closed-loop controller.

To find effective control of the duplex process of induction melting in conditions of uncertainties, a method was suggested that made it possible to obtain optimal controls for both stages: melting and thermal treatment. It is shown how the search for an optimal melting control strategy can be performed using the theory of statistical games. At the same time, it is selected which of the melting regimes will be the best with the existing provision of the shop with charge materials. The costs of melting are the total costs, consisting of: – costs for the selected technological mode of melting (including costs for materials), – costs from the potential rejection of castings due to the non-conformity of the chemical composition of the alloy to the specified – due to the incorrectly chosen melting mode, – costs from the downtime due to the fact that the necessary amount of metal from the furnace is not delivered to the conveyor. The choice of an optimal control strategy in accordance with the proposed procedure can remove uncertainty in the evaluation of input process variables if they are taken as indicators of the charge quality. To find the optimal control at the stage of the heat treatment, a multialternative description of the final state is proposed on the basis of solving the problem of ridge analysis. This makes it possible to remove the uncertainty in the estimation of the final state, which allows a lot of optimal solutions in the sense of achieving a given quality. It is shown that such approach makes it possible not only to synthesize the optimal controller of the temperature regime on the basis of an analysis of the system of differential equations describing the control object, but also the application of the Pontryagin maximum principle to search for optimal control of the thermal treatment. The proposed method allows to determine the optimal control in the sense of stabilization for a given process parameter of the duplex process of induction melting. The resulting solutions form the necessary logical conditions for the logic control unit for the control system of the duplex induction melting process.

A substantial number of tasks in production systems are executed in a repetitive, cyclic fashion. Specifically, production systems run different production cycles of different products as well as different instances of the same production cycle. In this paper, we consider the optimal control and interleaving of such production cycles in a supervisory control framework for discrete event systems (DESs). That is, different from other approaches, our work is based on a behavioral specification of each production cycle. First, we adapt an optimal control approach for DESs, in order to optimize the operation of individual production cycles. Second, we employ the interleaving composition to design a supervisor that enables the simultaneous execution of different production cycles. Combining both results, we can further determine the maximum number of production cycles that can be executed simultaneously on a given production system.

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.

Optimal yaw moment control law for improved vehicle handling

Discrete event models have become an important tool in theoretical biology. There is, at the same time, a growing awareness of the methodological issues that arise in connection with discrete event systems. Synchronization between discrete event and continuous model components and the handling of simulatenous events are technical issues that many simulation programs must address, and how they are handled can have a substantial impact on the results of a calculation. More fundamental, but not unrelated, is the question of comparing notionally equivalent models. This talk presents particular problems that have been encountered when modeling cell migration with a mix of discrete event and continuous processes. Specific examples are taken from our work on smooth muscle cell migration and similar models that are described in the extent literature.