Single Network Adaptive Critic aided nonlinear dynamic inversion for suboptimal command tracking

This paper presents an advanced single network adaptive critic (SNAC) aided nonlinear dynamic inversion (NDI) approach for simultaneous attitude control and trajectory tracking of a micro-quadrotor. Control of micro-quadrotors is a challenging problem due to its small size, strong coupling in pitch-yaw-roll and aerodynamic effects that often need to be ignored in the control design process to avoid mathematical complexities. In the proposed SNAC aided NDI approach, the gains of the dynamic inversion design are selected in such a way that the resulting controller behaves closely to a pre-synthesized SNAC controller for the output regulation problem. However, since SNAC is based on optimal control theory, it makes the dynamic inversion controller to operate near optimal and enhances its robustness property as well. More important, it retains two major benefits of dynamic inversion: (i) closed form expression of the controller and (ii) easy scalability to command tracking application even without any apriori knowledge of the reference command. Effectiveness of the proposed controller is demonstrated from six degree-of-freedom simulation studies of a micro-quadrotor. It has also been observed that the proposed SNAC aided NDI approach is more robust to modeling inaccuracies, as compared to the NDI controller designed independently from time domain specifications.

We consider the problem of aircraft motion control under uncertainties caused by incomplete and inaccurate knowledge of the aircraft characteristics, as well as by abnormal situations in flight that affect the properties of the aircraft as a control object. One of the effective tools for solving problems of this kind, providing the adjustment of aircraft control algorithms taking into account its changed dynamics, is reinforcement learning in the variant of Approximate Dynamic Programming (ADP), in combination with artificial neural networks. In the last decade, a family of methods known as Adaptive Critic Design (ACD) has been actively developed within the ADP approach to control the behavior of complex dynamical systems. The paper discusses the application of one variant of the ACD approach, namely SNAC (Single Network Adaptive Critic) and its development through combined use with the dynamic inversion method. This approach makes it possible to form an optimal adaptive control law for aircraft motion. Its effectiveness is demonstrated on the example of longitudinal motion control for supersonic transport airplane (SST).

This paper presents a Single Network Adaptive Critic (SNAC) aided Nonlinear Dynamic Inversion (NDI) optimization controller with a Proportional-Derivative (PD) based Optimal Control Modification (OCM) design for fast pitch rate adaptation in an MRAC flight control system. The advantages of adding SNAC to the NDI MRAC controller with OCM adaptation are presented. It is seen from simulation results that the SNAC aided NDI controller with the OCM adaptation has fewer high frequency oscillations and increased robustness to the different network architectures of the OCM. Three variations of OCM uncertainty estimation basis functions, namely the OCM – Linear (OCM-L), OCM – Bias (OCM-B) and OCM Linear and Bias (OCM-LB) are included in this paper. The responses of the SNAC aided NDI and traditional NDI controllers, using the same OCM adaptation technique are compared. Simulation results run on a nonlinear model of a small single piston engine are included for demonstration.

To synthesize fixed-final-time control-constrained optimal controllers for discrete-time nonlinear control-affine systems, a single neural network (NN)-based controller called the Finite-horizon Single Network Adaptive Critic is developed in this paper. Inputs to the NN are the current system states and the time-to-go, and the network outputs are the costates that are used to compute optimal feedback control. Control constraints are handled through a nonquadratic cost function. Convergence proofs of: 1) the reinforcement learning-based training method to the optimal solution; 2) the training error; and 3) the network weights are provided. The resulting controller is shown to solve the associated time-varying Hamilton-Jacobi-Bellman equation and provide the fixed-final-time optimal solution. Performance of the new synthesis technique is demonstrated through different examples including an attitude control problem wherein a rigid spacecraft performs a finite-time attitude maneuver subject to control bounds. The new formulation has great potential for implementation since it consists of only one NN with single set of weights and it provides comprehensive feedback solutions online, though it is trained offline.

Many public funding agencies adopt the one-stage call-based mode to select R&D projects. The funding decision-making in this mode is characterized by two primary features: 1) it is static, meaning that decision-makers solely concentrate on projects submitted within the current call, and 2) it is decentralized, indicating that funding decisions are made independently across diverse sectors/disciplines. As a result, the administrative burden on funding agencies is reduced. However, due to the budget constraint, the funding agencies may fail to fund better projects that emerge in future calls or appear in other sectors/disciplines. Therefore, we investigate whether a better project portfolio can be achieved by selecting public R&D projects in a dynamic and centralized manner. We formulate a Markov decision process (MDP) model for the dynamic centralized public R&D project portfolio selection problem. We develop an approximate dynamic programming (ADP) approach that combines learning-based Monte Carlo simulation and a two-stage rollout algorithm to efficiently solve the MDP model. Based on extensive computational experiments, we compare our ADP approach with a threshold heuristic that is representative of the static and decentralized funding mode, as well as two baseline algorithms. The results indicate that our ADP approach is effective, and it is beneficial to adopt the dynamic centralized decision-making mode in public R&D project portfolio selection. We also investigate our ADP approach utilizing a case study based on real-world data from the National Natural Science Foundation of China. Our ADP approach can be integrated into the decision support systems of funding agencies, enabling automated, dynamic, and centralized selection of public R&D projects.

Approximate dynamic programming (ADP) formulation implemented with an adaptive critic (AC)-based neural network (NN) structure has evolved as a powerful technique for solving the Hamilton-Jacobi-Bellman (HJB) equations. As interest in ADP and the AC solutions are escalating with time, there is a dire need to consider possible enabling factors for their implementations. A typical AC structure consists of two interacting NNs, which is computationally expensive. In this paper, a new architecture, called the ‘cost-function-based single network adaptive critic (J-SNAC)’ is presented, which eliminates one of the networks in a typical AC structure. This approach is applicable to a wide class of nonlinear systems in engineering. In order to demonstrate the benefits and the control synthesis with the J-SNAC, two problems have been solved with the AC and the J-SNAC approaches. Results are presented, which show savings of about 50% of the computational costs by J-SNAC while having the same accuracy levels of the dual network structure in solving for optimal control. Furthermore, convergence of the J-SNAC iterations, which reduces to a least-squares problem, is discussed; for linear systems, the iterative process is shown to reduce to solving the familiar algebraic Ricatti equation.

Most direct methods solve finite time horizon optimal control problems with nonlinear programming solver. In this paper, we propose a numerical method for solving nonlinear optimal control problem with state and control inequality constraints. This method used quasilinearization technique and Haar wavelet operational matrix to convert the nonlinear optimal control problem into a quadratic programming problem. The linear inequality constraints for trajectories variables are converted to quadratic programming constraint by using Haar wavelet collocation method. The proposed method has been applied to solve Optimal Control of Multi-Item Inventory Model. The accuracy of the states, controls and cost can be improved by increasing the Haar wavelet resolution.

A EROSPACE engineering applications greatly stimulated the development of optimal control theory during the 1950s and 1960s, where the objective was to drive the system states in such a way that some defined cost was minimized. This turned out to have very useful applications in the design of regulators (where some steady state is to be maintained) and in tracking control strategies (where some predetermined state trajectory is to be followed). Among such applications was the problem of optimal flight trajectories for aircraft and space vehicles. Linear optimal control theory in particular has been very well documented and widely applied, where the plant that is controlled is assumed linear and the feedback controller is constrained to be linear with respect to its input. However, the availability of powerful low-cost microprocessors has spurred great advantages in the theory and applications of nonlinear control. The competitive era of rapid technological change, particularly in aerospace exploration, now demands stringent accuracy and cost requirements in nonlinear control systems. This has motivated the rapid development of nonlinear control theory for application to challenging, complex, dynamical real-world problems, particularly those that bear major practical significance in aerospace, marine, and defense industries. Infinite-time horizon nonlinear optimal control (ITHNOC) presents a viable option for synthesizing stabilizing controllers for nonlinear systems by making a state-input tradeoff, where the objective is to minimize the cost given by a performance index. The original theory of nonlinear optimal control dates from the 1960s. Various theoretical and practical aspects of the problem have been addressed in the literature over the decades since. In particular, the continuous-time nonlinear deterministic optimal control problem associated with autonomous (time-invariant) nonlinear regulator systems that are affine (linear) in the controls has been studied by many authors. The long-established theory of optimal control offers quite mature and well-documented techniques for solving this control-affine nonlinear optimization problem, based on dynamic programming or calculus of variations, but their application is generally a very tedious task. Bellman’s dynamic programming approach reduces to solving a nonlinear first-order partial differential equation (PDE), expressed by the Hamilton–Jacobi–Bellman (HJB) equation. The solution to the HJB equation gives the optimal performance/cost value (or storage) function and determines an optimal control in feedback form under some smoothness assumptions. Alternatively, in the classical calculus of variations, optimal control problems can be characterized locally in terms of the Hamiltonian dynamics arising from Pontryagin’s minimum principle. These are the characteristic equations of the HJB PDE, which result in a nonlinear, constrained two-point boundary value problem (TPBVP) that, in general, can only be solved by successive approximation of the optimal control input using iterative numerical techniques for each set of initial conditions. Numerically, even though the nonlinear TPBVP is somewhat easier to solve than the HJBPDE, control signals can only be determined offline and are thus best suited for feedforward control of plants for which the state trajectories are known a priori. Therefore, contrary to the dynamic programming approach, the resultant control law is not generally in feedback form. Open-loop control, however, is sensitive to random disturbances and requires that the initial state be on the optimal trajectory. In contrast, nonlinear optimal feedback has inherent robustness properties (inherent in the sense that it is obtained by ignoring uncertainty and disturbances). The potential difficulty with the HJB approach is that no efficient algorithm is available to solve the PDE when it is nonlinear and the problem dimension is high, making it impossible to derive exact expressions for optimal controls for most nontrivial problems of interest. The optimal can only be computed in special cases, such as linear dynamics and quadratic cost, or very low-dimensional systems. In particular, if the plant is linear time invariant (LTI) and the (infinite-time) performance index is quadratic, then the corresponding HJB equation for this infamous linear-quadratic regulator (LQR) problem reduces to an algebraic Riccati equation (ARE). Contrary to the well-developed and widely applied theory and computational tools for theRiccati equation (for example, see [1]), theHJB equation is difficult, if not impossible, to solve for most practical applications. The exact solution for the optimal control policies is very complex

An adaptive system for linear systems with unknown parameters is a nonlinear system. The analysis of such adaptive systems requires similar techniques to analyse nonlinear systems. Therefore it is natural to treat adaptive control as a part of nonlinear control systems. Nonlinear and Adaptive Control Systems treats nonlinear control and adaptive control in a unified framework, presenting the major results at a moderate mathematical level, suitable for MSc students and engineers with undergraduate degrees. Topics covered include introduction to nonlinear systems; state space models; describing functions for common nonlinear components; stability theory; feedback linearization; adaptive control; nonlinear observer design; backstepping design; disturbance rejection and output regulation; and control applications, including harmonic estimation and rejection in power distribution systems, observer and control design for circadian rhythms, and discrete-time implementation of continuous-time nonlinear control laws.

This paper proposes an Approximate Dynamic Programming (ADP) approach for obtaining the optimal real-time operation strategy of microgrid with power-to-hydrogen (P2H) device. Firstly, the real-time operation model of microgrid with P2H is established, then a piece-wise linear function based approximate dynamic programming is proposed to solve the operation model. With the direction of the slope update and training policies under various training scenarios, the proposed ADP approach can obtain the approximate optimal real-time operation strategies to cope with uncertainties. Case studies verify the superiority of the proposed ADP approach compared with Model Predictive Control approach (MPC).

Construction of the left coprime fractional representation for a class of nonlinear control systems

An algorithm, based on quasi-linearization and state variable parametrization by using Chebyshev polynomials, is proposed to solve the nonlinear optimal control problem subject to terminal state constraints and control saturation constraints. By the virtue of the second method of quasi-linearization and state parametrization, the difficult nonlinear optimal control problem is converted into a sequence of standard quadratic programming problems. An example is given to show the effectiveness of the proposed algorithm.

Here, a two-phase algorithm based on integrating particle swarm optimization (PSO) with modified hybrid genetic algorithm (MHGA) is proposed for solving the associated nonlinear programming problem of a nonlinear optimal control problem. In the first phase, PSO starts with a completely random initial swarm of particles, where each of them contains two random matrices in time nodes. After phase 1, to achieve more accurate solutions, the number of time nodes is increased. The values of the associated new control inputs are estimated by linear or spline interpolations using the curves computed in the phase 1. In addition, to maintain the diversity in the population, some additional individuals are added randomly. Next, in the second phase, MHGA, starts by the new population constructed by the above procedure and tries to improve the obtained solutions at the end of phase 1. MHGA combines a GA with a successive quadratic programming, SQP, as a local search. Finally, we implement the proposed algorithm on some well-known nonlinear optimal control problems. The numerical results show that the proposed algorithm can find almost better solution than other proposed algorithms. Keywords: Optimal control problem, Hybrid genetic algorithm, ?particle swarm optimization, Spline interpolation.

A new fractional orthogonal basis and its application in nonlinear delay fractional optimal control problems

In this paper, the nonlinear optimal control problem is investigated for spacecraft formation flying with collision avoidance. Based on a nonlinear model of formation flying, two optimal tracking control laws are proposed to ensure the formation pattern converges to the predetermined configuration. The first optimal control law which combines Lyapunov optimizing control with a trajectory-following optimization technique is developed to solve the finite-time nonlinear optimal control problem. For the second controller, an extended [Formula: see text]-[Formula: see text] method is applied to design a closed-form feedback control scheme for nonlinear control problem of spacecraft formation flying with non-standard cost functions. By taking into account the flying safety, a repulsive control scheme is incorporated in the optimal controller to ensure collision avoidance and minimize the performance index. Finally, a numerical example is performed to demonstrate the effectiveness of the proposed approaches.