Adaptive Critic Control Design With Knowledge Transfer for Wastewater Treatment Applications

The design of fight control is full of challenges because of the complex model and the complicated fight environment. The traditional control methods no longer ensure the demand in the many fight modes. Advanced control methods attract more and more researchers to make application research in this field. Adaptive critic design (ACD) is an effective control method to learn the optimal control online. The performance of fight control with atmospheric disturbance needs to be improved using active control or robust control. This paper presents adaptive critic based controller design of aircraft with atmospheric disturbance. The controller does not rely on the parameters of the aircraft, and it can learn online very well. When there are atmospheric disturbances, such as random turbulence, discrete gust and wind shear, adaptive critic based controller can resist the disturbances effectively, and shows the good robust performance. Finally, the simulation proves the validity of the adaptive critic based controller for aircraft with atmospheric disturbance.

This paper proposes an adaptive critic design-based event-triggered optimal control method for input-constrained continuous-time nonlinear systems. Adaptive critic design is a special framework of adaptive dynamic programming that approximates the value function by a critic neural network and derives the approximate optimal control policy through analytical methods. The proposed adaptive critic design considers the control input constraints by introducing a non-quadratic cost function and employs an event-triggered mechanism to reduce the number of controller executions. Unlike the existing event-triggered adaptive critic design, this paper proposes a novel finite-time adaptive law based on regression filtering scheme. The adaptive law utilizes the error information of the network weights to ensure fast convergence to the optimal control law under the event-triggered mechanism, which improves the real-time performance of the system. Additionally, explicit bounds for each parameter in the compact set and the specific convergence time estimates are provided in the convergence analysis. Finally, the effectiveness and practicality of the proposed method for real-time online applications are validated through two simulation examples.

A new fuzzy identification method for unknown nonlinear discrete systems is presented by introducing adaptive critic designs into the generalized fuzzy hyperbolic model (GFHM). This method minimizes the long-time error other than the immediate error to improve the identification effect. We first represent the GFHM with a neural network structure, and then utilize the adaptive critic designs (ACDs) to get the optimal parameters of the network so that the long-time identification error is minimized. Finally, we give a simulation example to verify the effectiveness of this identification method.

This paper presents the design of a neuro-controller for a turbogenerator using a novel technique based on adaptive critic designs (ACD). This adaptive critic design based neuro-controller augments/replaces the traditional automatic voltage regulator (AVR) and the turbine governor of the generator. Simulation results are presented to show that neural network controllers with the ACD have the potential to control turbogenerators when system conditions and configuration changes.

This paper presents a method for real-time tuning of cost functions in coordinated wide area controller for improved power system transient stability. The algorithm is an optimal Wide Area System-Centric Controller and Observer (WASCCO) based on Adaptive Critic Design (ACD). ACD techniques using reinforcement learning (RL) are able to approximate dynamic programming based nonlinear optimal control but, the cost function for the optimal control requires online tuning. Here cost function defined in ACD are analyzed online with Lyapunov stability function. Further an online tuning method is designed based on eigenvalues extracted based on Prony analysis and linked to performance index generation. Results on a two area system shows better response compared to conventional schemes.

Traditionally, fed-batch biochemical process optimization and control uses complicated off-line optimizers, with no online model adaptation or re-optimization. This study demonstrates the applicability of a class of adaptive critic designs for online re-optimization and control of an aerobic fed-batch fermentor. Specifically, the performance of an entire class of adaptive critic designs, viz., heuristic dynamic programming, dual heuristic programming and generalized dual heuristic programming, was demonstrated to be superior to that of a heuristic random optimizer, on optimization of a fed-batch fermentor operation producing monoclonal antibodies.

In the present paper on-line action-dependent heuristic neuro-dynamic programming was applied for optimization of a complex nonlinear production process. The approach is based on recurrent neural network architecture — Echo state network (ESN) — as critic network within the frame of adaptive critic design (ACD) scheme. The subject for optimization is bio-ethanol fermentation. The aim was to increase process productivity defining optimal dilution rate profile. The obtained by simulations results are good basis for further experimental investigations.

The boiler combustion process of power plant is a typical process with the features of multi-input, multi-output, strong non-linearity, strong jamming and close coupling. The coupling relationship between its parameters correlated with combustion process is very anfractuous, so it is very hard to solve its optimal control problem with conventional control methods. And Adaptive Critic Designs (ACDs) is a good way to deal with the approximate optimal control problems over time in complex nonlinear systems. But most of the ACD structures were designed by BP (Back Propagation) neural network, the controllers are easy to fall into local minimum, so usually the learning efficiency is very low even fail to training. In order to speed up the learning process of the controllers, this paper try to design an optimal controller based on Dual Heuristic Programming (DHP) by Generalized Radial Basis Function Neural Network (GRBFNN) and applied it to the simulation control of the boiler combustion process. The results indicate that the designed controller is effective.

A neural net (NN)/fuzzy logic (FL) adaptive critic controller is described. This structure takes advantage of the decision‐making properties of a FL system to critique and tune a NN action‐generating network. The stability of the proposed structure is proven. NN and fuzzy weight tuning algorithms are given that do not require complicated initialization procedures or any off‐line learning phase. Tracking and bounded NN weights and control signals are guaranteed. The adaptive fuzzy critic controller given here is a model‐free controller' in the sense that it works for any system in a prescribed class without the need for extensive modeling and preliminary analysis to find a regression matrix'. There is no linearity in the parameter (LIP) requirement. Copyright © 2003 John Wiley & Sons, Ltd.

Optimal [formula omitted] tracking control of nonlinear systems with zero-equilibrium-free via novel adaptive critic designs

Strict effluent quality standards and physical limitations of the equipment make accurately controlling dissolved oxygen (DO) concentration in wastewater treatment processes (WWTPs) a challenging task. In this paper, a data-driven model predictive control strategy with performance constraint (PC-MPC) is proposed to ensure accurate tracking of DO concentration set-point. First, a data-driven model is established using the fuzzy neural network, effectively addressing the problem of complex nonlinear modeling of WWTPs and providing accurate predictive output for the control process. Second, a model predictive control (MPC) strategy with performance constraint is designed, in which tracking error is considered as a performance constraint index within the MPC framework, enabling accurate tracking of the DO concentration to the desired set-point. Third, an adaptive constraint optimization algorithm based on the Levenberg-Marquardt (L-M) method is developed. By utilizing the exponential penalty function to reformulate the system constraints, the finite horizon optimization control problem can be directly solved to determine the optimal control sequence. Finally, the stability of the proposed PC-MPC strategy is proven through theoretical analysis and its effectiveness is validated on the benchmark simulation model No. 1.

A new approach for automotive engine torque and air-fuel ratio control is presented in this paper. A class of adaptive critic designs that can be classified as (model-free) action-dependent heuristic dynamic programming is used in the present project. Adaptive critic designs are defined as designs that approximate dynamic programming in the general case, i.e., approximate optimal control over time in noisy, nonlinear environment. The present work uses a system, called "critic", to approximate the cost function in dynamic programming and thus to achieve optimal control. The goals of the present learning control design for automotive engines are emissions reduction and maintenance of optimal performance under various operating conditions. Using the data obtained from a test vehicle, we first develop a neural network model of the engine. A neural network controller is then designed based on the idea of approximate dynamic programming to achieve optimal control. In the simulation studies, the initial controller is trained using the neural network engine model developed rather than the actual engine. We have developed and tested self-learning neural network controllers for both engine torque and exhaust airfuel ratio control. For both control problems, good transient performance of the neural network controller has been observed. A distinct feature of the present technique is the controller's real-time adaptation capability based on real vehicle data which allows the neural network controller to be further refined and improved in real-time vehicle operation through continuous learning.

This paper is concerned with the robust optimal tracking control strategy for a class of nonlinear multi-input multi-output discrete-time systems with unknown uncertainty via adaptive critic design (ACD) scheme. The main purpose is to establish an adaptive actor-critic control method, so that the cost function in the procedure of dealing with uncertainty is minimum and the closed-loop system is stable. Based on the neural network approximator, an action network is applied to generate the optimal control signal and a critic network is used to approximate the cost function, respectively. In contrast to the previous methods, the main features of this paper are: 1) the ACD scheme is integrated into the controllers to cope with the uncertainty and 2) a novel cost function, which is not in quadric form, is proposed so that the total cost in the design procedure is reduced. It is proved that the optimal control signals and the tracking errors are uniformly ultimately bounded even when the uncertainty exists. Finally, a numerical simulation is developed to show the effectiveness of the present approach.

This paper investigates the feasibility of applying\nreinforcement learning (RL) concepts to industrial process\noptimisation. A model-free action-dependent adaptive critic\ndesign (ADAC), coupled with sequential learning neural\nnetwork training, is proposed as an online RL strategy\nsuitable for both modelling and controller optimisation. The\nproposed strategy is evaluated on data from an industrial\ngrinding process used in the manufacture of disk drives.\nComparison with a proprietary control system shows that\nthe proposed RL technique is able to achieve comparable\nperformance without any manual intervention.

We study a model of evolving populations of self-learning agents and analyze the interaction between learning and evolution. We consider agent-brokers that predict stock price changes and use these predictions for selecting actions. Each agent is equipped with a neural network adaptive critic design for behavioral adaptation. We discuss three cases in which either learning, or evolution, or both, are active in our model. We show that the Baldwin effect can be observed in our model, viz., originally acquired adaptive policy of best agent-brokers becomes inherited over the course of the evolution. Additionally, we analyze influence of neural network structure of adaptive critic design on learning processes.