Reinforcement learning-based optimal fault-tolerant control for offshore platforms

This chapter presents an optimal tracking control methodology for an offshore steel jacket platform subject to external wave force. Based on a dynamic model of an offshore steel jacket platform with an AMD mechanism and a linear exogenous system model of the external wave force on the offshore platform, an optimal tracking control scheme with feedforward compensation is proposed to attenuate wave-induced vibration of the offshore platform. A feedforward and feedback optimal tracking controller (FFOTC) can be obtained by solving an algebraic Riccati equation and a Sylvester equation, respectively. It is demonstrated that the wave-induced vibration amplitudes of the offshore platform under the FFOTC are much smaller than the ones under the feedback optimal tracking controller (FOTC) and the feedforward and feedback optimal controller (FFOC). Furthermore, the required control force under the FFOTC is smaller than the ones under the FOTC and the FFOC.

In this paper, a novel active optimal fault-tolerant control (FTC) scheme is designed based on adaptive dynamic programming (ADP) for modular manipulator when sensor and actuator faults are concurrency. Firstly, the sensor fault is transformed into the pseudo-actuator fault by constructing a nonlinear transformation with diffeomorphism theory. Secondly, the faults estimated by the adaptive fault observer are applied to establish an improved performance index function. Next, the online policy iteration (PI) algorithm is used to solve the Hamilton-Jacobi-Bellman (HJB) equation via establishing a critic neural network. The optimal fault-tolerant controller is proved to be uniformly ultimately bounded (UUB) based on Lyapunov stable theory. Finally, the effectiveness of the proposed multi-fault-tolerant control algorithm is verified by simulation results.

1 A Survey on Computational Optimal Control.- Issues in the Direct Transcription of Optimal Control Problems to Sparse Nonlinear Programs.- Optimization in Control of Robots.- Large-scale SQP Methods and their Application in Trajectory Optimization.- Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity.- 2 Theoretical Aspects of Optimal Control and Nonlinear Programming.- Continuation Methods In Boundary Value Problems.- Second Order Optimality Conditions for Singular Extremals.- Synthesis of Adaptive Optimal Controls for Linear Dynamic Systems.- Control Applications of Reduced SQP Methods.- Time Optimal Control of Mechanical Systems.- 3 Algorithms for Optimal Control Calculations.- Second Order Algorithm for Time Optimal Control of a Linear System.- An SQP-type Solution Method for Constrained Discrete-Time Optimal Control Problems.- Numerical Methods for Solving Differential Games, Prospective Applications to Technical Problems.- Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances.- Repetitive Optimization for Predictive Control of Dynamic Systems under Uncertainty.- Optimal Control of Multistage Systems Described by High-Index Differential-Algebraic Equations.- A New Class of a High Order Interior Point Method for the Solution of Convex Semiinfinite Optimization Problems.- A Structured Interior Point SQP Method for Nonlinear Optimal Control Problems.- 4 Software for Optimal Control Calculations.- Automated Approach for Optimizing Dynamic Systems.- ANDECS: A Computation Environment for Control Applications of Optimization.- Application of Automatic Differentiation to Optimal Control Problems.- OCCAL: A mixed symbolic-numeric Optimal Control CALculator.- 5 Applications of Optimal Control.- A Robotic Satellite with Simplified Design.- Nonlinear Control under Constraints of a Biological System.- An Object-Oriented Approach to Optimally Describe and Specify a SCADA System Applied to a Power Network.- Near-Optimal Flight Trajectories Generated by Neural Networks.- Performance of a Feedback Method with Respect to Changes in the Air-Density during the Ascent of a Two-Stage-To-Orbit Vehicle.- Linear Optimal Control for Reentry Flight.- Steady-State Modelling of Turbine Engine with Controllers.- Shortest Paths for Satellite Mounted Robot Manipulators.- Optimal Control of the Industrial Robot Manutec r3.

This paper studies the problem of the design method of the active fault-tolerant controller, under the constraint of control input saturation, for the circumstance of simultaneous actuator and sensor faults. Considering the system with control input saturation, the design method of fault-tolerant tracking controller based on adaptive law and LMI is proposed, the saturation constraint is described by convex combination method, the adaptive law applied in actuator fault is designed, and the problem of the design of controller is transformed into optimization problem under the constraint of linear matrix inequalities. Besides, the proposed method avoids the design of fault reconstruction observer module, which simplifies the system design, and the Effectiveness of the proposed method is verified by a numerical example.

The problem of fault tolerant shape control for particle size distribution (PSD) process subject to simultaneous time‐varying actuator and sensor faults is studied. In this work, sensor and actuator faults are taken into consideration in a unified framework, and estimated by using adaptive observer technique. For the PSD process, the output distribution, rather than the system output signal itself, is adopted for shape control. Square root rational B‐spline approximation is used to fit the distribution shape. An innovative on‐line fault estimation scheme is proposed for simultaneous actuator and sensor faults estimation. Then an augmented controller based on the virtual actuator and virtual sensor techniques is designed to compensate for the faults and realise PSD shape tracking. A simulation example of emulsion polymerisation is used to validate the proposed scheme and satisfactory performance is obtained.

This paper investigates the problem of attitude fault tolerant control (FTC) for satellites subject to multiple disturbances and actuator faults. Due to the limitation of the physical characteristic, actuator saturation exists quite widely in practical processes. An anti-windup FTC approach based on disturbance observer is presented for satellites with actuator faults, input saturation and multiple disturbances in this paper. The multiple disturbances can be divided into an uncertain modeled disturbance and a norm-bounded equivalent disturbance. Disturbance observer and fault diagnosis observer are designed to estimate the uncertain modeled disturbance and time-varying fault, respectively. By virtue of fault accommodation and disturbance observer-based control, an anti-windup FTC strategy is designed with polytopic representation of the saturation function. Different from the previous results, disturbance rejection, fault accommodation and state feedback controller are considered as a whole for input saturation. The proposed fault-tolerant controller design can guarantee the closed-loop system to be quadratically stable with a prescribed upper bound of the cost function. Simulation results are given to show the efficiency of the proposed approach.

This paper presents the design of fault tolerant flight control (FTC) law to deal with actuators loss of effectiveness. A Linear Quadratic Regulator (LQR) approach was considered as a baseline controller augmented with tracking equations to follow command reference inputs. Fault-compensation-based approach was applied to act against actuator faults. The designed FTC law was examined by analysing the aircraft's motion and the changes in actuators deflections. A mathematical model of the lateral motion of a DC-8 aircraft was used in this experiment. Simulation results showed that this technique is very effective to deal with actuator faults and to maintain the stability of the closed-loop system.

In this work, the output-feedback fault-tolerant tacking control issue for underactuated autonomous underwater vehicle (AUV) with actuators faults is investigated. Firstly, an output-feedback error tacking system is constructed based on the theoretical model of underactuated AUV with actuators faults. Then, an adaptive dynamic programming (ADP) based fault-tolerant control controller is developed. In our proposed control scheme, a neural-network observer is designed to approximate the system states with actuators faults. An online policy iteration algorithm is designed with critic network and action network in order to improve the tracking accuracy. Based on Lyapunov stability theorem, the stability of the error tracking system is guaranteed by the proposed controller. At last, the simulation results show that the underactuated AUV achieves better tracking performance.

Active distribution networks (ADNs) with integrated source-network-load-storage units play a vital role in supporting renewable energy integration and improving system flexibility. However, during long-term operation, inevitable communication faults and actuator faults may severely impair critical control objectives, including frequency regulation, voltage stability, active power sharing, and state-of-charge (SoC) equalization. These faults introduce significant risks to system stability and reliable operation, highlighting the need for effective resilience mechanisms. To address this challenge, this paper develops a fault-tolerant control framework that combines real-time fault estimation with resilient control design. First, a deep neural network (DNN)-based estimator is constructed to identify the real-time effects of communication and actuator faults without relying on detailed physical models. Based on the estimated information, a DNN-assisted fault-tolerant controller is then designed to ensure frequency and voltage recovery, equitable power sharing among distributed generators (DGs), and balanced SoC management of energy storage systems (ESSs), even under concurrent communication and actuator faults. This forms a data-driven compensation mechanism that enables resilient performance without relying on conventional reallocation or redundancy-based strategies. To further demonstrate the effectiveness of the proposed method, quantitative simulation results indicate that, compared with existing approaches, the frequency error is reduced by approximately 97%, the voltage error by 60–74%, and the active and reactive power errors by around 90%. Finally, the MATLAB/Simulink analysis of an ADN testing system verifies that these improvements significantly enhance the robustness and reliability of source-network-load-storage ADNs under both normal and faulty conditions, providing a promising solution for resilient operation in modern power systems.

This paper proposes a novel fault-tolerant tracking control strategy for over-actuated systems subject to actuator fault, control input saturation and external disturbance. The proposed method integrates the effect of actuator saturation and the actuator fault into a generalized disturbance. The generalized disturbance is considered as an auxiliary state vector to construct an augmented system. Based on the augmented system, an $H$ ∞ dynamic output feedback controller is designed to achieve fault-tolerant control. The main advantage of the proposed method is that it can achieve robust fault-tolerant control without depending on the fault diagnosis module and on-line control allocation algorithm. The design conditions are formulated as a set of linear matrix inequalities, which can be solved efficiently. Simulation results of a flight vehicle are given to demonstrate the effectiveness of the proposed method.

The fault-tolerant formation control problem for unmanned helicopters (UHs) with actuator faults under fixed and switching communication topologies is investigated in this paper. Firstly, the high-order fully actuated system model of the 6-DOF UH is established, which is divided into the position outer-loop subsystem and the attitude inner-loop subsystem. Secondly, a prescribed-time disturbance observer is constructed to rapidly and accurately estimate the composite disturbances composed of external disturbances and actuator faults. Then, with the aid of the fully actuated system approach, the formation fault-tolerant controller and attitude tracking fault-tolerant controller are designed for the inner and outer loops respectively, which can guarantee the prescribed-time stability of multiple UHs under fixed and switching topologies. Finally, the simulation results are provided to demonstrate the effectiveness of the proposed control strategy.

An integral TSMC-based adaptive fault-tolerant control for quadrotor with external disturbances and parametric uncertainties

In this investigation, we study an optimal control problem for a parabolic‐hyperbolic free boundary problem modeling the growth of a tumor with drug application. In this model, the heterogeneity or different types of tumor cells (caused by mutations and different values of drug and nutrient concentrations inside the tumor) are considered that are included in the model by considering a variable . It is assumed that converting from mutation state to mutation state happens with probability . This model consists of a first‐order hyperbolic equation describing the evolution of tumor cells depending on . It also includes two time‐fractional parabolic equations describing the diffusions of nutrients (e.g., oxygen and glucose) and drug concentrations. In this paper, the concentrations of drug and nutrients on the boundary of the tumor are the control variables, which diffuse inside the tumor and affect the growth of the tumor. A cost function is presented in which the radius of the tumor and the boundary values of drug and nutrients (to limit the side effects of decreasing nutrient and increasing drug) are included. The adjoint equations are presented, and the necessary conditions in terms of adjoint equations are given. The existence and uniqueness of optimal control are also proved. In order to obtain the optimal control values, we have used the direct method. Hence, the problem is discretized using a combination of spectral method and the product trapezoidal approximation for the Caputo fractional order derivative. Then, we have minimized the discretized cost function employing Trust‐region reflective algorithm. Taking into account heterogeneity and moving boundary increases the complexity of the model but provides us with a model that satisfies more properties of real tumors. Despite these details added to the model, we have proved the existence and uniqueness of the optimal control. Finally, some figures are also presented to illustrate the effects of optimal controls and noisy controls on the cost functions and radius of the tumor. Noisy controls are obtained by adding noises generated by normal distributions. The cost function and radius of tumor for noisy and optimal boundary values of nutrient and drug are plotted. It is shown the lowest cost function and radius are for optimal controls when the noisy values satisfy the constraints for the control variables considered in the optimal control problem.


 
 
 This paper presents an adaptive fault-tolerant control (FTC) scheme for leader-follower consensus control of uncertain mobile agents with actuator faults. A local FTC component is designed for each agent in the distributed system by using local measurements and certain information exchanged between neighboring agents. Each local FTC component consists of a fault detection module and a reconfigurable controller module comprised of a baseline controller and an adaptive fault-tolerant controller activated after fault detection. Under certain assumptions, the closed-loop system stability and leader-follower consensus properties of the distributed system are rigorously established. A simulation example is used to illustrate the effectiveness of the FTC method.
 
 

This paper investigates the consensus control problem for unknown nonlinear multi-agent systems (MASs) with both sensor and actuator faults. Different from the existing data-driven control methods for unknown nonlinear MASs without faults, a new data-driven adaptive fault-tolerant control (DDC-FTC) method is proposed. Firstly, the data mapping of agents with sensor and actuator faults is established by the locally dynamic linearisation technique. Then, by employing an adaptive fault compensation method, a fault-tolerant controller is designed for unknown nonlinear MASs. Especially, through the stability analysis, it is proven that the consensus tracking error under the designed controller is bounded, where the error bound is related with the parameters in the proposed compensation mechanism, which can be effectively decreased by tuning the parameters. Finally, simulations are given to verify the correctness of the proposed DDA-FTC approach.