In addressing control problems such as regulation and tracking through reinforcement learning (RL), it is often required to guarantee that the acquired policy meets essential performance and stability criteria such as a desired settling time and steady-state error before deployment. Motivated by this, we present a set of results and a systematic reward-shaping procedure that: 1) ensures the optimal policy generates trajectories that align with specified control requirements and 2) allows to assess whether any given policy satisfies them. We validate our approach through comprehensive numerical experiments conducted in two representative environments from OpenAI Gym: the Pendulum swing-up problem and the Lunar Lander. Utilizing both tabular and deep RL methods, our experiments consistently affirm the efficacy of our proposed framework, highlighting its effectiveness in ensuring policy adherence to the prescribed control requirements.

In this article, we present a control design framework for autonomous visual tracking of a dynamic target by a quadrotor vehicle. The considered problem poses important challenges related to the scale ambiguity, which affects monocular vision systems, and the lack of knowledge of the target dynamic response. These are addressed by leveraging a modular architecture that combines a state-of-the-art visual object tracker with a robust output feedback controller. In particular, by relying on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mu$</tex-math> </inline-formula> -synthesis technique, we design a controller providing suitable stability and performance guarantees. As opposed to previous solutions found in the literature, the proposed design does not require information about the target object depth or its geometry. The effectiveness of our approach is demonstrated through photorealistic simulations, showing that monocular tracking of aerial targets is consistently achieved.

In the aero-engine control, fast response capability is significant in transient performance, especially when aircraft work in emergency scenarios. To improve the response speed of aero-engines, this brief proposes a type of fixed-time control using the switched system approach. First, aero-engines are modeled as switched systems to better reflect the multimode characteristics under various operating tasks. Then, the fixed-time controller is proposed, which has advantages of small computation and easy implementation. Novel criteria are proposed for fixed-time stability analysis of the closed-loop system under restricted switchings. Furthermore, specific calculations are provided to derive the upper bound for the convergence time. Finally, superiorities of the proposed method are validated through experimental results.

In this article, we consider a dc microgrid composed of distributed generation units (DGUs) trading energy among each other, where the energy price depends on the total current generated by all the DGUs. We then use a Cournot aggregative game to describe the self-interested interaction among the DGUs, where each DGU aims at minimizing the deviation with respect to the given reference signals and maximizing the revenue from the sale of the generated power. Thus, we design a fully distributed continuous-time equilibrium-seeking algorithm to compute the generalized Nash equilibrium (GNE) of the game. We interconnect the designed decision-making algorithm with the dynamics of the microgrid in a passive way, and, by leveraging passivity theory, we prove the convergence of the closed-loop system trajectory to a feasible operating point that is also a Nash equilibrium of the collective aggregative game. Finally, we present extensive simulation results that validate the proposed distributed optimal control scheme, showing excellent performance.

Reliable and provably correct performance of control systems in the presence of various types of uncertainties and disturbances are essential objectives that require high integrity of the whole control system. Symbolic control is a candidate method for achieving these objectives which is capable of addressing both robustness and optimality of controller. However, several challenges may arise when the symbolic control method is applied in practice. These include the load of design computations, high memory requirements for design and implementation, and oscillations caused by discretization of inputs. The focus of this work is on treatment of the mentioned challenges during symbolic control synthesis for a boiler system as the central safety critical element in a broad range of industries. To reduce the load of design computations and the required memory while preserving the control precision, a two-stage control scheme is used in which symbolic models with different resolutions are employed. The input oscillations are also reduced by minimizing the changes in the input at the steady state. The performance of the synthesized controllers for several different designs are evaluated and compared during a simulation study. The results demonstrate the suitable performance of the symbolic control method when applied to a boiler system.

This article studies the problem of dynamic modeling, fault detection, and fault identification (FDI) for small spacecraft propulsion systems, specifically cold gas (CG) and resistojet thrusters. CG thrusters propel spacecraft by discharging pressurized propellant through a nozzle. The resistojet thruster is identical to the CG thruster with the addition of a heating section, which heats up the propellant before discharging it through the nozzle, thereby reducing fuel consumption. To implement model-based FDI on these systems, the laws of conservation of mass, energy, and momentum are employed to develop control-oriented dynamical models for both CG and resistojet thrusters. Both thruster models are then tuned and validated using experimental data from a thruster test stand. Investigating the mismatch between the model prediction and experimental data, the dominant sources of uncertainty affecting the models are identified. A suitable model-based FDI technique based on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$H_{\infty}$</tex-math> </inline-formula> framework is then selected from the literature and is tailored to the problem at hand. Finally, the performance of the FDI technique is assessed in both simulation-based studies and real experiments. In particular, as part of the simulation-based study, a Monte Carlo analysis is employed to show that the proposed technique can robustly detect and isolate common faults without triggering false positives resulting from uncertainties and disturbances.

Modern vehicles are commonly sold with air conditioning systems. Especially in climates with traditionally high temperatures, these systems are not just for comfort but highly relevant also to safety. Various hardware setups for cooling and heating have been used to climatize the vehicle cabin. No matter which setup is used though, a cabin temperature controller needs to be implemented. Recent developments see an increase of multizone climate control methods. Even though various approaches have been presented, the task of temperature tracking and disturbance rejection remains challenging. In this article, it is shown using experiments that separating cooling circuit control from cabin temperature control does not affect the performance of the cabin control loop. For cabin temperature control, an approach is presented which allows for simultaneous air outlet temperature and mass flow management based on previous work. This is possible in multiple vehicle zones. The approach is based on a gray-box model of vent system and cabin. A mass flow management system based on real-time steady-state optimization grants simultaneous tracking and disturbance rejection. The weakness of this previous work, namely, the dependence on careful hand-tuning and mass flow independent tuning is overcome by applying an adaptive tuning approach. All results are experimentally validated using an experimental vehicle in real traffic for the two-zone control case. Consistent behavior over a wide range of mass flows and temperatures was reached. This correlates to a consistent thermal experience for passengers. Overshoot was effectively eliminated by applying the adaptive tuning. Concluding, an analysis of the move to practical implementation is shown. In this, a cabin temperature observer is presented that demonstrates the difficulties of implementing accurate three-zone control on a limited set of sensors. Based on the implemented cabin temperature observer, tracking can be achieved on a reduced sensor setup albeit at the cost of worse closed-loop controller performance.

Unmanned surface vehicles (USVs) operating in marine environments are often subjected to external disturbances, such as water waves and currents that might considerably affect system dynamics. Due to the complexity of existing tools used to capture their effects on system dynamics, they are often discarded or at most replaced by a mild form of noise. Indeed, traditional estimation methods often rely on complex procedures involving a battery of experiments conducted both in laboratory settings and outdoors in actual operating environments, followed by intensive hydrodynamics computations. Stochastic noise has been shown in the literature to better capture the uncertainties acting on marine vehicles. In this work, we propose a stochastic model to recreate the disturbances observed on small catamaran USVs moving at lower speed in their operating environment. A maximum likelihood method is proposed to identify the noisy dynamics of the USVs using limited amounts of experimental data gathered in the operating environment. Analytical expressions are derived which reduce the computational effort required to estimate model parameters. The proposed framework is shown to be effective at replicating the distribution of the noise and predicting the future trajectories of the USVs by a few time horizons in actual operating environments.

The safety of ship hulls, reservoir dams, and bridge piers must be ensured through routine fracture detection and damage repair. An underwater robot with eight thrusters and six C-shaped legs has been built to be able to walk on these intricate underwater structures. In this article, a slip-regulated optimal walking control strategy is proposed while taking into account vehicle dynamics. The controller also produces the necessary longitudinal force and yaw moment, so that the robot follows the desired forward velocity and heading direction. It then equivalently transfers the needed force and moment to the contact points of all supporting legs. The suggested controller also includes appropriate constraints and a cost function to regulate the slippage of supporting legs. In addition to preventing longitudinal slippage and enabling lateral slippage, the proposed control framework enhances walking stability and steering ability. Finally, the effectiveness of the proposed method is verified through the Gazebo platform and the hexapod robot.

This work proposes an innovative path-following control method, anchored in deep reinforcement learning (DRL), for unmanned underwater vehicles (UUVs). This approach is driven by several new designs, all of which aim to enhance learning efficiency and effectiveness and achieve high-performance UUV control. Specifically, a novel experience replay strategy is designed and integrated within the twin-delayed deep deterministic policy gradient algorithm (TD3). It distinguishes the significance of stored transitions by making a trade-off between rewards and temporal-difference (TD) errors, thus enabling the UUV agent to explore optimal control policies more efficiently. Another major challenge within this control problem arises from action oscillations associated with DRL policies. This issue leads to excessive system wear on actuators and makes real-time application difficult. To mitigate this challenge, a newly improved regularization method is proposed, which provides a moderate level of smoothness to the control policy. Furthermore, a dynamic reward function featuring adaptive constraints is designed to avoid unproductive exploration and expedite learning convergence speed further. Simulation results show that our method garners higher rewards in fewer training episodes compared with mainstream DRL-based control approaches (e.g., deep deterministic policy gradient (DDPG) and vanilla TD3) in UUV applications. Moreover, it can adapt to varying path configurations amid uncertainties and disturbances, all while ensuring high tracking accuracy. Simulation and experimental studies are conducted to verify the effectiveness.