Adaptive wastewater flow control using AquaFlowNet with reinforcement learning for urban drainage optimization

We calculate the energy demand of automated vehicles for different driving cycles. We alter standard driving cycles to depict the driving behavior of automated vehicles. We further assume additional energy demand for automation systems and investigate trade-offs between reductions in mechanical energy demand and increases in auxiliary energy demand. In the case of trucks, we find that smoother driving and the additional energy demand offset one another for highway driving. However, a notable reduction in energy demand can be achieved by lowering the maximum driving speed. For cars, we find that the additional energy demand slightly outweighs the effects of smoother driving on highways. When considering city driving, the additional energy demand increases the energy demand of a mid-size car in the standard driving cycle by one third. Reducing driving speeds and stops is not able to offset this increase in energy demand.

It is known that glazed openings are very important elements in the energetic behavior of buildings, especially in Mediterranean climates where there are many hours of solar radiation. The objective of this study was to determine the influence of solar protection on the energy demand of public housing structures in these climates. To this end, the reduction in energy demand achieved by fixed solar protectors in combination with mobile protectors (blinds) was quantified, including an evaluation of the influence of their geometry, dimensions, degree of openness, and the orientation of the opening. To analyze and quantify energetic demand, a block of public housing units in a neighborhood of Seville (latitude 37°23' N) was used as a model. This block is typical of public housing in the Mediterranean region. Simulated energetic models were created using DesignBuilder, achieving reductions in the annual energy demand from 10% to 27% according to the orientation chosen. The results and conclusions of the study are applicable to new construction, energetic rehabilitation projects, and/or the improvement of existing buildings.

SummaryIn this study, we present a reinforcement learning (RL)‐based flight control system design method to improve the transient response performance of a closed‐loop reference model (CRM) adaptive control system. The methodology, known as RL‐CRM, relies on the generation of a dynamic adaption strategy by implementing RL on the variable factor in the feedback path gain matrix of the reference model. An actor‐critic RL agent is designed using the performance‐driven reward functions and tracking error observations from the environment. In the training phase, a deep deterministic policy gradient algorithm is utilized to learn the time‐varying adaptation strategy of the design parameter in the reference model feedback gain matrix. The proposed control structure provides the possibility to learn numerous adaptation strategies across a wide range of flight and vehicle conditions instead of being driven by high‐fidelity simulators or flight testing and real flight operations. The performance of the proposed system was evaluated on an identified and verified mathematical model of an agile quadrotor platform. Monte‐Carlo simulations and worst case analysis were also performed over a benchmark helicopter example model. In comparison to the classical model reference adaptive control and CRM‐adaptive control system designs, the proposed RL‐CRM adaptive flight control system design improves the transient response performance on all associated metrics and provides the capability to operate over a wide range of parametric uncertainties.

Keeping cool in acute liver failure: Rationale for the use of mild hypothermia

Work of breathing accounts for a significant proportion of total oxygen consumption in neonates and infants. Endotracheal tube inner diameter and length significantly affect airflow resistance and thus work of breathing. While endotracheal tube shortening reduces endotracheal tube resistance, the impact on work of breathing in mechanically ventilated neonates and infants remains unknown. The objective of this in vitro study was to quantify the effect of endotracheal tube shortening on work of breathing in simulated pediatric lung settings. We hypothesized that endotracheal tube shortening significantly reduces work of breathing. We used the Active-Servo-Lung 5000 to simulate different clinical scenarios in mechanically ventilated infants and neonates under spontaneous breathing with and without pressure support. Endotracheal tube size, lung resistance, and compliance, as well as respiratory settings such as respiratory rate and tidal volume were weight and age adapted for each lung model. Work of breathing was measured before and after maximal endotracheal tube shortening and the reduction of the daily energy demand calculated. Tube shortening with and without pressure support decreased work of breathing to a maximum of 10.1% and 8.1%, respectively. As a result, the calculated reduction of total daily energy demand by endotracheal tube shortening was between 0.002% and 0.02%. In this in vitro lung model, endotracheal tube shortening had minimal effects on work of breathing. Moreover, the calculated percentage reduction of the total daily energy demand after endotracheal tube shortening was minimal.

Traffic signal control can be naturally regarded as a reinforcement learning problem. Unfortunately, it is one of the most difficult classes of reinforcement learning problems owing to its large state space. A straightforward approach to address this challenge is to control traffic signals based on continuous reinforcement learning. Although they have been successful in traffic signal control, they may become unstable and fail to converge to near-optimal solutions. We develop adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that is more stable. The effect of three feature functions is empirically investigated in a microscopic traffic simulation. Furthermore, the effects of departing streets, more actions, and the use of the spatial distribution of the vehicles on the performance of CRL-TSCs are assessed. The results show that the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller.

Control methodologies of traffic signals have significantly improved during the recent past along with advancements in technology. Adaptive traffic signal control is the most recent and advanced control type of traffic signals. Adaptive control is able to efficiently relieve traffic congestion by continuously adjusting signal timings according to real-time traffic conditions. Conventional optimization methods such as integer programming, hill climbing, or descent gradient searching have been gradually overshadowed by genetic algorithms in many areas including traffic signal operation. The research presented in this article is distinct from previous studies in that it focuses on real-time adaptive signal optimization using genetic algorithms. The proposed adaptive signal system provides acyclic signal operation based on a rolling horizon real-time control approach. The algorithm was tested using microsimulation for on-line evaluation and comparison to fixed-time plans generated from the latest TRANSYT-7F version 9.7, which has a genetic optimization feature. The developed signal system consists of three major components including a genetic algorithm optimization module, an internal traffic simulation module, and a database management system all working in cooperation to optimize signal timings in real time. Using the pseudo on-line simulation platform, three testing scenarios for high, medium, and low level of traffic demands were conducted focusing on evaluating several important features of the proposed adaptive signal control system. The test results indicated that real-time genetic control outperformed fixed-signal timing plan in all scenarios based on total vehicle delay.

The passivity-based control (PBC) is not robust and it relies upon the system model. Moreover, partial differential equations (PDE) are encountered during its designing process which are difficult to be solved and in some cases unfeasible. In this article, reinforcement learning (RL) designs the PBC parameters via solving PDE online. RL and adaptive control are employed in order to make the nonlinear closed-loop system robust against the disturbance and model uncertainty. Through the utilisation of adaptive control technique, the passivity-based controller design along with learning could be executed as though the disturbance within the system could also be eliminated. The simulations and the comparison made with the previous methods manifest the greater advantage and superiority of the proposed method.

Cloud computing has reached the core of the digital ecosystems and provides high-availability and scale-able access to the services of computational services. Nevertheless, the active resource requirements, enforcement of service-level agreement (SLA), energy usage restrictions, and a variation in the cost-performance represent significant challenges to the cloud resource management. The conventional heuristic-based schedulers in clouds orchestrators like Kube and OpenStack fail to adjust to real-time workload variability, fractional resources, violation of SLA and inflated operational costs frequently occur. The adaptive, reward-driven decision-making properties of reinforcement learning (RL) have been discovered to be an interesting alternative to a heuristic policy. However, centralized RL has scalability and state observability scale in distributed clouds. As a way of overcoming these limitations, this study hypothesizes a Multi-Agent Reinforcement Learning (MARL) model of autonomous resource scheduling and optimization in cloud infrastructures. The offered structure suggests the use of decentralized intelligent agents executing a coordinated management of computational, storage, and network allocations. The agents are optimized to adapt local behaviors under environment feedback with the aid of cooperative communication to enhance global optimization. The system uses hybrid in reinforcement system consisting of Deep Q-Learning (DQN) and Proximal Policy Optimization (PPO) to trade between exploration, stability, and on-policy improvement. The goal of an adaptive reward model is to maximize important performance indicators (KPIs): SLA adherence, task completion latency, energy efficiency and balance resource utilization. The Cloud sim Plus experimental measurements were performed in a simulated multi-cluster environment and supported by extra features of container virtualization. The proposed version of the MARL scheduler showed a better throughput (18.7%), SLA violations reduction (27.4%) and energy consumption (14.3) than the baseline strategies like Round-Robin, Min-Min, and centralized DQN models. Convergence time was reduced and decision correlation under the workload bursts was improved due to the incorporation of the inter-agent communication. The research paper has a tremendous impact in the field by proposing a foresight, smart cloud orchestration framework that has the capability to adapt itself in the distributed large scale systems. Additionally, we offer an improvement in algorithms, performance analysis, system workflow optimization, and theoretical support of a cooperative reinforcement learning in clouds. The implication of these findings in the future is that the applicability of the research to edge and fog computing settings can be applied to Industry 5.0 automation

This study presents an artificial intelligence (AI)-enabled framework for adaptive and predictive control in Laser Metal Arc Welding (LMAW) operations. The proposed system integrates machine learning (ML) for defect prediction and reinforcement learning (RL) for dynamic process optimization. Multi-sensor data—including thermal, optical, and acoustic signals—are acquired in real time to train predictive models and guide parameter control. A 50 kW industrial-like LMAW case study is developed to evaluate system performance under variable operating conditions. Results demonstrate that the AI-optimized control strategy reduces energy consumption by 22 %, decreases defect occurrence by 30 %, and lowers CO₂ emissions by approximately 18 % compared with baseline operation. The predictive model achieves an accuracy of 96 % in identifying weld defects, while RL-based control stabilizes the arc temperature within ±3 % of the target range (2500–3100 °C). The findings confirm that AI-driven process intelligence can significantly enhance welding efficiency, quality, and sustainability in alignment with Industry 4.0 objectives

Convergent evolution in organismal function can arise from nonconvergent changes in traits that contribute to that function. Theory predicts that low resource availability and high maintenance costs in extreme environments select for reductions in organismal energy demands, which could be attained through modifications of body size or metabolic rate. We tested for convergence in energy demands and underlying traits by investigating livebearing fish (genus Poecilia) that have repeatedly colonized toxic, hydrogen sulphide-rich springs. We quantified variation in body size and routine metabolism across replicated sulphidic and non-sulphidic populations in nature, modelled total organismal energy demands, and conducted a common-garden experiment to test whether population differences had a genetic basis. Sulphidic populations generally exhibited smaller body sizes and lower routine metabolic rates compared to non-sulphidic populations, which together caused significant reductions in total organismal energy demands in extremophile populations. Although both mechanisms contributed to variation in organismal energy demands, variance partitioning indicated reductions of body size overall had a greater effect than reductions of routine metabolism. Finally, population differences in routine metabolism documented in natural populations were maintained in common-garden reared individuals, indicating evolved differences. In combination with other studies, these results suggest that reductions in energy demands may represent a common theme in adaptation to physiochemical stressors. Selection for reduced energy demand may particularly affect body size, which has implications for life history evolution in extreme environments.

Many challenging real-world control problems require adaptation and learning in the presence of uncertainty. Examples of these challenging domains include aircraft adaptive control under uncertain disturbances [1], [2], multiple-vehicle tracking with space-dependent uncertain dynamics [3], [4], robotic-arm control [5], blimp control [6], [7], mobile robot tracking and localization [8], [9], cart-pole systems and unicycle control [10], gait optimization in legged robots [11] and snake robots [12], and any other system whose dynamics are uncertain and for which limited data are available for model learning. Classical model reference adaptive control (MRAC) [13]-[15] and reinforcement learning (RL) methods [16]-[23] have been developed to address these challenges and rely on parametric adaptive elements or control policies whose number of parameters or features are fixed and determined a priori. One example of such an adaptive model are radial basis function networks (RBFNs), with RBF centers pre-allocated based on expected operating domains [24], [25].

Safe Chance Constrained Reinforcement Learning for Batch Process Optimization and Control

Ensuring the safe and reliable operation of quadrotor unmanned aerial vehicles (UAVs) under actuator faults is critical for practical applications. Conventional control methods often face challenges to simultaneously accommodate system constraints and adapt to actuator faults, potentially degrading performance under uncertainty. To address these limitations, this paper proposes an adaptive fault-tolerant control framework integrating Model Predictive Control (MPC) with Deep Deterministic Policy Gradient (DDPG) reinforcement learning. Within this framework, MPC serves as the baseline controller by generating feasible inputs that satisfy constraints. Key to this approach is the introduction of a Lyapunov-based contraction constraint, which replaces conventional terminal constraints to guarantee system stability and recursive feasibility while significantly reducing the required prediction horizon and computational burden. Concurrently, the DDPG agent learns to generate compensatory and auxiliary control signals, thereby mitigating external disturbances and actuator faults. A key innovation lies in employing a parameterized NMPC formulation as the Actor within the DDPG framework. This enables online adaptation of cost function weights and fault compensation parameters through policy optimization. The simulation results demonstrate that our hybrid MPC-DDPG approach achieves accelerated trajectory recovery and exhibits superior tracking accuracy, fault tolerance, and robustness under actuator fault scenarios.

Evolution of cooperation in spatial public goods games driven by reinforcement learning and environmental feedback