Design of Self-Organizing Systems Using Multi-Agent Reinforcement Learning and the Compromise Decision Support Problem Construct

To reduce development cost and to avoid late design fixes in aircraft design, methods that are fast and economic in estimating the aerodynamic characteristics of complex flight vehicles at the preliminary design stage are desired. This work and thesis focus on the adaptive-fidelity CFD approach, with emphasis on the high end of the CFD tools available today. The core idea of the method is to use computationally cheap modeling in the part of the flight envelope where it is applicable. When the complexity in the flow field increases more details and realism is included in the mathematical model, at a computationally higher cost. A typical case where this would be required could be at the border of the flight envelope, where flow phenomena such as shocks, flow separation, and interacting vortex systems could occur. Since the number of cases needed to resolve the flight envelope could be in the order of ten thousands automation is required. The bottlenecks are the discretization of the fluid volume and evaluation of raw CFD data and post processing of the data. These issues are also discussed in this work. The method has been tested on two real flying aircraft, the X-31 delta-winged aircraft with vector thrust, and the Ranger 2000 Jet trainer, as well as on the SACCON preliminary wing-body UCAV design. The results provide improved understanding of the usefulness of this method as an analysis tool during the preliminary design phase all the way into the flight test diagnostic phase of a new aircraft.

In the preliminary design phase of aircraft design, estimating the production cost accurately is a challenging task. At this stage, many design parameters that affect the overall cost are still undefined. This paper establishes cost-estimation models for civil, commercial aircraft using a parametric cost analysis (PCA) approach. Aircraft are characterised based on their size, ranging from a wide body to executive jets, into four categories. Key design parameters, such as maximum take-off weight, number of passengers, range, wing area, span, fuselage length, to name a few, are likely to be available in the preliminary design stage and significantly impact the aircraft design. These variables either directly or indirectly affect the overall production cost or performance. The PCA approach includes both correlation and multiple linear regression techniques. The empirical models thus developed were able to predict the aircraft cost with an error of less than ±4% for all aircraft categories considered. Two aircraft in each defined category were not part of the PCA models and were used to verify the models. The proposed models provide the ability to estimate the aircraft cost quickly in the early stages of the preliminary design phase and provide the possibility of performing parametric studies involving the key variables to determine the cost sensitivity to the main design parameters.

Preliminary design on high-end workstation cooling system using loop heat pipes

In this article, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec-POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the local information aggregation multiagent deep deterministic policy gradient (LIA-MADDPG) algorithm, which merges centralized training with distributed execution. During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various centralized training and decentralized execution (CTDE)-based multiagent reinforcement learning (MARL) methods, significantly enhancing their performance. Additionally, by comparing LIA-MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA-MADDPG's outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.

In this article, a topology optimization method is proposed to obtain a new preliminary shape design of the twin-web turbine disc considering the burst speed constraint. The burst speed constraint is converted to the constraints on the averaged hoop stress and the maximum averaged radial stress. The linear density filter and Heaviside projection are employed to prevent the mesh dependency and chequerboard pattern and ensure distinct solid/void interfaces in the optimization result. The geometric extraction is easily implemented using the least square fitting with B-spline method. Through illustrative examples, the importance of considering the burst speed constraint in the design of turbine disc is verified, which contributes towards obtaining a safer design at the preliminary design stage. The effectiveness of the proposed method in obtaining a lighter preliminary twin-web turbine disc design is also demonstrated compared with the parametric optimization method.

The design process of new air- and rotorcraft is commonly separated into three different consecutive phases. In the conceptual design phase, the viability of different designs is investigated with respect to customer requirements and/or the market situation. It usually ends with the identification of a basic aircraft lay-out. In the subsequent preliminary design stage the various disciplines are introduced, thus redefining the design process as a multidisciplinary optimization (MDO) task. The objective of this design stage is to enhance the initial aircraft configuration by establishing an advanced design comprising a loft provided with primary structure. This updated aircraft configuration represents a global optimum solution for the specified requirements which will then be optimized on a local level in the concluding detailed design phase with particular regard to manufacturing aspects. The investigations in the preliminary design phase comprise the generation of numerous similar but still different analytical and finite element (FE) models. Even though computational power is constantly increasing the model generation process is still a time-consuming task. Moreover, it is also a potential source of errors which — in the worst case — may lead to time- and cost-intensive redesign activities during the detailed design. As the preliminary design stage, therefore, is of particular importance during the overall design process the model generation process benefits from parametric models and automated process chains. The presented paper overviews the tools used for the automated generation of FE models developed and used at the Institute of Structures and Design (BT) of the German Aerospace Center (DLR) for the subsequent use in numerical simulations. Furthermore, basic requirements for the effective use of parametrization and automation like a common data format and infrastructure will be introduced. Exemplary models and applications will be presented to illustrate the positive impact on efficiency in aircraft design. Concluding, future development steps and possible applications will be discussed.

A procedure for evaluation of multiple attributes in the preliminary design stage is necessary to the development of a more formal theory and methodology of design. In this paper we prescribe, demonstrate, and compare two such techniques: fuzzy set analysis and multiattribute utility analysis. We analyze the problem of preliminary material selection for an automotive bumper beam to illustrate the application of both analytical procedures. Preliminary assessment of incommensurate performance measures of weight, cost, shape restriction, stiffness, and corrosion are made for steel and polymer composite alternatives. Fuzzy analysis readily accommodates design evaluation based on semantic assessment of relative attribute levels, although it can also employ information which is entirely numerical. In contrast, utility analysis requires considerable numerical quantification. For the example studied, the ordinal ranking of alternatives was the same for both methods. Recommendations include using fuzzy analysis in the earliest stages of preliminary design evaluation or in situations limited to semantic input from design decision makers. Utility analysis may be used in later stages of preliminary design, where numerical quantification of attribute levels is possible and when decision makers are able to assess the individual attribute utility functions in the standard way. The results of utility analysis can be used to quantify acceptable design tradeoffs between attribute levels and can be used in later stages of the iterative design process.

A multi-robot system (MRS) is a group of coordinated robots designed to cooperate with each other and accomplish given tasks. Due to the uncertainties in operating environments, the system may encounter emergencies, such as unobserved obstacles, moving vehicles, and extreme weather. Animal groups such as bee colonies initiate collective emergency reaction behaviors such as bypassing obstacles and avoiding predators, similar to muscle conditioned reflex which organizes local muscles to avoid hazards in the first response without delaying passage through the brain. Inspired by this, we develop a similar collective conditioned reflex mechanism for multi-robot systems to respond to emergencies. In this study, Collective Conditioned Reflex (CCR), a bio-inspired emergency reaction mechanism, is developed based on animal collective behavior analysis and multi-agent reinforcement learning (MARL). The algorithm uses a physical model to determine if the robots are experiencing an emergency; then, rewards for robots involved in the emergency are augmented with corresponding heuristic rewards, which evaluate emergency magnitudes and consequences and decide local robots' participation. CCR is validated on three typical emergency scenarios: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">turbulence, strong wind, and hidden obstacle</i> . Simulation results demonstrate that CCR improves robot teams' emergency reaction capability with faster reaction speed and safer trajectory adjustment compared with baseline methods.

This paper proposes an integrated design and optimization approach for radial inflow turbines consisting of an automated preliminary design module and a flexible three-dimensional multidisciplinary optimization module. The latter was constructed by an evolution algorithm, a genetic algorithm-assisted self-learning artificial neural network and a dynamic sampling database. The 3-D multidisciplinary optimization approach was validated by the original T-100 turbine and the T-100re turbine obtained from the automated preliminary design approach, for maximizing the total-to-static efficiency and minimizing the rotor weight while keeping the mass flow rate constant and stress limitation satisfied. The validation results indicate that the total-to-static efficiency is 89.6%, increased by 1.3%, and the rotor weight is reduced by 0.14 kg (14.6%) based on the T-100re turbine, while the efficiency is 88.2%, increased by 2.2% and the weight is reduced by 0.49 kg (37.4%) based on the original T-100 turbine. Moreover, the T-100re turbine shows better performance at the preliminary design stage and conserves this advantage to the end, though both the aerodynamic performance of the T-100 and the T-100re turbine are improved after 3-D optimization. At the same time, it is implied that the preliminary design plays an essential role in the radial inflow turbine design process, and it is hard for only 3-D optimization to get a further performance improvement.

A design of DSS for mass production machining systemsIn this paper, we present a decision support tool (DSS) for preliminary design of transfer machines with rotary or mobile tables. In these transfer machines, the machining operations are executed on working positions equipped by standard multi-spindle heads. A part is sequentially machined onmworking positions and is moved from one position to the next using a rotary or a mobile table. The operations are grouped into blocks, where the operations of the same block are simultaneously performed by one multi-spindle head. At the preliminary design stage, the goal is to select the number of working positions and to decide which spindle heads will be installed minimizing the machine cost while respecting a given production rate. The paper presents the overall approach and depicts mathematical and decision-support methods developed and implemented in a software for the optimization of preliminary design (or reconfiguration) of such machining systems.

The increasing complexity of autonomous drone swarm operations, particularly in adversarial air combat scenarios, presents significant challenges in Multi-Agent Reinforcement Learning (MARL). Existing approaches often face issues such as poor coordination, suboptimal convergence, and overestimation bias,undermining their applicability to high-stakes environments. We propose the Sequential Decision Distribution Soft Actor-Critic (SD2AC), a novel off-policy MARL framework for drone swarm coordination to address these challenges.SD2AC introduces three key innovations:(1) a sequential decision-making framework in which agents adaptively condition their decisions on the policies of preceding agents, enhancing coordination and reducing conflicts; (2) a distributional Q-value critic that models the full return distribution to mitigate overestimation bias and improve policy robustness; and (3) an adaptive twin value distribution learning mechanism that leverages dual critics to dynamically select conservative value estimates, ensuring stable learning under uncertainty.Rigorous evaluation in diverse environments, including Close Air Combat, MECA, and MAMuJoCo, demonstrates the superiority of SD2AC over state-of-the-art MARL methods in terms of reward optimization, convergence speed, and coordination efficiency. Ablation studies further validate the individual contributions of each component.

Correct evaluation of rudder performance is a key issue in assessing ship maneuverability. This paper presents a simplified approach based on a viscous flow solver to address propeller and rudder interactions. Viscous flow solvers have been applied to this type of problems, but the large computational requests limit (or even prevent) their application at a preliminary ship design stage. Based on this idea, a simplified approach to include the propeller effect in front of the rudder is considered to speed up the solution. Based on the concept of body forces, this approach enables sufficiently fast computation for a preliminary ship design stage, thereby maintaining its reliability. To define the limitations of the proposed procedure, an extensive analysis of the simplified method is performed and the results are compared with experimental data presented in the literature. Initially, the reported results show the capability of the body-force approach to represent the inflow field to the rudder without the full description of the propeller, also with regard to the complex bollard pull condition. Consequently, the rudder forces are satisfactorily predicted at least with regard to the lift force. However, the drag force evaluation is more problematic and causes higher discrepancies. Nevertheless, these discrepancies may be accepted due to their lower influence on the overall ship maneuverability performance.

Inherent safety index module (ISIM) to assess inherent safety level during preliminary design stage

Leader-follower navigation is a popular class of multi-robot algorithms where a leader robot leads the follower robots in a team. The leader has specialized capabilities or mission-critical information (e.g. goal location) that the followers lack, and this makes the leader crucial for the mission's success. However, this also makes the leader a vulnerability - an external adversary who wishes to sabotage the robot team's mission can simply harm the leader and the whole robot team's mission would be compromised. Since robot motion generated by traditional leader-follower navigation algorithms can reveal the identity of the leader, we propose a defense mechanism of hiding the leader's identity by ensuring the leader moves in a way that behaviorally camouflages it with the followers, making it difficult for an adversary to identify the leader. To achieve this, we combine Multi-Agent Reinforcement Learning, Graph Neural Networks and adversarial training. Our approach enables the multi-robot team to optimize the primary task performance with leader motion similar to follower motion, behaviorally camouflaging it with the followers. Our algorithm outperforms existing work that tries to hide the leader's identity in a multi-robot team by tuning traditional leader-follower control parameters with Classical Genetic Algorithms. We also evaluated human performance in inferring the leader's identity and found that humans had lower accuracy when the robot team used our proposed navigation algorithm.

Border surveillance presents unique challenges requiring persistent monitoring, rapid response, and adaptive coordination across large, dynamic environments. Drone swarms, with their scalability and flexibility, offer a promising solution, but effective deployment demands intelligent decision-making under uncertainty. This paper proposes a multi-agent reinforcement learning (MARL) framework for drone swarm border surveillance, where each drone acts as an autonomous agent capable of cooperative sensing, patrolling, and threat detection. The framework leverages decentralized policies with shared global objectives, enabling drones to balance exploration and exploitation while adapting to adversarial intrusions and environmental variability. A reward structure is designed to encourage coverage efficiency, minimize energy consumption, and maximize detection accuracy. Simulation experiments demonstrate that the proposed MARL approach outperforms baseline strategies in terms of surveillance coverage, resilience to agent loss, and adaptability to dynamic border conditions. The results highlight the potential of MARL-driven drone swarms as a scalable, intelligent, and robust solution for next-generation border security operations.