Stochastic game approach to air operations

Observing the progress in the technology of unmanned combat aerial vehicles (UCAVs) than it can be foreseen that in the future the role of manned combat aircraft will be taken over more and more by unmanned systems. The abandonment of pilots allows for more freedom in the aerodynamic design of the vehicle in regard to weight and acceleration, however, new stealth constraints have a severe impact. The design of UCAV configurations is driven by the special requirements of upcoming missions, as for instance the capabilities of long endurance fl
\nights joined with a low observability. The new demands have a crucial impact on the aerodynamic shape, and, hence, require new solutions for maneuver con-
\ntrol in respect to integration of engine in- and outlets, actuators and other devices. Additionally the new capabilities of long endurance 
\nights has to be joined with low observability. In particular UCAVs are suited to the exploitation of non-conventional control technologies, such as aerodynamic morphing, 
\nflow control, or thrust vectoring. This paper presents a numerical investigation of innovative morphing technologies for aircraft applications and explores the feasibility for such technologies to enhance the maneuverability of unmanned combat aerial vehicles. In Aerodynamics the morphing is understood as a smooth, continuous change in geometry of the outer surface, e.g. the twist of complete wing can be changed in order control role moments. In the considered case morphing is used to provide additional 
\now control mechanisms taking into account the constraint of fllow radar signature. In particular morphing can be used to change the geometry of the airfoil leading edge to control the 
\now around the wings in order to generate additional lift or induce role moments. This paper is concerned with the investigation of the feasibility and effectiveness of morphing devices for the aerodynamic control of UCAVs. Focal point of this work is the morphing of leading edges in order to generate additional lift or role moments for UCAVs 
\nflying in high angle of attack mode. Therefore, the objective of the numerical investigations is the evaluation of the potential of morphing leading edges especially for the enhancement of the maneuverability of Delta- and Lambda-wing configurations with their vortex-dominated 
\now field. Of special interest is 
\now control at high angles of attack (AoA) by targeting an minimized shape adaptation in order to generate a needed additional lift, to induce role moments or to reduce the risk of the potential deleterious effects of vortex breakdown. The morphing of the leading edge must not confused with classical Kruger leading edge 
\nflaps. Although very effective in regard to the increase of lift such fl
\naps can't be used for UCAVs since gaps between fl
\naps and the wing have to be avoided in order to fulll the hard conditions of radar camouflage
\n.
\nThe presented work is part of the DLR Project FaUSST which is the successor of the DLRProject UCAV-2010. The UCAV-2010 project was set up to identify and assess UCAV relevant technologies. The investigations have covered the pre-design process with fast,
\nlow fidelity methods as well as the detailed examination using high fidelity methods like URANS simulations. The verification of the promissing technologies and tools have been done by virtual and experimental prototypes. The combined research has lead to a Lambda wing type UCAV configuration with a medium sweep of the leading edge. The outcoming
\nDLR-F17 conguration has been mainly derived from the so called Saccon geometry developed by EADS-MAS for the RTO/AVT-161 task group.
\nSince the capability of medium to high AoA maneuverability will be investigated the angle of attack has been varied from 6 to 16 degrees.
\nAfter presentation of the technical part, the results of leading edge manipulation are discussed. One focal point is the comparison of the impact of differently morphed leading edge shapes on the fl
\night performance at specific 
\nflight situations. The aerodynamical coefficients, in particular Cl, Cd and Cm, are related to morphing parameters of the changed shape of the leading edges. Another key aspect is the varied effective angle of attack of
\nthe middle part of the wings, see Figure 2. In case of a basic high angle of attack the effective AoA (due to morphing) can change the vortical 
\nflow above the deformed wings enormously, even when the geometrical angle of the airfoil has been morphed only slightly.
\nThis work discusses the effectiveness of counter-rotating morphing of leading edges, since the requirements in regard to radar signature do not allow large geometry changes. This point reduces the degree of freedom of any shape morphing drastically. Thereto, only proper combinations of left and associated right wing deformations have been analyzed and discussed. After the discussion of the most interesting deformations an evaluation of the presented test cases in regard to the maneuverability enhancement of UCAVs concludes the presentation.

Unmanned combat air vehicles (UCAV) are finding increasing use in military applications due to their autonomy, processing capabilities and low cost with respect to manned flights. In order to use the UCAVs in the air to air combat, required algorithms must be carefully designed to enhance them with aerial combat competence. There are challanges for the design of this kind of an algorithm such as real-time operability and the case with inadequate number of sensors on the UAVs. In this paper a novel attack algorithm is introduced for UCAVs, with the assumption that in order to launch an effective attack the pursuer aircraft must be able to follow the target aircraft from behind with a predetermined distance. The attack algorithm consists of navigation and reference velocity calculation steps. In navigation part the reference values for bank, heading and flight path angles are computed. In velocity generation part, a PID based method and a fuzzy inference based method are employed to compute the reference velocity of the pursuer and the results obtained with these two approaches are compared. The performance of this algorithm is assessed with simulations performed in three different scenarios with different initial conditions for the pursuer and the target. Simulation results show that the effectiveness of the proposed fuzzy inference based air to air attack algorithm is better in action with respect to PID based algorithm since various effects that cause the change in velocity, can be incorporated into the fuzzy inference mechanism.

With the development of unmanned combat air vehicles (UCAVs) and artificial intelligence (AI), within visual range (WVR) air combat confrontations utilizing intelligent UCAVs are expected to be widely used in future air combats. As controlling highly dynamic and uncertain WVR air combats from the ground stations of the UCAV is not feasible, it is necessary to develop an algorithm that can generate highly intelligent air combat strategies in order to enable UCAV to independently complete air combat missions. In this paper, a 1-vs.-1 WVR air combat strategy generation algorithm is proposed using the multi-agent deep deterministic policy gradient (MADDPG). A 1-vs.-1 WVR air combat is modeled as a two-player zero-sum Markov game (ZSMG). A method for predicting the position of the target is introduced into the model in order to enable the UCAV to predict the target’s actions and position. Moreover, to ensure that the UCAV is not limited by the constraints of the basic fighter maneuver (BFM) library, the action space is considered to be a continuous one. At the same time, a potential-based reward shaping method is proposed in order to improve the efficiency of the air combat strategy generation algorithm. Finally, the efficiency of the air combat strategy generation algorithm and the intelligence level of the resulting strategy is verified through simulation experiments. The results show that an air combat strategy using target position prediction is superior to the one that does not use target position prediction.

Weapon configuration, allocation and route planning with time windows for multiple unmanned combat air vehicles

The future of air combat relies on humans controlling large teams of unmanned combat air vehicles (UCAVs) within a dynamic battle environment. Under the DARPA Mixed Initiative Control of Automata (MICA) program, we have been challenged to design a system that empowers a human operator to control teams of up to thirty UCAVs. To address these challenges we are designing an interaction system that defines and provides adequate situation and automation awareness without overloading human operators to the point where their performance degrades gracelessly. The proposed mixed initiative system is situated within a complex and highly dynamic information space that could easily overload the multi-tasking human operators. Dozens of system parameters could be updated thousands of times during a typical mission so it is neither feasible nor prudent for human operators to maintain complete situation and automation awareness. This interaction system will provide appropriately abstracted situation awareness and notification capability that includes: general mission monitoring and automation awareness; task specific information requirements; and user initiated information requests. Our approach involves defining adequate situation awareness as a function of mission phase, human operator role, and abstracted information required to oversee tasked UCAV.

Unmanned combat air vehicle (UCAV) combat is a challenging scenario with high-dimensional continuous state and action space and highly nonlinear dynamics. In this article, we propose a general hierarchical framework to resolve the within-vision-range (WVR) air-to-air combat problem under six dimensions of degree (6-DOF) dynamics. The core idea is to divide the whole decision-making process into two loops and use reinforcement learning (RL) to solve them separately. The outer loop uses a combat policy to decide the macro command according to the current combat situation. Then the inner loop uses a control policy to answer the macro command by calculating the actual input signals for the aircraft. We design the Markov decision-making process for the control policy and the Markov game between two aircraft. We present a two-stage training mechanism. For the control policy, we design an effective reward function to accurately track various macro behaviors. For the combat policy, we present a fictitious self-play mechanism to improve the combat performance by combating against the historical combat policies. Experiment results show that the control policy can achieve better tracking performance than conventional methods. The fictitious self-play mechanism can learn competitive combat policy, which can achieve high winning rates against conventional methods.

Conceptual aircraft design is characterised by the requirement to analyse a large number of configurations rapidly and cost effectively. For unusual configurations such as those typified by unmanned combat air vehicles (UCAVs) adequately predicting their aerodynamic characteristics through existing empirical methods is fraught with uncertainty. By utilising rapid and low cost experimental tools such as the water tunnel and subscale flight testing it is proposed that the required aerodynamic characteristics can rapidly be acquired with sufficient fidelity for the conceptual design phase. Furthermore, the initial design predictions can to some extent be validated using flight-derived aerodynamic data from subscale flight testing.

The application of a new class of actuators is considered. The actuators under development combine a high energy density smart material, specifically a piezoelectric material, with internal servohydraulic components. Large displacement outputs are produced, while the high force capacity of the stiff smart material is retained, for a net high-energy output. The actuator is considered “power-by-wire” because only electrical power is provided from the vehicle or system controller. A primary motivating application is in unmanned combat air vehicles (UCAVs). The particular actuation needs of these vehicles, in flight control and other utility functions, are described and distilled to a set of relevant device requirements. Other potential applications, such as flight motion simulation, are also highlighted. The new actuation architecture offers specific advantages over centralized hydraulic systems and has capabilities not present in electromechanical actuators (EMAs). The main advantage over centralized hydraulic systems is the elimination of the need for hydraulic lines. Compared to motor-driven ball screw type EMAs, the new actuators offer higher frequency response, and a larger peak-to-average output. A laboratory test facility designed to represent the loading experienced by a UCAV control surface is described. Key steps necessary to flight qualify the actuator are introduced.

This paper introduces a novel prediction model designed to mitigate the substantial data dependency associated with maneuver trajectory prediction in unmanned combat air vehicles (UCAVs) during air combat. Considering the characteristics of high noise, dynamic complexity, and variable data lengths inherent in short‐range air combat scenarios, we employ dynamic time warping (DTW) to assess the similarity of 3D trajectory data. This approach allows us to identify and select the most analogous historical data, which we then utilize as our training dataset. In pursuit of enhanced precision for online trajectory prediction, we propose an improved convolutional neural network (CNN) that not only offers “after‐zero” information but also incorporates delay compensation mechanisms. Our experimental findings indicate that the proposed prediction model not only satisfies the stringent timeliness requirements but also outperforms benchmark models in terms of prediction accuracy across various operating conditions.

An important subproblem in the area of cooperative control of multiple, autonomous, unmanned air vehicles is the determination of the minimum-time optimal paths for the agents to fly from one destination to the next. The tasks for the air vehicles are usually tightly coupled in time, and hence estimates of the times taken for each air vehicle to fly from one destination to the next is highly critical for correct assignment of tasks. In this article, we discuss the existence and uniqueness of minimum time solutions for the trajectory planning problem for a Micro Air Vehicle (MAV) under wind conditions. We show that there exists a minimum time solution for the trajectory planning problem with a minimum turn radius constraint for the air vehicle, and for a non-zero, time-varying wind vector field satisfying certain easily checked sufficient conditions. We also prove uniqueness for almost every combination of initial and final conditions in the case of a wind vector field that can vary with time but is constant in the spatial variable at each time instant.

An air combat situation assessment method is developed for multiple unmanned combat air vehicles (UCAVs) based on interval data in this paper. Due to the complexity and uncertainty of the air combat environment, it is difficult to obtain the accurate data of the air combat. Thus, interval number is considered to represent the inaccurate data. An air combat situation index system based on interval numbers is established, which consists of angle advantage, velocity advantage, height advantage, distance advantage and performance advantage. Then, an interval number eigenvector method is developed to give the final situation assessment. An example is given to illustrate the air combat situation assessment method.

This paper presents and discusses the wind tunnel results obtained during Phase 1 and Phase 2 of the Smart Wing program. A series of four tests was conducted, two in each phase. The purpose of the tests was to evaluate the performance of smart material based control effectors in representative aerodynamic environments. In Phase 1, wing twist using a Shape Memory Alloy (SMA) torque tube and smoothly contoured trailing edge surfaces enabled by SMA wires were evaluated on a semi-span, 16% scale model of a typical fighter wing. The focus of Phase 2 was to address the design and demonstration of a high-frequency, large-deflection, and smoothly contoured trailing edge control surface capable of a spanwise variation in deflection. The actuation system for this control surface was based on piezoelectric (PZT) ultrasonic motors. A smoothly contoured leading edge control surface was also built and tested using SMA wires. These designs were evaluated on a 30%, full-span model of a representative Unmanned Combat Air Vehicle (UCAV). In each wind tunnel entry comparisons were made to the performance of conventional control surfaces. Successful results included: improved aileron effectiveness at high dynamic pressures; demonstrated improvements in lateral and longitudinal control effectiveness with smoothly contoured control surfaces over conventional hinged control surfaces; continuous spanwise shape control; and, large deflections at rates over 80°/s, which is well within the desired deflection aileron rates for control of fighter aircraft.

The paper addresses the application of a parametric design process for a flying wing configuration. The multi-disciplinary configuration (MULDICON) is a generic unmanned combat air vehicle (UCAV) developed for research purposes, a further development of the DLR-F19 configuration, which was used for research activities in the scope of the DLR project Mephisto and its predecessors FaUSST and UCAV2010. For the MULDICON, the DLR parametric design process MONA is applied. Special emphasis is placed on the structural modeling with composite material where each layer is modeled and analyzed. Various failure criteria are compared to define suitable constraints for the optimization of the load carrying structure. In contrast to optimize the thickness of composites with global allowable strains, such strategy allows for a detailed analysis of every layer. The number of constraints due to the set-up of every ply is substantially increased compared to the strain allowables but the structural optimization is still applicable. The detailed structural and mass models represent the global stiffness and structural dynamic characteristics of the aircraft. For the loads analysis part of the design process, 9 different mass configurations with a total of 306 maneuvering load cases as well as 336 1-cos gust load cases are taken into account. Furthermore, a new simplified landing impact simulation is introduced to consider 12 landing load cases. All load cases are defined according to regulations like CS-25. Such number of load cases is necessary to cover a sufficient number of flight conditions. For the selection of the design loads for the structural optimization, the essential loads are analyzed for a subset of locations. Together with a parametrized optimization model, the structural optimization is conducted. The result is a weight-optimized structural model for the MULDICON. This entire model allows for the investigation of physics-based effects already at an early stage of the design process.

In this chapter, preliminary sizing and performance calculations of unmanned air vehicles (UAV), with relation to its propulsion system, are explained. Starting with the mission profile of the air vehicle, which is one of the important design drivers, the UAV is sized for the requirements. The requirements in design consist of a set of target values (e.g., payload amount to be carried, range, endurance time, cost, etc.). Additionally, the parameters within mission profile such as cruising altitude and speed of aircraft affect engine type, power level required, fuel quantity, and therefore general dimensions and the gross weight of the aircraft. It is often an iterative process to size the air vehicle and engine together. UAV is designed in a loop of calculations in which sizing, flight performance, and engine performance are done for each phase of the mission or flight profile to satisfy the overall design mission and requirements.

Nowadays, Unmanned Air Vehicles (UAVs) have become more and more important. These vehicles are employed in many applications from military operations to civilian tasks. Under situations where global positioning system (GPS) and inertial navigation system (INS) do not function, or as an additional sensor, vision systems can be used. In recent years omnidirectional camera usage has experienced a remarkable increase. In many fields innovative research has been done by this camera which captures 360° view in a single frame. However, employment of omnidirectional cameras in UAVs is very new. In this paper, we present the use of catadioptric systems in UAVs. The parallel lines that exist on many structures in an urban environment are used by the algorithm for the estimation of the attitude of the UAV. After explanation of the algorithm, the UAV modelling and control will be presented. Various simulations have been done to present the effectiveness of the estimation algorithms as well as the UAV controllers. Finally, we will present the experiments and the results of the estimation and control algorithms on a real model helicopter.