Air combat game and simulation

Discrete event dynamic systems (DEDS), are systems, which change state over time, based on current state and state transition rules. Each state is separated from its neighbour by a step rather than a continuum of infinitesimal states. This continuum in DEDS needs a representation between 0 and 1. The application of fuzzy logic in defining and modelling the DEDS is suggested as a possible solution because the nature of DEDS may be asynchronous, distributed, parallel, non-deterministic and/or stochastic. This results in the generalisation of crisp finite automata into fuzzy automata for a DEDS. In this paper, a DEDS in the form of hopper feed system is considered. The system is reformulated using state vectors and event transition matrices and is extended to fuzzy vectors and matrices. An algorithm is proposed for the final control action to be implemented using a programmable logic controller (PLC).

In a typical game scenario, the attention in unmanned aerial vehicle (UAV) air combat should be focused on both sides' maneuvering decision strategies. However, most existing studies focus only on improving their own maneuvering decisions, ignoring the importance of the opponent's strategy in the two-player game. This paper proposes a reinforcement learning (RL)-based air combat decision method considering the opponent's maneuvering strategy. Through the proposed limited imitation offline RL (LIORL) method, the opponent's air combat decision method is refined using existing air combat data. Based on the enemy's superior strategy, an air combat simulation environment is created, and the RL method is applied to train the agent for UAV air combat maneuvering decision strategies. No current research utilizes static datasets for the acquisition of adversary strategies. This study, in contrast, offers a more reliable approach in comparison to methodologies predicated on assuming adversary strategies. Ablation experiments have been meticulously executed to showcase the capability of the LIORL algorithm in identifying the optimal enemy strategy while exerting a lesser impact on the dataset than incumbent algorithms. The employment of the LIORL algorithm enables the agent to refine the adversary's strategy, resulting in an augmented win rate against our UAV. Through air combat simulations, we empirically validate the agent's decision-making process based on the adversary's strategy, thereby affirming the efficacy of the proposed methodology.

In this paper, air combat simulation data is reconstructed into a dynamic Bayesian network. It gives a compact probabilistic model that describes the progress of air combat and allows efficient computing for study of different courses of the combat. This capability is used in what-if type analysis that investigates the effect of different air combat situations on the air combat evolution and outcome. The utilization of the dynamic Bayesian network is illustrated by analyzing simulation results produced with a discrete event air combat simulation model called X-Brawler.

This paper presents a new game-theoretic approach toward the validation of discrete-event air combat (AC) simulation models and simulation-based optimization. In this approach, statistical techniques are applied for estimating games based on data produced by a simulation model. The estimation procedure is presented in cases involving games with both discrete and continuous decision variables. The validity of the simulation model is assessed by comparing the properties of the estimated games to actual practices in AC. These games are also applied for simulation-based optimization in a two-sided setting in which the action of the opponent is taken into account. In optimization, the estimated games enable the study of effectiveness of AC tactics as well as aircraft, weapons, and avionics configurations. The game-theoretic approach enhances existing methods for the validation of discrete-event simulation models and techniques for simulation-based optimization by incorporating the inherent game setting of AC into the analysis. It also provides a novel game-theoretic perspective to simulation metamodeling which is used to facilitate simulation analysis. The utilization of the game-theoretic approach is illustrated by analyzing simulation data obtained with an existing AC simulation model.

ABSTRACTIn this paper, the static output feedback stabilisation of discrete event dynamic systems (DEDSs) is investigated via the semi-tensor product (STP) of matrices. Firstly, the dynamics of DEDSs modelled by deterministic Moore-type finite automata are converted into an matrix expression in the STP frame. Secondly, necessary and sufficient conditions for the existence of a static output feedback control pattern, stabilising the controlled discrete event dynamic systems to some equilibrium point, are given, and constructive algorithms to seek the static output feedback control pattern including an effective specific solution algorithm and an analytic solution algorithm are proposed. Thirdly, the equilibrium-based static output feedback stabilisation of DEDSs is extended to the set-based static output feedback stabilisation and necessary and sufficient conditions and constructive algorithms to seek the corresponding control pattern are provided. Finally, some examples are presented to illustrate the effectiveness of the proposed approach.

Reinforcement learning (RL) has recently proven itself as a powerful instrument for solving complex problems and even surpassed human performance in several challenging applications. This signifies that RL algorithms can be used in the autonomous air combat problem, which has been studied for many years. The complexity of air combat arises from aggressive close-range maneuvers and agile enemy behaviors. In addition to these complexities, there may be uncertainties in real-life scenarios due to sensor errors, which prevent estimation of the actual position of the enemy. In this case, autonomous aircraft should be successful even in the noisy environments. In this study, we developed an air combat simulation, which provides noisy observations to the agents, therefore, make the air combat problem even more challenging. Thus, we present a state stacking method for noisy RL environments as a noise reduction technique. In our extensive set of experiments, the proposed method significantly outperforms the baseline algorithms in terms of the winning ratio, where the performance improvement is even more pronounced in the high noise levels. In addition, we incorporate a self-play scheme to our training process by periodically updating the enemy with a frozen copy of the training agent. By this way, the training agent performs air combat simulations to an enemy with smarter strategies, which improves the performance and robustness of the agents. In our simulations, we demonstrate that the self-play scheme provides important performance gains compared to the classical RL training.

Air combat decision-making is a process in which the both sides choose certain strategies to maximize the probability of winning according to the current air combat situation. Imitation learning is a way for an agent to make intelligent decisions by imitating expert examples. In this paper, aircraft confrontational strategies are trained by behavior repetition, which is a kind of imitative learning. Aiming at the aircraft model with three degrees of freedom, seven basic maneuvers were designed as alternative strategies by taking 1V1 and 2V2 air combat confrontations as research objects. The victory samples were obtained by Monte Carlo simulation, and state-action pairs were extracted from the victory samples. On this basis, neural network training was carried out, and the model that could be used for air combat decision-making was obtained under the condition of 1V1. In 2V2 condition, we focus on finding the winning strategy through the cooperation between agents. The validity of the decision model is proved by simulation.KeywordsImitation learningAir combat simulationCooperative strategy

Algebraic Characteristics and Structure Decomposition for “Linear” Discrete Event Dynamic Systems

Discrete event dynamic systems (DEDS) are prevalent in many areas of engineering, like manufacturing, maintenance, distributed computing, etc. Activities or discrete events in DEDS are generally large in numbers, highly parallel, distributed, and highly intertwined. Modeling DEDS systems are further complicated by the fact that the decisions to be made from the model may incorporate inference engines that uses advanced statistical analysis on massive data and may also include fuzzy decisions. This paper talks about development of a new tool for modeling and simulation of DEDS that can satisfy the characteristics mentioned above; the new tool is called General purpose Petri net Simulator (GPenSIM). This paper further presents a recent extension to the new tool, which is the real-time simulation capability.

ALGEBRAIC CHARACTERISTICS AND STRUCTURE DECOMPOSITION FOR “LINEAR” DISCRETE EVENT DYNAMIC SYSTEMS

An approximate dynamic programming approach was developed to generate intelligence maneuvering decisions for CGF (Computer Generated Forces) during one-on-one air combat simulation. Simplified kinematic equations of aircraft were built up in flight-path coordinate system, the 3 DOF (degree of freedom) control command was improved, and the common basic flight maneuvering set was described quantitatively. Then, a new method of air combat maneuvering decision was designed according to actual battlefield situation, using trajectory sampling, feature extraction, policy learning. With this method, CGF can learn an approximately optimal policy of air-combat maneuvering during real-time simulation. Result of simulation confirms that proposed method can improve the intelligent decision ability of CGF during simulated air combat training.

<p indent="0mm">An unmanned aerial vehicle (UAV) broadly refers to all kinds of remotely controlled aerial vehicles that do not require pilots to board and drive. Due to its small size, low cost, large quantity, and safety, a UAV is widely used in modern air combat. However, current studies on UAV air combat algorithms are mostly conducted in highly simplified scenarios with low precision. In addition, most of the methods used in these studies are limited by existing expert knowledge and cannot fully exploit the advantages of intelligent air combat algorithms. Therefore, this study investigates a multi-UAV cooperative swarm algorithm in air combat based on the predictive game tree. First, Unity3D is used to build a simulation environment close to a real air combat scene. In addition, a human-machine interaction environment, including UI, VR, weather system, and multifunctional screen, is realized. Then, a set of tactical maneuvers is encapsulated on the basis of existing air combat knowledge. Scripted aircraft formations are realized, and a set of air combat evaluation functions is designed. On this basis, an air combat artificial intelligence framework based on the predictive game tree is proposed. This algorithm completes the task of role assignment and maneuver decision-making, and XGBoost is used to transform it into an online real-time algorithm. Using the state machine algorithm as the baseline, the effectiveness of our air combat algorithm is verified through air combat confrontation experiments on a high-precision simulation platform.

Discrete Event Dynamic System (DEDS) is a dynamic system in which unexpected events take place at irregular intervals. Such systems can describe many different real phenomena occurring in transport networks, technological processes and computer systems and also in examples of design systems presented in this article. This article describes methods to support managing the design of IT infrastructure as a DEDS dealing with the allocation of human resources, the synchronisation of tasks and risk management. These methods are based on the max-plus system modelling theory , while the example uses a schedule of managing the design, implementation and deployment of software which helps monitor and control complex technological processes. The proposed solution can be used in both standard and agile & lean management principles promoted in recent years.

Situation assessment for air combat based on novel semi-supervised naive Bayes

Rule-based Composable Modelling of Industrial Automation Automata under Nominal and Faulty Conditions