Analyzing air combat simulation results with dynamic Bayesian networks

We propose a dynamic differential Bayesian networks (DDBNs) and nonparametric regression model. This model is an extended model of traditional dynamic Bayesian networks (DBNs), which can incorporate temporal information in a natural way and directly handle real-valued data obtained from microarrays without any transformation. In addition, it can cope with differential information between gene expression levels, without any loss to the traditional advantage, i.e., the capability of estimating non-linear relationships between genes. We apply DDBNs to analyze simulated data and real data, i.e., Saccharomyces cerevisiae cell cycle gene expression data. We have confirmed the effectiveness of our approach in the sense that some edges have been successfully detected only by DDBNs, not by DBNs.

In a typical game scenario, the attention in unmanned aerial vehicle (UAV) air combat should be focused on both sides' maneuvering decision strategies. However, most existing studies focus only on improving their own maneuvering decisions, ignoring the importance of the opponent's strategy in the two-player game. This paper proposes a reinforcement learning (RL)-based air combat decision method considering the opponent's maneuvering strategy. Through the proposed limited imitation offline RL (LIORL) method, the opponent's air combat decision method is refined using existing air combat data. Based on the enemy's superior strategy, an air combat simulation environment is created, and the RL method is applied to train the agent for UAV air combat maneuvering decision strategies. No current research utilizes static datasets for the acquisition of adversary strategies. This study, in contrast, offers a more reliable approach in comparison to methodologies predicated on assuming adversary strategies. Ablation experiments have been meticulously executed to showcase the capability of the LIORL algorithm in identifying the optimal enemy strategy while exerting a lesser impact on the dataset than incumbent algorithms. The employment of the LIORL algorithm enables the agent to refine the adversary's strategy, resulting in an augmented win rate against our UAV. Through air combat simulations, we empirically validate the agent's decision-making process based on the adversary's strategy, thereby affirming the efficacy of the proposed methodology.

This paper presents a new game-theoretic approach toward the validation of discrete-event air combat (AC) simulation models and simulation-based optimization. In this approach, statistical techniques are applied for estimating games based on data produced by a simulation model. The estimation procedure is presented in cases involving games with both discrete and continuous decision variables. The validity of the simulation model is assessed by comparing the properties of the estimated games to actual practices in AC. These games are also applied for simulation-based optimization in a two-sided setting in which the action of the opponent is taken into account. In optimization, the estimated games enable the study of effectiveness of AC tactics as well as aircraft, weapons, and avionics configurations. The game-theoretic approach enhances existing methods for the validation of discrete-event simulation models and techniques for simulation-based optimization by incorporating the inherent game setting of AC into the analysis. It also provides a novel game-theoretic perspective to simulation metamodeling which is used to facilitate simulation analysis. The utilization of the game-theoretic approach is illustrated by analyzing simulation data obtained with an existing AC simulation model.

Cooperative maneuver decision making for multi-UAV air combat based on incomplete information dynamic game

The paper presents a new game theoretic approach towards the validation of discrete event air combat simulation models. In the approach, statistical techniques are applied for estimating game models based on simulation data. The estimation procedure is presented in cases involving games with both discrete and continuous decision variables. The validity of the simulation model is assessed by comparing the properties of the estimated games to actual practices in air combat. The approach enhances existing methods for the validation of discrete event simulation models by incorporating the inherent game setting of air combat into the analysis. The estimated games also provide a novel game theoretic perspective to simulation metamodeling which is used to facilitate simulation analysis. The utilization of the game theoretic approach is illustrated by analyzing simulation data produced with an existing air combat simulation model.

Reinforcement learning (RL) has recently proven itself as a powerful instrument for solving complex problems and even surpassed human performance in several challenging applications. This signifies that RL algorithms can be used in the autonomous air combat problem, which has been studied for many years. The complexity of air combat arises from aggressive close-range maneuvers and agile enemy behaviors. In addition to these complexities, there may be uncertainties in real-life scenarios due to sensor errors, which prevent estimation of the actual position of the enemy. In this case, autonomous aircraft should be successful even in the noisy environments. In this study, we developed an air combat simulation, which provides noisy observations to the agents, therefore, make the air combat problem even more challenging. Thus, we present a state stacking method for noisy RL environments as a noise reduction technique. In our extensive set of experiments, the proposed method significantly outperforms the baseline algorithms in terms of the winning ratio, where the performance improvement is even more pronounced in the high noise levels. In addition, we incorporate a self-play scheme to our training process by periodically updating the enemy with a frozen copy of the training agent. By this way, the training agent performs air combat simulations to an enemy with smarter strategies, which improves the performance and robustness of the agents. In our simulations, we demonstrate that the self-play scheme provides important performance gains compared to the classical RL training.

The air combat process is regarded as the discrete event dynamic system which is composed of a series of discrete-states, and its discrete state sub-model is established. In order to choose the tactical maneuver, the evaluation function is constructed by the air combat situation index and the air combat capability index. Combining with the 3 degree-of-freedom (3DOF) aircraft model, the maneuver command generator is designed, controlling the moving state of aircraft. Finally, the simulation of the combat-turning is carried out using Matlab.

An approximate dynamic programming approach was developed to generate intelligence maneuvering decisions for CGF (Computer Generated Forces) during one-on-one air combat simulation. Simplified kinematic equations of aircraft were built up in flight-path coordinate system, the 3 DOF (degree of freedom) control command was improved, and the common basic flight maneuvering set was described quantitatively. Then, a new method of air combat maneuvering decision was designed according to actual battlefield situation, using trajectory sampling, feature extraction, policy learning. With this method, CGF can learn an approximately optimal policy of air-combat maneuvering during real-time simulation. Result of simulation confirms that proposed method can improve the intelligent decision ability of CGF during simulated air combat training.

With breakthroughs in modern weaponry and equipment technology and coordinated formation flying, coordinated operations beyond visual range have become the main form of modern air combat. Accurately identifying enemy maneuvers and formation tactics is of great significance for grasping the dominance of the battlefield and grasping the direction of war. Traditional situation assessment research lacks pertinence to coordinated formation tactics and strategies in the BVR battlefield. This paper considers the influence of the characteristics of BVR fighters on tactical strategies, analyzes the causal relationship between the characteristics of flight parameters and the evolution of tactical movements, and constructs a dynamic Recognition model of typical maneuver movements beyond the visual range based on Bayesian network; on this basis, a recognition model of cooperative formation tactics is constructed. Finally, simulation experiments verify the accuracy and effectiveness of the model’s recognition. Further, this model can be used as a threat module of the beyond-horizon cooperative formation tactics for the Air combat situation assessment.KeywordsCoordinated operations of beyond visual rangeManeuver recognitionTactical identificationBayesian networkCloud model

In order to find out the problems of students' learning in the online learning process in time and ensure that students can successfully complete the online learning course, the dynamic Bayesian network is used to predict students' online learning scores. First, the online course Bayesian network is established, and then the dynamic Bayesian network is established by combining the Bayesian network and hidden Markov according to the time slice. The maximum expectation algorithm is used for Bayesian parameter learning, and the scores are predicted according to video learning, chapter test and other influencing factors. The results show that the dynamic Bayesian network can effectively predict the online learning results, the prediction results provide teachers with decision-making, and students can carry out targeted reinforcement learning according to the prediction results in the learning process.

Nowadays, several components and systems are designed and produced based on reliability. Since the reliability criterion has an important role in purchasing and implementation of these systems. In the design of a reliable system, fault and failure analysis must be carried out in order to reduce fault probability of the system. When dependency and the relation between components of a complex system are important and should be mentioned, determination of system reliability is very difficult. In this paper, dynamic fault tree is used to evaluate the systems reliability that their behavior is varied with time. Dynamic fault tree is constructed and then it converted to dynamic bayesian network. In this paper, the principle of dynamic fault tree gates and their mapping into dynamic bayesian are explained and some new relations between events and gates for this mapping are proposed. GeNIe package is used to determine dynamic bayesian network based on stochastic sampling algorithms. Four systems (cardiac assist system, hypothetical cascaded priority-and system, inertial navigation system/ global positioning system integrated, and emergency detection system) are investigated; reliability and fault probability of these systems are calculated. Comparison of the results with those obtained by other researches shows the proposed method effectiveness for systems reliability modeling and assessment via dynamic bayesian network.

We simulate and analyze pilot decision making in one-on-one air combat by using an ine uence diagram. Unlike most of the existing approaches, an ine uence diagram graphically describes the factors of a decision process and explicitly handles the decision maker’ s preferences under conditions of uncertainty. In the pilot decision model, the possible combat situations related to each maneuver alternative are associated with a probability and a utility. Ine uence diagram analysis produces a probability distribution of the overall utility that represents the successfulness of a maneuver and gives information to make rational maneuvering decisions. Sensitivity analysis determines the impacts of different factors on the outcome of the maneuvering decision. The effects of sensor information that will reduce the uncertainty of the model are evaluated with Bayesian reasoning. The model can be utilized in the analysis of a single decision situation or as an automated decision-making system that selects combat maneuvers in air combat simulators.

One of the major restrictions of dynamic Bayesian networks (DBNs) is their inability to account for topological features such as shape descriptors, homeomorphy, homotopy, and invariance. The main reason for this shortcoming is explained by the fact that even if dynamic Bayesian networks encode statistical relationships; they are not embedded in a Euclidean space where mathematical structures abound. The goal is to embed DBNs into a Euclidean space such that these topological features can be exploited. This extension of DBNs to topological DBNs (TDBNs) leapfrogs the task of pattern recognition and machine learning by not only classifying objects but revealing how they are related topologically. We have applied the TDBN formalism to facial aging for person identification. Preliminary results reveal that the TDBNs outperform the traditional DBN with an accuracy margin of 8% in average.

<div class="htmlview paragraph">Air combat simulator studies consistently show large fighter effectiveness payoffs due to thrust vectoring and/or thrust reversing exhaust nozzles. However, the air-to-air missile related kill ratios achieved during these evaluations vary considerably as a function of maximum missile launch angle-of-attack (AOA).</div> <div class="htmlview paragraph">This paper will review the ways thrust vectoring/reversing is used during air combat and then discuss some missile related multi-function nozzle survivability benefits. Typical air combat simulator fighter agility advantages and disadvantages attributable to vectoring/reversing nozzles will then be examined. These include relative antenna train (look) angles, capture times, and capture frequencies during close-in combat. The high AOA launch desirability will be defined along with the payoffs of attaining that capability. Nozzle max vector angles and utilization frequencies along with high AOA missile launch problems and potential solutions will also be addressed.</div> <div class="htmlview paragraph">In addition, the relationship between pilot training and missile launch AOA will be shown as it relates to kill ratio trends.</div>

The supply chain is the central organizing unit in today's global industries and has gained significance as one of the 21st century manufacturing paradigms for improving organizational competitiveness. In this paper, we propose a dynamic Bayesian network to represent the cause-and-effect relationships in an industrial supply chain. Based on the Quick Scan, a systematic data analysis and synthesis methodology, a dynamic Bayesian network is employed as a more descriptive mechanism to model the causal relationships in the supply chain. Dynamic Bayesian networks can be utilized as a knowledge base of the reasoning systems where the diagnostic tasks are conducted. We finally solve this reasoning problem with stochastic simulation.