Can Intelligent Agent Improve Human-Machine Team Performance Under Cyberattacks?

Unmanned aerial vehicle (UAV) is an aircraft without a human pilot onboard and a sort of unmanned vehicle. UAV Network research activities in the real world can pose several challenges, especially in urban environments or in densely populated environments. This is due to the nature of the UAV which requires a large area of operation. Therefore, digital simulation is vital in conducting research related to UAV and its networks. One of the UAV simulators recently developed is FlyNetSim. FlyNetSim is an open-source UAV Network Simulator based on NS-3 and Ardupilot. In the application, UAV can have some network-related problems. One of them is when a UAV is in an environment that has a lot of interfering networks because it is contending a network channel. This can cause network delay. These delays will raise with the increase in the number of nodes that interfere with the UAV. To overcome the problem of interfering nodes, we propose to add a priority set module for UAV and Ground Control Station (GCS). The proposed module will set priority for the UAV and GCS socket. This module is intended to give priority to packets sent by UAVs or GCS. Later the packet that has priority will be sent first to reduce the delay caused by the appearance of interfering nodes. Setting priority in the socket may reduce the average delay caused by contending nodes. Test result shows that there is a 16.2% reduction of average delay. Test result also shows that there is a 4% raise in packet delivery ratio.

: Unmanned aerial vehicles (UAVs) are critical components of the future naval forces. UAV control and monitoring with autonomous operation will become an absolute necessity and adaptive cooperation of vehicles is the only practical alternative. The objective of this project is to develop and evaluate new methodologies for cooperative (formation) control of multiple unmanned air vehicles. The goal is to have multiple UAVs working together as a group. Instead of separately assigning distinct tasks to each vehicle, the operator would assign tasks to the UAV group, which then determines the best way to accomplish each task, freeing the operator to maintain surveillance over the entire operation. In this project we investigated Path Tracking and obstacle avoidance of UAVs using fuzzy logic method. Algorithms for close formation control of multi-UAVs are developed and simulated. We also investigated fault-tolerant control of single UAVs by neuro-adaptive method. Detailed description of this method is provided in this document. The project has supported S graduate students with 9 technical papers published.

This paper introduces a virtual reality (VR)-based unmanned aerial vehicle (UAV) simulator designed to support immersive training and early-stage human factors evaluations in rescue and emergency mission contexts. The system simulates a leader-follower UAV configuration, where a human operator controls a lead drone through a VR headset and joystick, while autonomous drones maintain any geometric formations using onboard sensing and dynamic obstacle avoidance [1]. This swarm-based coordination reflects real-world search and rescue scenarios, where rapid decision-making, spatial awareness, and teamwork are essential.The simulator is developed with a focus on human-in-the-loop design, providing an immersive teleoperation experience that places the user in high-pressure environments with dynamic spatial constraints. Visual and auditory feedback, along with the ability to switch between multiple drone perspectives, is intended to support situational awareness, mission control, and error recovery in complex terrain.From a human factors perspective, the simulator serves as a flexible testbed for evaluating cognitive and ergonomic variables in safety-critical tasks. It allows the assessment of operator workload, interface usability, attention allocation, and situation awareness under realistic but controlled conditions. Subjective evaluation tools such as the NASA Task Load Index (NASA-TLX) and the Situation Awareness Rating Technique (SART) [2] can be embedded directly within the VR experience. At the same time, objective data from head and hand movement, control inputs, and UAV performance are recorded to gain insight into operator behavior.The simulator integrates high-fidelity UAV dynamics using closed-loop reference model adaptive controllers [3], and can be equipped with tools such as eye tracking, physiological sensors for workload estimation, and visual attention assessment methods like ATTENDO [4]. These additions support deeper analysis of how operators acquire information, prioritize tasks, and shift attention during mission-critical events.Overall, the VR-based UAV simulator addresses a growing need to evaluate and enhance human performance in multi-agent control systems used in emergency operations. It offers a scalable, immersive environment that supports both training and cognitive engineering. Its modular structure allows for future integration of real UAVs, digital mapping capabilities, and adaptive interfaces, making it a valuable platform for advancing human-centered UAV system design.

Hybrid modeling and control issues provide a flexible powerful tool for Unmanned Aerial Vehicles (UAVs) as complex control platforms. This paper aims to briefly review recent results in this area. Firstly, existing works on hybrid modeling and control of a single UAV are reviewed. Then multiple UAVs control in a synchronously cooperative task planning, which gives much more maneuverability, has been surveyed in the hybrid framework. Formation control of UAVs is a typical cooperative strategy with considerable applications in multi-agent aerial robotics area and can leverage limited abilities of single UAVs into complicated tasks of group missions. There are a few works in this field which we briefly reviewed their results and explored the possible future directions of these research activities.

Autonomous formation flight has been a fast growing research area during the last decade. Unmanned aerial vehicles (UAVs) can achieve a wide range of benefits from formation flight, such as fuel saving and aerial refueling. In this paper, the development of a small UAV system, called Swallow UAV system, as testbed of formation flight, is presented. Swallow UAV system includes Swallow UAV itself and a Hardware-In-the-Loop Simulation Environment (HILSE). Swallow UAV is a fixed wing, pusher-type, electric motor driven aircraft with 1.4 meter wing span. Its avionics system includes an onboard computer and a sensor system with GPS, IMU, air data sensor, and motor monitoring unit to provide position, velocity, inertial data, and system health. Communications between UAVs and the ground control station (GCS) are conducted by wireless modem. A fuzzy navigation and control system (FNCS) is developed for the waypoint navigation and attitude control of the aircraft. GPS velocity based pseudo attitude and fuzzy logic control are applied to FNCS for attitude estimation and fast implementation of controllers. HILSE is built to verify FNCS with onboard computer on the ground. X-Plane flight simulator is applied to simulate aircraft dynamics and output sensor data. GCS software, built with data distributor, transmits these data to the onboard computer and receive control signal from it so the onboard computer can control the aircraft in X-Plane. HILSE can perform pre-flight test for software debug, which improves safety and efficiency of flight test. The design of Swallow UAV is verified by flight test and the result shows that it has sufficient flight performance with waypoint navigation and attitude control capability to be used as a formation flight testbed.

As a complicated multi-input, multi-output, and time-varying nonlinear system, flight control system of unmanned aerial vehicle (UAV), which determines the whole system’s performance directly, is crucial for the simulation training system design. This chapter mainly focuses on parameter identification of flight control system based on the modeling of UAVs and a specific controller design for the pendulum-like oscillation in micro aerial vehicle (MAV). A predator–prey particle swarm optimization (PSO) algorithm for identifying parameters of UAV flight control system is presented, with the aim of reducing the workload of the designers during the process of designing complicated UAV control system. Besides, a software environment for UAV controller design was developed based on the UAV model and the proposed method. Then a specific kind of controller design involving the pendulum-like oscillation in the hover and stare state for a MAV in the presence of external disturbances is discussed in detail, since the pendulum-like oscillation caused by uncertainty and external disturbances badly jeopardize the performance of hover and stare, resulting in blurred images or even MAV’s overturning. A novel type of PSO-based linear-quadratic regulator (LQR) controller for stabilizing the pendulum-like oscillation is developed, which can enhance the MAV’s performance efficiently.

Unmanned vehicles (UAVs) are technological tools whose application is carried out in the civil and military areas for surveillance, recognition and intelligence tasks. Given its versatility and features, in this paper the implementation of an autopilot is determined, after an analysis of commercial models. Said device must comply with technical requirements for the automatic flight control of a medium-sized UAV, considering the operation scenarios in the northern border of Ecuador. In addition, the parameters that intervene in the control loops and the relevant simulations are specified. The results obtained from the experimental flights of the aircraft are reflected in graphs, where the stability of the plant properly calibrated for autonomous flight is verified.

Since one GRS used for Control and Non-Payload Communication (CNPC) of Unmanned Aerial Vehicle (UAV) cannot support all operational ranges of an UAV, the handover technology of CNPC is essential to operate an UAV in a wide range. However, the CNPC physical layer standard of Radio Technical Commission for Aeronautics (RTCA) does not have a reference signal for measuring the signal strength of a neighboring base station used for handover decision in an existing mobile communication system. Therefore, in this paper, we propose a method to transmit measurement signals for stable handover of CNPC. The proposed method starts transmission of the measurement signal when the UAV enters the handover zone for the preventing waste of power and generates the measurement signal using the same data that is transmitted through the currently connected Ground Radio Station (GRS) for the preventing packet loss during the handover operation.

Solar energy is one of the most strategic energy sources for the world’s economic development. This has caused the number of solar photovoltaic plants to increase around the world; consequently, they are installed in places where their access and manual inspection are arduous and risky tasks. Recently, the inspection of photovoltaic plants has been conducted with the use of unmanned aerial vehicles (UAV). Although the inspection with UAVs can be completed with a drone operator, where the UAV flight path is purely manual or utilizes a previously generated flight path through a ground control station (GCS). However, the path generated in the GCS has many restrictions that the operator must supply. Due to these restrictions, we present a novel way to develop a flight path automatically with coverage path planning (CPP) methods. Using a DL server to segment the region of interest (RoI) within each of the predefined PV plant images, three CPP methods were also considered and their performances were assessed with metrics. The UAV energy consumption performance in each of the CPP methods was assessed using two different UAVs and standard metrics. Six experiments were performed by varying the CPP width, and the consumption metrics were recorded in each experiment. According to the results, the most effective and efficient methods are the exact cellular decomposition boustrophedon and grid-based wavefront coverage, depending on the CPP width and the area of the PV plant. Finally, a relationship was established between the size of the photovoltaic plant area and the best UAV to perform the inspection with the appropriate CPP width. This could be an important result for low-cost inspection with UAVs, without high-resolution cameras on the UAV board, and in small plants.

Information acquisition and rescue operation management can be facilitated by the use of unmanned aerial vehicles (UAVs) equipped with cameras and sensors that are capable of aerial photography and acquire important information from disaster-stricken areas. UAV technology is a relatively low-cost, flexible, and safe solution in missions that are difficult for humans or other systems. UAV usage in pre-and post-disaster assessment, response, and management is an active area of research. There are two important technical aspects of UAV applications in disaster situations. The first is the limited communication range. Information acquired and photography produced by UAVs are only transmitted in real-time to the ground control station (GCS) within its communication range. Remote real-time monitoring of distant areas beyond the UAV's communication range is not possible using a single UAV. It is only possible through communication among several UAVs. The second is the relatively short flight time due to limited battery capacity. UAV replacement is required in order to carry out a persistent monitoring mission. In this paper, we propose a new remote real-time monitoring system of a distant disaster area using multiple UAVs. Wireless communications between multiple UAVs constitutes a UAV network in our proposed system. Moreover, the proposed system replaces the UAVs in-flight with standby UAVs according to the remaining energy levels. We compare the performance of the proposed system with an existing UAV system. Our simulation results show that the proposed system can achieve efficient remote real-time monitoring.

Contents: Foreword Part 1 Characteristics of Complex Decision Making: Decisions about 'what' and decisions about 'how', Erik Hollnagel Corruption and recovery of sensemaking during navigation, Gary Klein Time and design in decision making environments, GA ran Pettersson Risk and decision making, Damien J. Williams Extreme risk-taking and decision making, Bruno Sicard, Elisabeth Jouve, and Olivier Blin. Part 2 Areas of Application: Human requirements in automated weapons systems, Jennifer McGovern Narkevicius and Peggy Heffner Automation and decision making, Jan Noyes The flightdeck of the future: field studies in datalink and freeflight, Gemma Cox, Sarah Sharples, Alex Stedmon, John Wilson and Tracey Milne The flightdeck of the future: perceived urgency of speech and text, Alex Stedmon, Sarah Sharples, Edward Nicholson, Gemma Cox and John Wilson Operator interface research testbed for supervisory control of multiple unmanned aerial vehicles (UAVs), Gloria Calhoun, Mark Draper and Heath Ruff Virtual environments for military decision making, John Kostaras and Georgias Detsis ROLF 2010: A Swedish command post of the future, Berndt Brehmer. Part 3 Complex Decision Making in Civil Applications: Human information processing aspects of effective emergency incident management decision making, Jim McLennan, Mary Omodei, Alina Holgate and Alexander Wearing Decision making by operational incident commanders in a nuclear emergency response organisation: decision strategy selection, Margaret Crichton, Peter McGeorge and Rhona Flin Presentation of verbal material: the impact of modality on situation awareness and performance on the flightdeck, Alastair Nicholls, Tracey Milne, Eric Farmer and Anne Melia Air traffic controller strategies in holding scenarios, Lynn Springall How roles change when disaster strikes: lessons learnt from the manufacturing domain, Carys Siemieniuch and Murray Sinclair. Part 4 Complex Decision Making in Military Applications: A history lesson on the use of technology to support military decision making and command and control, Robert Bolia, Michael Vidulich, Todd Nelson and Malcolm Cook The intuitive vs. analytic approach to real world problem solving: misperception of dynamics in military operations, BjA,rn Tallak Bakken, Stig Johannessen, Dag SA,berg and Morten Ruud Critical thinking in tactical decision games-training, Karel van den Bosch and Anne Helsdingen. Part 5 Teams and Complex Decision Making: Why training team decision making is not as easy as you think: guiding principles and needs, Eduardo Salas, Joseph Guthrie and Shawn Burke The migration of authority in tactical decision making, Sidney Dekker, and Nalini Suparamaniam The analysis of team decision making architectures, Richard Breton and Robert Rousseau Surgical team self-review: enhancing organisational learning in the Royal Cornwall Hospital Trust, Simon Henderson, Matt Mills, Adrian Hobbs, Alan Bleakley, James Boyden and Linda Walsh. Part 6 Assessment and Measurement: A method for need analysis before decision making based on ecological psychology, Thierry Morineau Operational net assessment: a Canadian human factors analysis, Philip Farrell Psycho-physiological measures of situation awareness, Han Tin French, Elizabeth Clarke, Diane Pomeroy, Melanie Seymour and C.Richard Clark Signal detection theory and the assessment of situation awareness, Barry McGuinness Psycho-physiological measurements of mental activity, stress reactions and situation awareness in the maritime full mission simulator, Thomas Koester Measures of attention and cognitive effort in tactical decision making, Sandra Marshall Crew mental workload for the vetronics technology testbed vehicle, Christopher Smyth Full spectrum analysis: practical operational research in the face of the human variable, Graham Mathieson Effective taxonomies in organisational safety, Brendan Wallace and Alastair Ross Using signal detection theory to measure situation awareness: the technique, the tool (QUASAa ), the test, the way forward, Graham Edgar and Helen Edgar. Part 7 A Final Comment: Intelligence, uncertainty, interpretations and prediction failure, Malcolm Cook, Corinne Adams and Carol Angus Index.

This paper presents a fine-grained decomposition of situation awareness (SA) as it pertains to the use of unmanned aerial vehicles (UAVs), and uses this decomposition to understand the types of SA attained by operators of the Desert Hawk UAV. Since UAVs are airborne robots, we adapt a definition previously developed for human-robot awareness after learning about the SA needs of operators through observations and interviews. We describe the applicability of UAV-related SA for people in three roles: UAV operators, air traffic controllers, and pilots of manned aircraft in the vicinity of UAVs. Using our decomposition, UAV interaction designers can specify SA needs and analysts can evaluate a UAV interface's SA support with greater precision and specificity than can be attained using other SA definitions.

In this paper, cyberattack detection and isolation is studied for a network of formation-flying unmanned aerial vehicles. As the unmanned aerial vehicles communicate to reach consensus on their states while making the formation, the communication network among the unmanned aerial vehicles makes them vulnerable to a potential attack from malicious adversaries. Two types of attacks pertinent to a network of unmanned aerial vehicles have been considered: a node attack on the unmanned aerial vehicles and a deception attack on the communication between the unmanned aerial vehicles. Unmanned aerial vehicle formation control involves using a consensus algorithm to maintain a prespecified formation. Two kinds of cyberattacks in the unmanned aerial vehicle network (node and communication path deception) are considered with their respective models in the formation setup. A bank of unknown input observer-based distributed fault detection schemes as well as with a rule based on residual generated using the bank of unknown input observers are proposed to detect the attacks and to identify the compromised unmanned aerial vehicle in the formation. Furthermore, an algorithm is developed to remove the faulty unmanned aerial vehicle from the network and to keep the compromised unmanned aerial vehicle isolated once an attack is detected, while maintaining the flight formation with a missing unmanned aerial vehicle node.

Traditional bulky and complex control devices such as remote control and ground station cannot meet the requirement of fast and flexible control of unmanned aerial vehicles (UAVs) in complex environments. Therefore, a data glove based on multi-sensor fusion is designed in this paper. In order to achieve the goal of gesture control of UAVs, the method can accurately recognize various gestures and convert them into corresponding UAV control commands. First, the wireless data glove fuses flexible fiber optic sensors and inertial sensors to construct a gesture dataset. Then, the trained neural network model is deployed to the STM32 microcontroller-based data glove for real-time gesture recognition, in which the convolutional neural network-Attention mechanism (CNN-Attention) network is used for static gesture recognition, and the convolutional neural network-bidirectional long and short-term memory (CNN-Bi-LSTM) network is used for dynamic gesture recognition. Finally, the gestures are converted into control commands and sent to the vehicle terminal to control the UAV. Through the UAV simulation test on the simulation platform, the average recognition accuracy of 32 static gestures reaches 99.7%, and the average recognition accuracy of 13 dynamic gestures reaches 99.9%, which indicates that the system’s gesture recognition effect is perfect. The task test in the scene constructed in the real environment shows that the UAV can respond to the gestures quickly, and the method proposed in this paper can realize the real-time stable control of the UAV on the terminal side.

Ensuring UAV safety: A deep learning-based fault-tolerant approach with control barrier functions