Automation in operator assistant systems

Edge computing (EC) extends the ability of cloud computing to the network edge to support diverse resource-sensitive and performance-sensitive IoT applications. However, due to the limited capacity of edge servers (ESs) and the dynamic computing requirements, the system needs to dynamically update the task allocation policy according to real-time system states. Service migration is essential to ensure service continuity when implementing dynamic task allocation. Therefore, this article investigates the long-term dynamic task allocation and service migration (DTASM) problem in edge-cloud IoT systems where users’ computing requirements and mobility change over time. The DTASM problem is formulated to achieve the long-term performance of minimizing the load forwarded to the cloud while fulfilling the seamless migration constraint and the latency constraint at each time of implementing the DTASM decision. First, the DTASM problem is divided into two subproblems: 1) the user selection problem on each ES and 2) the system task allocation problem. Then, the DTASM problem is formulated as a Markov decision process (MDP) and an approach based on deep reinforcement learning (DRL) is proposed. To tackle the challenge of vast discrete action spaces for DTASM task allocation in the system with a mass of IoT users, a training architecture based on the twin-delayed deep deterministic policy gradient (DDPG) is employed. Meanwhile, each action is divided into a differentiable action for policy training and one mapped action for implementation in the IoT system. Simulation results demonstrate that the proposed DRL-based approach obtains the long-term optimal system performance compared to other benchmarks while satisfying seamless service migration.

In this article, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec-POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the local information aggregation multiagent deep deterministic policy gradient (LIA-MADDPG) algorithm, which merges centralized training with distributed execution. During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various centralized training and decentralized execution (CTDE)-based multiagent reinforcement learning (MARL) methods, significantly enhancing their performance. Additionally, by comparing LIA-MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA-MADDPG's outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.

Dynamic task allocation of unmanned aerial vehicle swarms for ground targets is an important part of unmanned aerial vehicle (UAV) swarms task planning and the key technology to improve autonomy. The realization of dynamic task allocation in UAV swarms for ground targets is very difficult because of the large uncertainty of swarms, the target and environment state, and the high real-time allocation requirements. Hence, dynamic task allocation of UAV swarms oriented to ground targets has become a key and difficult problem in the field of mission planning. In this work, a dynamic task allocation method for UAV swarms oriented to ground targets is comprehensively and systematically summarized from two aspects: the establishment of an allocation model and the solution of the allocation model. First, the basic concept and trigger scenario are introduced. Second, the research status and the advantages and disadvantages of the two allocation models are analyzed. Third, the research status and the advantages and disadvantages of several common dynamic task allocation algorithms, such as the algorithm based on market mechanisms, intelligent optimization algorithm, and clustering algorithm, are evaluated. Finally, the specific problems of the current UAV swarm dynamic task allocation method for ground targets are highlighted, and future research directions are established. This work offers important reference significance for fully understanding the current situation of UAV swarm dynamic task allocation technology.

With the application of automation technology, Human operators rely on automation functions or intelligent agents to conduct complex cognitive tasks, but this also leads to a series of human factor risks such as out-of-the-loop and first failure problems. An appropriate level of automation (LOA) design will help to alleviate the above human factors risks and improve the performance of human-machine cooperation, but the traditional LOA taxonomy is difficult to directly guide the human-machine function allocation of operational supervisory tasks. To characterize current LOA design practices, a literature review was conducted to review the LOA taxonomy of supervision and control tasks in related fields. This research summarizes the taxonomy dimensions of LOA. The intelligent operational supervisory task requires the operators to maintain a high degree of interaction and cooperation with the automation system. Therefore, we must shift the focus of LOA design to cognitive interaction tasks and takeover tasks. This research analyzes the characteristics of different dimensions of LOA taxonomy in the literature, and summarizes the LOA granularity of system task, cognitive interaction task and takeover task. On this basis, from the perspective of human-machine interaction, the LOA taxonomy of intelligent operational supervisory tasks is proposed. This research provides an important theoretical basis for human-machine function allocation scheme and system LOA design, and has important theoretical and practical significance for improving the human-machine interaction efficiency.

This paper studies a task allocation problem in which a team of UAVs collaboratively tracking multiple moving targets with different priorities. In this mission, task allocation is required to decide which UAVs will be assigned to track which targets at each time instant. A dynamic task allocation framework is proposed which integrates a distributed election algorithm and a centralized target assignment algorithm. First, an election algorithm is designed to determine in a distributed manner a leader UAV to make task allocation decisions by which guarantees a new leader will be regenerated immediately when suffering leader failures. A fast online target assignment algorithm is proposed which adopts a relaxation technique to balance efficiency and optimality. The effectiveness of the proposed algorithms is verified by simulation experimental results.

When it comes to Multi-Robot Systems (MRS), Multi-Robot System Task Allocation (MRTA) is one of the key issues in realizing multi-robot systems for real-world applications. In this paper, we propose an optimization-based Hungarian algorithm to solve the task allocation problem of multi-robot systems. Firstly, each robot is allowed to autonomously evaluate the task cost and send it to the central processor and an equilibrium compensation approach is proposed to compensate the constraints of the algorithm for the cost matrix to be square. Then a task cache pool is used to avoid the problem from degenerating into Single-Robot (SR) task allocation, and a prioritization strategy considering time is also designed to prevent the task points with lower generation values from lingering for a long time in the dynamic task allocation. Through simulation experiments, we demonstrate that the algorithm has a positive effect on the efficiency of assigned tasks in both dynamic and static situations.

Potential game for dynamic task allocation in multi-agent system

This paper discusses the use of cognitive models as augmented metacognition on task allocation for tasks requiring visual attention. In the domain of naval warfare, the complex and dynamic nature of the environment makes that one has to deal with a large number of tasks in parallel. Therefore, humans are often supported by software agents that take over part of these tasks. However, a problem is how to determine an appropriate allocation of tasks. Due to the rapidly changing environment, such a work division cannot be fixed beforehand: dynamic task allocation at runtime is needed. Unfortunately, in alarming situations the human does not have the time for this coordination. Therefore system-triggered dynamic task allocation is desirable. The paper discusses the possibilities of such a system for tasks requiring visual attention.KeywordsVisual attentioncognitive modelingaugmented metacognition

In accordance with the allocation of random arriving tasks, queuing network is proposed and then utilized to establish combat model for air-to-ground attack of formation. With the methodology of Markov Decision Processes (MDP), the paper establishes the dynamic task assignment model under the complete information condition. Allowing for the issue of dynamic task allocation with incomplete information, load threshold and value function sharing mechanism are proposed. Afterwards, two layers Q-learning algorithm is presented in this context to establish the dynamic task allocation model, which aims to achieve the cooperation with in formation and between formations. The simulation result indicates that the model aforementioned can enhance the long-term profit value, as well as avoid omitting the target with bigger income. Furthermore, it sheds light on the study on the dynamic task allocation and combat effectiveness evaluation for air-to-ground attack of formation problem in theory and method.

In collective robotics, researchers are successfully using models derived from swarm insect behavior to solve problems like coordination or task allocation. It is often assumed that in a homogeneous group of agents, every agent has to become more complicated when the complexity of the task increases, which decreases simplicity of design and robustness. Role diversification and therefore task specialization within a group may help counter the need for more complex agents. Because experiments on self-organization and dynamic task allocation in robot populations focused mainly on homogeneous groups, the relation between these models and pre-specified role diversification remains mainly unknown. In this work the interchangeability of homogeneous and heterogeneous agent populations is investigated. It is shown that in a simple simulated environment, a mixed population of specialized agents can not be easily substituted by a homogeneous group of multi-tasking agents. Results lead to the conclusion that the ability of dynamic task switching, i.e. adaptive task allocation in respect to changes in the environment, have strong effects on the behavior on a population level. Although a pre-defined heterogeneous group can produce the same result for a given environment and a specific population composition, the group behavior differs when the environment changes

Use of neural networks to achieve dynamic task allocation: a flexible manufacturing system example

To address the challenges of real-time decision-making and resource optimization in multi-agent cooperative interception tasks within dynamic environments, this paper proposes a hierarchical framework for reinforcement learning-based interception algorithm (HFRL-IA). By constructing a hierarchical Markov decision process (MDP) model based on dynamic game equilibrium theory, the complex interception task is decomposed into two hierarchically optimized stages: dynamic task allocation and distributed path planning. At the high level, a sequence-to-sequence reinforcement learning approach is employed to achieve dynamic bipartite graph matching, leveraging a graph neural network encoder–decoder architecture to handle dynamically expanding threat targets. At the low level, an improved prioritized experience replay multi-agent deep deterministic policy gradient algorithm (PER-MADDPG) is designed, integrating curriculum learning and prioritized experience replay mechanisms to effectively enhance the interception success rate against complex maneuvering targets. Extensive simulations in diverse scenarios and comparisons with conventional task assignment strategies demonstrate the superiority of the proposed algorithm. Taking a typical scenario of 10 agents intercepting as an example, the HFRL-IA algorithm achieves a 22.51% increase in training rewards compared to the traditional end-to-end MADDPG algorithm, and the interception success rate is improved by 26.37%. This study provides a new methodological framework for distributed cooperative decision-making in dynamic adversarial environments, with significant application potential in areas such as maritime multi-agent security defense and marine environment monitoring.

The collected information related to the activities along the life cycle of an automation solution serves as basis for the comparative evaluation between a classically developed system and a mechatronicly developed decentralized system. The following results have been worked out with the characteristics from decentralized automation use cases as introduced in Section 3 in combination with the described mechatronic engineering approaches. The general advantages of engineering with mechatronic modules, such as, for instance, cross-discipline- work and the improved reuse or standardization of partial solutions, were already discussed in other papers and shall not be described here any further (Thramboulidis 2008, Löwen & Wagner 2009, VDI 2206). Application Effort and Complexity: The applicability and manageability of a modular, decentralized automation approach ­ particularly regarding the modularization of automation and control functionalities ­ must still be ensured. For this purpose, one must differentiate between the creation of the mechatronically modules in a predevelopment project and their application in the particular engineering projects. Overall, it can be expected that the specification and implementation effort to be performed in advance and the complexity for the creation of mechatronic modules with integrated automation and control functions, which are autonomous and can be integrated independently of each other, will increase substantially. For this purpose, bases for modules have to be created that are usable for different projects and configurable for the specific applications. This requires detailed knowledge of the domain, extensive technological expertise, and an understanding of the core functionality of the modules. One example for a starting point for this approach is the INTEGRA standard specifying first consistent mechatronic viewpoints for automotive production lines. These pre-produced modules facilitate the engineering for project-specific applications. The more plug&play modules are available the less effort is required for the engineering. In addition, project risks can be reduced by early validation of fulfillment of customer requirements due to availability of module descriptions and models. This, however, only applies if module adaptations occur which can be done by configuration, selection of variants, or extensions, and in which do not require any changes to the predefined core

In this article we review and assess human-centered level of automation (LOA), an alternate approach to traditional, technology-centered design of automation in dynamic-control systems. The objective of human-controlled LOA is to improve human-machine performance by taking into account both operator and technological capabilities. Automation literature has shown that traditional automation can lead to problems in operator situation awareness (SA) due to the out-of-the (control) loop performance problem, which may lead to a negative impact on overall systems performance. Herein we address a standing paucity of research into LOA to deal with these problems. Various schemes of generic control system function allocations were developed to establish a LOA taxonomy. The functions allocated to a human operator, a computer, or both, included monitoring system variables, generating process plans, selecting an “optimal” plan and implementing the plan. Five different function allocation schemes, or LOAs, were empirically investigated as to their usefulness for enhancing telerobot system performance and operator SA, as well as reducing workload. Human participants participated in experimental trials involving a high fidelity, interactive simulation of a telerobot performing nuclear materials handling at the various LOAs. Automation failures were attributed to various simulated system deficiencies necessitating operator detection and correction to return to functioning at an automated mode. Operator performance at each LOA, and during the failure periods, was evaluated. Operator SA was measured using the Situation Awareness Global Assessment Technique, and perceived workload was measured using the NASA-Task Load Index. Results demonstrated improvements in human-machine system performance at higher LOAs (levels involving greater computer control of system functions) along with lower operator subjective workload. However, under the same conditions, operator SA was reduced for certain types of system problems and reaction time to, and performance during, automation failures was substantially lower. Performance during automation failure was best when participants had been functioning at lower, intermediate LOAs (levels involving greater human control of system functions). © 2000 John Wiley & Sons, Inc.

This paper presents a task allocation-oriented framework to enable efficient in-network processing and cost-effective multi-hop resource sharing for dynamic multi-hop multimedia wireless sensor networks with low node mobility, e.g., pedestrian speeds. The proposed system incorporates a fast task reallocation algorithm to quickly recover from possible network service disruptions, such as node or link failures. An evolutional self-learning mechanism based on a genetic algorithm continuously adapts the system parameters in order to meet the desired application delay requirements, while also achieving a sufficiently long network lifetime. Since the algorithm runtime incurs considerable time delay while updating task assignments, we introduce an adaptive window size to limit the delay periods and ensure an up-to-date solution based on node mobility patterns and device processing capabilities. To the best of our knowledge, this is the first study that yields multi-objective task allocation in a mobile multi-hop wireless environment under dynamic conditions. Simulations are performed in various settings, and the results show considerable performance improvement in extending network lifetime compared to heuristic mechanisms. Furthermore, the proposed framework provides noticeable reduction in the frequency of missing application deadlines.