Performance guaranteed human-robot collaboration with POMDP supervisory control

With the advances of Artificial Intelligence (AI) and autonomous robotics, robots are increasingly involved closely with human society, such as in manufacturing system, autonomous driving and intelligent service robots. While robots are good at handling repeated routine work, human are more adaptive and flexible to changing factors that may bring uncertainties and cost non-trivial efforts for robots to overcome. Thus efficient collaboration between human and robots can bring huge economic profits, but also require high standard for robot controller during task completion because the safety of human must be guaranteed. In this paper, we build a formal design theory enabling an automated human-robot collaboration with performance guarantees. Markov Decision Processes (MDPs) are used to model robots with uncertainties and Partially Observable Markov Decision Processes (POMDPs) are used to model human with hidden intents. Within a supervisory control framework, we propose to use L* learning algorithm to learn a correct supervisor such that global collaboration task given as Probabilistic Computation Tree Logic (PCTL) can be satisfied. An example from manufacturing system is discussed to further illustrate the design method.

The human-robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human's state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications.

Partially observable Markov decision processes (POMDPs) are widely used for planning under uncertainty. In many applications, the huge size of the POMDP state space makes straightforward optimization of plans (policies) computationally intractable. To solve this, we introduce an efficient POMDP planning algorithm. Many current methods store the policy partly through a set of “value vectors” which is updated at each iteration by planning one step further; the size of such vectors follows the size of the state space, making computation intractable for large POMDPs. We store the policy as a graph only, which allows tractable approximations in each policy update step: for a state space described by several variables, we approximate beliefs over future states with factorized forms, minimizing Kullback-Leibler divergence to the non-factorized distributions. Our other speedup approximations include bounding potential rewards. We demonstrate the advantage of our method in several reinforcement learning problems, compared to four previous methods.

During the last decades, collaborative robots capable of operating out of their cages are widely used in industry to assist humans in mundane and harsh manufacturing tasks. Although such robots are inherently safe by design, they are commonly accompanied by external sensors and other cyber-physical systems, to facilitate close cooperation with humans, which frequently render the collaborative ecosystem unsafe and prone to hazards. We introduce a method that capitalizes on partially observable Markov decision processes (POMDP) to amalgamate nominal actions of the system along with unsafe control actions posed by the System Theoretic Process Analysis (STPA). A decision-making mechanism that constantly prompts the system into a safer state is realized by providing situation awareness about the safety levels of the collaborative ecosystem by associating the system safety awareness with specific groups of selected actions. POMDP compensates the partial observability and uncertainty of the current state of the collaborative environment and creates safety screening policies that tend to make decisions that balance the system from unsafe to safe states in real time during the operational phase. The theoretical framework is assessed on a simulated human–robot collaborative scenario and proved capable of identifying loss and success scenarios.

Ambiguous partially observable Markov decision processes: Structural results and applications

Optimal cost almost-sure reachability in POMDPs

Partially Observable Markov Decision Process (POMDP) has been widely used in the robotics to model uncertainties from sensors, actuators and the environment. However, such comprehensiveness makes the planning in POMDP generally very difficult. Existing work often searches for an optimal control policy with respect to predefined reward functions, which may require a large memory and is computationally expensive. We propose to use formal methods and learn a Deterministic Finite Automaton (DFA) as a supervisor to regulate the behavior of a Partially Observable Markov Decision Process (POMDP), such that it satisfies the given specification in Probabilistic Computation Tree Logic (PCTL). For such a purpose, we modify the L* learning algorithm and define oracles for membership queries and conjectures. We further show that the termination and correctness of the design algorithm are guaranteed. A simple example is used for illustration.

Trust in autonomy is essential for effective human-robot collaboration and user adoption of autonomous systems such as robot assistants. This article introduces a computational model that integrates trust into robot decision making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human trust, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table clearing task in simulation (201 participants) and with a real robot (20 participants). In our studies, the robot builds human trust by manipulating low-risk objects first. Interestingly, the robot sometimes fails intentionally to modulate human trust and achieve the best team performance. These results show that the trust-POMDP calibrates trust to improve human-robot team performance over the long term. Further, they highlight that maximizing trust alone does not always lead to the best performance.

In this paper, an approach for computing online safety zones for collaborative robotics in a robust way, despite uncertain robot dynamics, is proposed. The strategy implements the speed and separation monitoring paradigm, and considers human and robot enclosed in bounding volumes. The human-robot collaboration is monitored by a supervisory controller that guides the robot to stop along a path-consistent trajectory in case of collision danger between human and robot. The size of the robot safety zone is minimized online according to the stop time of the manipulator, and the uncertain robot dynamics is considered using interval arithmetic to ensure compliance with the joint torques limits even in case of imperfect knowledge of the dynamic model parameters. The results verify the effectiveness of the proposed approach, and evaluate the influence of dynamics variations on human-robot collaboration.

With increased competitive challenges in the manufacturing sector, the need for operational excellence has led to an increased presence of robotics in factory settings. Traditionally, robotics in manufacturing has been relegated to routine and monotonous tasks performed in isolation to ensure human safety. Now, the advancements in robotics are encouraging a paradigm shift --- human-robot collaboration. These new robotic systems are imbued with the ability to 1) perceive anomalies in work environments and to correct/workaround these deficiencies, 2) adapt to changes in workload by means of reconfiguring their layout in a facility, and 3) autonomously navigate factory floors. Although we would like to believe that these innate abilities of second-generation robotics allow for immediate implementation of human-robot collaboration on factory floors, the truth is that more research is required to ensure safety, analyze performance, and define standards. This paper explores potential interaction objectives for human-robot communication in flexible robotic cells.

In human supervisory control, machines control the process according to directives given by human operators, where the humans have the final authority over the machines or automation. There are many human–machine systems in our society that are modeled nicely by the human supervisory control model. A glass-cockpit passenger aircraft is a typical example of such a system. However, even in transportation, human–automation relation for technologically advanced automobile may be quite different from that for a glass-cockpit aircraft. In automobile, it is only the automation that may be able to help the driver by backing up or replacing him/her to assure safety: Neither air traffic controller-like function nor copilot-like colleague is available there (Inagaki 2010). An advanced driver assistance system (ADAS) is a machine to assist a human to drive a car in a dynamic environment. Some functions of ADAS may include: (a) perception enhancement that helps the driver to perceive the traffic environment around his/her vehicle, (b) arousing attention of the driver to encourage paying attention to potential risks around his/her vehicle, (c) setting off a warning to urge the driver to take a specific action, and (d) automatic safety control that is activated when the driver takes no action even after being warned or when the driver’s control action seems to be insufficient. The first two functions, (a) and (b), are to help the driver to recognize or understand the situation. Understanding of the current situation determines what action needs to be done (Hollnagel and Bye 2000). Once situation diagnostic decision is made, action selection is usually straightforward (Klein 1993). However, the driver may not be successful in action selection decision. Function (c) is to help the driver in such a circumstance. Any ADAS that uses only the three functions, (a)–(c), is completely compatible with the human-centered automation principle (Billings 1997) in which the human is assumed to have the final authority over the automation. Suppose an ADAS contains the forth function, (d). Then, the ADAS may not be fully compatible with the human-centered automation, because the automation can implement a safety control action without any human intervention. Should we ban such an ADAS just because it can implement an action that is not ordered by the driver? It is well known that highly automated machines sometimes bring negative effects, such as the out-of-the-loop performance problem, loss of situation awareness, complacency or overtrust, automation surprises. However, humans have limited capabilities, and they might fail to understand the situation, select a right action and implement it appropriately, especially when available time and information are quite limited. Today’s machine can sense and analyze a situation, decide what must be done, and implement control actions. Cannot such a smart machine help the humans in more a positive manner as a teammate to the humans? The systems in which humans and technology collaborate together to achieve a common goal are called joint cognitive systems (Hollnagel and Woods 2005), in which human–automation coagency is central for realizing sensible human–automation partnership. This special issue of Cognition Technology and Work intends to discuss human–automation coagency for collaborative control of automobile. The first topic is authority and responsibility. The first three papers argue the need of situationand context-dependent sharing of authority between the human and the automation, without assuming T. Inagaki (&) Department of Risk Engineering, University of Tsukuba, Tsukuba 305-8573, Japan e-mail: inagaki@risk.tsukuba.ac.jp

Energy consumption in commercial buildings is significantly affected by the performance of heating, ventilation, and air-conditioning (HVAC) systems, which are traditionally operated using centralized controllers. HVAC control requires adjusting multiple setpoints such as chilled water temperatures and supply air temperature (SAT). Supervisory control framework in a distributed setting enables optimal HVAC operation and provides scalable solutions for optimizing energy across several scales from homes to regional areas. This paper proposes a distributed optimization framework for achieving energy efficiency in large-scale building energy systems. It is highly desirable to have building management systems that are scalable, robust, flexible, and are low cost. For addressing the scalability and flexibility, a modular problem formulation is established that decouples the distributed optimization level from local thermal zone modeling level. We leverage a recently developed generalized gossip algorithm for robust distributed optimization. The supervisory controller aims at minimizing the energy input considering occupant comfort. For validating the proposed scheme, a numerical case study based on a physical testbed in the Iowa Energy Center is presented. We show that the distributed optimization methodology outperforms the typical baseline strategy, which is a rule-based controller to set a constant supply air temperature. This paper also incorporates a software architecture based on the volttron platform, developed by the Pacific Northwest National Laboratory (PNNL), for practical implementation of the proposed framework via the BACnet system. The experimental results show that the supervisory control framework proposed in this paper can save energy by approximately 11%.

This paper designs a web-based Arduino supervisory control system. Arduino controller is utilized as a local controller to fit into a supervisory control framework. The framework includes a central server, a SMS device attached to the server, multiple local controllers, a remote control program and a ladder logic computer-aided design program. The Arduino controller contains an Arduino USB board and an Ethernet shield. The USB board provides basic input/output and processing power, while the Ethernet shield provides TCP/IP connection capability. The Arduino controller communicates with supervisory server with a specific m2m protocol which is based on http protocol. In this paper, the http protocol and m2m protocol is fulfilled based on TCP connection in an Arduino controller. Once an Arduino controller is plugged into the supervisory control system, it can be monitored and controlled remotely with any browser. The web-based Arduino supervisory control system is used to provide remote supervisory control service of water pumping systems of multistorey buildings scattered in big Taipei area by a small company.

Starting in the 1970s with robots that were physically isolated from contact with their human co-workers, robots now collaborate with human workers towards a common task goal in a shared workspace. This type of robotic device represents a new era of workplace automation. Industrial robotics is rapidly evolving due to advances in sensor technology, artificial intelligence (AI), wireless communications, mechanical engineering, and materials science. While these new robotic devices are used mainly in manufacturing and warehousing, human-robot collaboration is now seen across multiple goods-producing and service-delivery industry sectors. Assessing and controlling the risks of human-robot collaboration is a critical challenge for occupational safety and health research and practice as industrial robotics becomes a pervasive feature of the future of work. Understanding the physical, psychosocial, work organization, and cybersecurity risks associated with the increasing use of robotic technologies is critical to ensuring the safe development and implementation of industrial robotics. This commentary provides a brief review of the uses of robotic technologies across selected industry sectors; the risks of current and future industrial robotic applications for worker and employer alike; strategies for integrating human-robot collaboration into a health and safety management system; and the role of robotic safety standards in the future of work.

In the food and medical packaging industries, clean packaging is crucial to both customer satisfaction and hygiene. An operational Quality Assurance Department (QAD) is necessary for detecting contaminated packages. Manual examination becomes tedious and may lead to instances of contamination being missed along the production line. To address this issue, a system for contamination detection is proposed using an enhanced deep convolutional neural network (CNN) in a human–robot collaboration framework. The proposed system utilizes a CNN to identify and classify the presence of contaminants on product surfaces. A dataset is generated, and augmentation methods are applied to the dataset for nine classes such as coffee, spot, chocolate, tomato paste, jam, cream, conditioner, shaving cream, and toothpaste contaminants. The experiment was conducted using a mechatronic platform with a camera for contamination detection and a time-of-flight sensor for safe machine–environment interaction. The results of the experiment indicate that the reported system can accurately identify contamination with 99.74% mean average precision (mAP).