As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually programming their behavior has become increasingly challenging and expensive. Often, it is easier for a teacher to demonstrate a desired behavior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning. This work provides an introduction to imitation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the problem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to familiarize machine learning experts with the challenges of imitation learning, particularly those arising in robotics, and the interesting theoretical and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial intelligence a broader appreciation for the frameworks and tools available for imitation learning. We organize our work by dividing imitation learning into directly replicating desired behavior (sometimes called behavioral cloning [Bain and Sammut, 1996]) and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control [Kalman, 1964] or inverse reinforcement learning [Russell, 1998]). In addition to method analysis, we discuss the design decisions a practitioner must make when selecting an imitation learning approach. Moreover, application examples—such as robots that play table tennis Kober and Peters, 2009 and programs that play the game of Go Silver et al. 2016—illustrate the properties and motivations behind different forms of imitation learning. We conclude by presenting a set of open questions and point towards possible future research directions.

This monograph surveys and informs the design and usage of nonverbal signals for human-robot interaction. With robots increasingly being utilized for tasks that require them to not only operate in close proximity to humans but to interact with them as well, there has been great interest in the communication challenges associated with the varying degrees of interaction in these environments. The success of such interactions depends on robots’ ability to convey information about their knowledge, intent, and actions to co-located humans. The monograph presents a comprehensive review of literature related to the generation and usage of nonverbal signals that facilitate legibility of non-humanoid robot state and behavior. To motivate the need for these signaling behaviors, it surveys literature in human communication and psychology and outlines target use cases of non-humanoid robots. Specifically, the focus is on works that provide insight into the cognitive processes that enable humans to recognize, interpret, and exploit nonverbal signals. From these use cases, information is identified that is potentially important for non-humanoid robots to signal and organize it into three categories of robot state. The monograph then presents a review of signal design techniques to illustrate how signals conveying this information can be generated and utilized. It concludes by discussing issues that must be considered during nonverbal signaling and open research areas, with a focus on informing the design and usage of generalizable nonverbal signaling behaviors for task-oriented non-humanoid robots.

The last decade has witnessed an increasing interest in the more active use of soft materials in robotic systems. Having a soft body like the ones in biological systems can potentially provide a robot with superior capabilities. For example, soft body can help the robots to adaptively navigate through small openings, softness can prevent injuries in human-robot interaction, while deformable body can also store and release energy, which may lead to energy effciency in locomotion tasks. By building robotic systems with soft materials, we can realize systems that are safer, cheaper, and more adaptable than the level that conventional rigid-material robots can achieve. Soft-Material Robotics introduces the fundamental aspects of this vibrant research topic from history, modelling, control, and system integration. This content sets it apart from previous review efforts that have focused on presenting individual case studies.