Developing a Portable Human–Robot Interaction (HRI) Framework for Outdoor Robots Through Selective Compartmentalization

Incorporating hand gesture recognition in human–robot interaction has the potential to provide a natural way of communication, thus contributing to a more fluid collaboration toward optimizing the efficiency of the application at hand and overcoming possible challenges. A very promising field of interest is agriculture, owing to its complex and dynamic environments. The aim of this study was twofold: (a) to develop a real-time skeleton-based recognition system for five hand gestures using a depth camera and machine learning, and (b) to enable a real-time human–robot interaction framework and test it in different scenarios. For this purpose, six machine learning classifiers were tested, while the Robot Operating System (ROS) software was utilized for “translating” the gestures into five commands to be executed by the robot. Furthermore, the developed system was successfully tested in outdoor experimental sessions that included either one or two persons. In the last case, the robot, based on the recognized gesture, could distinguish which of the two workers required help, follow the “locked” person, stop, return to a target location, or “unlock” them. For the sake of safety, the robot navigated with a preset socially accepted speed while keeping a safe distance in all interactions.

Real world virtual world communication and interaction will be a cornerstone of future intelligent manufacturing ecosystems. Human robotic interaction is considered to be the basic element of factories of the future. Despite the advancement of different technologies such as wearables and Augmented Reality (AR), human-robot interaction (HRI) is still extremely challenging. Whilst progress has been made in the development of different mechanisms to support HRI, there are issues with cost, naturalistic and intuitive interaction, and communication across heterogeneous systems. To mitigate these limitations, RoSTAR is proposed. RoSTAR is a novel open-source HRI system based on the Robot Operating System (ROS) and Augmented Reality. An AR Head Mounted Display (HMD) is deployed. It enables the user to interact and communicate through a ROS powered robotic arm. A model of the robot arm is imported directly into the Unity Game engine, and any interactions with this virtual robotic arm are communicated to the ROS robotic arm. This system has the potential to be used for different process tasks, such as robotic gluing, dispensing and arc welding as part of an interoperable, low cost, portable and naturalistically interactive experience.

Research on Human-Robot Interaction (HRI) requires the substantial consideration of an experimental design, as well as a significant amount of time to practice the subject experiment. Recent technology in virtual reality (VR) can potentially address these time and effort challenges. The significant advantages of VR systems for HRI are: 1) cost reduction, as experimental facilities are not required in a real environment; 2) provision of the same environmental and embodied interaction conditions to test subjects; 3) visualization of arbitrary information and situations that cannot occur in reality, such as playback of past experiences, and 4) ease of access to an immersive and natural interface for robot/avatar teleoperations. Although VR tools with their features have been applied and developed in previous HRI research, all-encompassing tools or frameworks remain unavailable. In particular, the benefits of integration with cloud computing have not been comprehensively considered. Hence, the purpose of this study is to propose a research platform that can comprehensively provide the elements required for HRI research by integrating VR and cloud technologies. To realize a flexible and reusable system, we developed a real-time bridging mechanism between the robot operating system (ROS) and Unity. To confirm the feasibility of the system in a practical HRI scenario, we applied the proposed system to three case studies, including a robot competition named RoboCup@Home. via these case studies, we validated the system’s usefulness and its potential for the development and evaluation of social intelligence via multimodal HRI.

This paper describes novel interdisciplinary approaches for the design and training of cyborg technologies, specifically upper body exoskeletons. EXACT: Exoskeletons, Art and Choreographic Training is a multi-faceted research effort that uses dance performance and experimental trials to study the effects of movement and live performance on exoskeleton training. The goal is to combine research methods from the arts with human robot interaction (HRI) research. The rationale for using ethnographic methods (which privilege qualitative analysis through video data and multimodal interaction analysis) within an HRI framework is to develop nuanced approaches for studying embodiment and techno-corporeality in socially-situated contexts. This investigation has led to the development of new evaluation tools and frameworks for studying human-machine interaction, including human-centred assessments and custom virtual reality tools that allow for fine-grained analysis. An interdisciplinary approach is essential for studying the corporeal experience in human-machine interactions.

Human Robot Interaction (HRI) is one of the biggest research field in the Robotics research world. Understanding the semantic meaning of the user instruction is very important to establish the communication between user and robot. When a user instructs to all heterogeneous service Robots with high level instructions who are working at different locations in smart house, all robots need to operate by understanding the semantic of the instruction and complete the task uniformly without considering underline software and hardware implementations. Ontology is used to extract the semantic meaning of the instructions. Each robot is assigned a specific task for specific time period for each day. User can issue commands to all robots at different time slots but the same command can be issued with different sentence by different users. We have used ontology to represents the semantic of the sentence with different commands. We have used three robots (TurtleBot, TiaGo and Husky) for our experiment in Gazebo simulator. Robot Operating System (ROS) is the middleware which is hiding more complex implementation of different functions for different robots and provide the interoperability for heterogeneous robot operations. The ROS nodes and topics can be different for different robots, therefore our implementation can solve this issue by autonomous robot registration algorithm which is working with Robot ontology.

As Human-Robot Interaction (HRI) develops, robots are expected to learn more complex and demanding interaction skills. Complex HRI tasks are often embodied in robots that are jointly constrained by task space and joint space. In the field of Imitation Learning, scholars have explored the topic of joint constraints from task space and joint space, but there are few relevant studies in Human-Robot Interaction. In this paper, based on the Interaction Primitives framework (a HRI framework), we propose an interaction inference method that first generalizes the robot's movements in two spaces synchronously and then probabilistically fuses the two movements based on Bayesian estimation. This work was validated in the task that the robot follows a human handheld object, and the inference errors (RMSE and MAE) of the method are smaller in both task space and joint space than in IP using only singlespace inference.

In the process of human-robot interaction (HRI), the controleffect cannot meet the needs of HRI tasks if a control strategy is developed solely from the robot's point of view. It's necessary to take into account the characteristics of the human operator. In this letter, a novel HRI framework is developed to realise robot humanoid control by transferring the human arm stiffness and human hand motion characteristics to the robot. Firstly, adaptive impedance control is used to adjust the robot's stiffness through human arm active endpoint stiffness, which is determined by the geometric information and muscle activation of the human arm. In addition, a direction adaptive compensation method is also proposed by introducing a direction compensating quantity to regulate the robot's trajectory for minimising the effect on the robot of changes in the motion direction of the human hand. The proposed method can not only stabilize the process of HRI but also decrease the interaction force to reduce the human operator's workload. Comparative experiments demonstrate the enhanced performance of the proposed method.

The field of human-robot interaction (HRI) covers a wide range of interactions between humans and robots in various settings. In most cases, both the human and the robot should know the status of each other and interchange data accordingly. Particularly, informing the robot about the human poses and gestures is critical for safe and reliable interaction. We present an open-source inertial measurement unit (IMU)-based human motion estimation framework designed for HRI applications which can be used with multiple robotic systems. The presented framework is developed as a Robot Operating System (ROS) package and takes on a bridge role between the human and the robot by utilizing the estimated human motions to produce a desired robot goal pose. Although there are different human motion tracking systems that are used in gaming, film making, industrial and medical applications today, they are either quite costly, tedious to set up or system dependent. Also, those systems have some limitations to be used in industrial environments such as suffering from occlusion and obstacles between the tracker system and the human, sensitive to light changes etc., and require long calibration steps before each use. The availability of a versatile human motion estimation framework that can be easily used in different HRI scenarios is handy for industrial applications. Some examples of usages of the proposed framework both in simulation and real-world applications are demonstrated in this paper. We aim our package to be a useful boosting tool to develop IMU-based human motion estimated HRI applications and auxiliary templates to develop more complex HRI scenarios for research and development purposes.

Towards establishing future-oriented relations between humans and robots, not only verbal, but also nonverbal robot functions have been studied. However, it is not clear how these different functions should be combined and implemented in robots in order to achieve ‘natural’ human-robot communication or interaction. In this paper, we focus on the timing for the coordination between utterance as a verbal function and body motion as a nonverbal function. To begin with, we investigated the temporal order and correlation between utterance and gesture in an interpersonal (human-human) interaction scenario as a preliminary experiment (Experiment 1). Then, we controlled the robot's response timing based on findings of the first experiment and evaluated the effectiveness of the robot's timing control in human-robot interaction (Experiment 2). It was shown that elderly participants significantly preferred the controlled robot's response. In addition, the timing of utterance and gestures affected the impression of the robot and interpretation of the verbal message. These results are important findings for designing socially interactive robots for the aging society.

This paper introduces a new ROSbag-based multimodal affective dataset for emotional and cognitive states generated using the Robot Operating System (ROS). We utilized images and sounds from the International Affective Pictures System (IAPS) and the International Affective Digitized Sounds (IADS) to stimulate targeted emotions (happiness, sadness, anger, fear, surprise, disgust, and neutral), and a dual N-back game to stimulate different levels of cognitive workload. 30 human subjects participated in the user study; their physiological data were collected using the latest commercial wearable sensors, behavioral data were collected using hardware devices such as cameras, and subjective assessments were carried out through questionnaires. All data were stored in single ROSbag files rather than in conventional Comma-Separated Values (CSV) files. This not only ensures synchronization of signals and videos in a data set, but also allows researchers to easily analyze and verify their algorithms by connecting directly to this dataset through ROS. The generated affective dataset consists of 1,602 ROSbag files, and the size of the dataset is about 787GB. The dataset is made publicly available. We expect that our dataset can be a great resource for many researchers in the fields of affective computing, Human-Computer Interaction (HCI), and Human-Robot Interaction (HRI).

Integrating real-time, complex social signal processing into robotic systems – especially in real-world, multi-party interaction situations – is a challenge faced by many in the Human-Robot Interaction (HRI) community. The difficulty is compounded by the lack of any standard model for human representation that would facilitate the development and interoperability of social perception components and pipelines. We introduce in this paper a set of conventions and standard interfaces for HRI scenarios, designed to be used with the Robot Operating System (ROS). It directly aims at promoting interoperability and re-usability of core functionality between the many HRI-related software tools, from skeleton tracking, to face recognition, to natural language processing. Importantly, these interfaces are designed to be relevant to a broad range of HRI applications, from high-level crowd simulation, to group-level social interaction modelling, to detailed modelling of human kinematics. We demonstrate these interfaces by providing a reference pipeline implementation, packaged to be easily downloaded and evaluated by the community.

Actually, the research on human interactive robot (HIR) has been a topic of both science fiction and academic speculation for a long time. The origin of HIR as a discrete issue was stated by 20th century author Isaac Asimov in 1941, in his novel “I, Robot”. He stated the Three Laws of Robotics1 as, “ a. A robot may not injure a human being or, through inaction, allow a human being to come to harm. b. A robot must obey any orders given to it by human beings, except where such orders would conflict with the First Law. c. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. ” The three laws of robotics determine the idea of safe interaction which constitutes the basic rules of HIR. With the advances of artificial intelligence (AI), the HIR could eventually have more proactive behaviours, planning their motion in complex unknown environments. Nowadays, HIR are artificial agents with capacities of perception and action in the human’s environment. Their use has been tended to be found in the most technologically advanced societies in critical domains as search and rescue, military battle, law enforcement, entertainment, hospital care, etc. These domains of applications imply a closer interaction with human. The concept of closeness is to be taken in its full meaning, HIR and humans not only share the workspace but also share goals in terms of task achievement. The HIR has to adapt itself to human’s way of expressing desires and fulfill its task. Taking lifting up human in elder care for example, the human interactive robot RI-MAN, designed by the RIKEN Bio-Mimetic Control Research Center, communicates with human by listening and speaking, which makes it understand the human will (Onish, Luo et al. 2007). To fulfil the task, it also estimates the attitude of human body in real-time by tactile sense (Mukai, Onishi et al. 2008). This example contains two aspects of HIR, one is to understand the human mind and the other is to accomplish the manipulation. The former is based on AI techniques, like language comprehension, and the latter is relied on force control. On the other hand, human’s environments are much more complex. Thus, the HIR needs perceiving and understanding capacities to build dynamic models of its surroundings. It

Nowadays, there is an increasing demand for underwater intervention systems around the world in several application domains. The commercially available systems are far from what is demanded in many aspects, justifying the need of more autonomous, cheap and easy-to-use solutions for underwater intervention missions. The chapter begins making a review of the most important research projects that have been able to demonstrate some results in sea conditions. Then, the expertise and know-how developed in the context of our research group in the last years is presented. Maybe, one of the main achieved results, from the methodological point of view, is a three-layer general system architecture based on the Robot Operating System (ROS), which allows an underwater vehicle to perform intervention missions with a high degree of autonomy, independently of the targeted scenario. Moreover, the use of an underwater simulator as a 3D simulation tool for benchmarking and Human Robot Interaction (HRI) is also discussed. In summary, a methodology has been developed for experimental validation, independently of the specific underwater intervention problem to solve. It consists on the use of the simulator, as a prior step before moving to any of the testbeds used for experimental validation. The reliability and feasibility of this methodology has been demonstrated for intervention missions in sea trial conditions.KeywordsUnderwater robotROSMission planningGrasping and manipulation planning

A wearable sensor vest for social humanoid robots with GPGPU, IoT, and modular software architecture

The Robot Operating System (ROS) is a popular framework and ecosystem that allows developers to build robot software systems from reusable, off-the-shelf components. Systems are often built by customizing and connecting components via configuration files. While reusable components theoretically allow rapid prototyping, ensuring proper configuration and connection is challenging, as evidenced by numerous questions on developer forums. Developers must abide to the often unchecked and unstated assumptions of individual components. Failure to do so can result in misconfigurations that are only discovered during field deployment, at which point errors may lead to unpredictable and dangerous behavior. Despite misconfigurations having been studied in the broader context of software engineering, robotics software (and ROS in particular) poses domain-specific challenges with potentially disastrous consequences. To understand and improve the reliability of ROS projects, it is critical to identify the types of misconfigurations faced by developers. To that end, we perform a study of ROS Answers, a Q&A platform, to identify and categorize misconfigurations that occur during ROS development. We then conduct a literature review to assess the coverage of these misconfigurations by existing detection techniques. In total, we find 12 high-level categories and 50 sub-categories of misconfigurations. Of these categories, 27 are not covered by existing techniques. To conclude, we discuss how to tackle those misconfigurations in future work.